The Accidental Data Engineer: How to Transition from Software Dev to Data Pipelines
It usually starts with a innocent-sounding request from a product manager or a tech lead.
“Hey, we need to export our user activity logs to an S3 bucket so the analytics team can run some reports. Can you whip up a quick script for that?”
As a backend or full-stack software engineer, you write a quick Python script, set up a cron job, and call it a day. But a few weeks later, the...