Episode 5: Airflow, DAG and Automation Efforts

Weekly Highlights

1. Machine Learning Course
2. Multi-touch Attribution Model
3. Interview
4. People who inspired me this week

Machine Learning Course

I'm in the final stretch of my Google Cloud Machine Learning Certification, and this week I've been diving into the last module on Machine Learning Operations (MLOps).

This section covers the core technologies and best practices needed to support effective MLOps, including:

• Adopting CI/CD for ML systems.
• Configuring Google Cloud architectures for reliable MLOps environments.
• Implementing repeatable training and inference workflows.

The module goes into detail on tools like TFX,Cloud Composer, and Airflow.

This week was a bit slower for me. I haven't been feeling my best, and with everything happening in Nepal, it's been hard to concentrate. Still, I'm trying my best to push through. I can't wait to finish this final module!

Multi-touch Attribution Model

Last week, I finalized my data model, machine learning model, and results, and added all the charts and graphs to my React app. This week, my focus has shifted to a completely new and challenging territory: automating my entire workflow.

My experience with automation was limited to scheduling a Dataform refresh to update tables. Now, I need to figure out how to automatically trigger my Python ML model script, process its results back in Dataform,React front end.

Honestly, this has been an extremely complicated process, and I haven't successfully built a working automation yet. Here's a look at what I've tried so far:

Initial Attempts with Cloud Composer and Airflow:

• I set up a Cloud Composer instance to access Airflow and, with the help of an LLM, wrote my first DAG (Directed Acyclic Graph).
• My initial plan was to put my entire workflow into a single DAG, but this was a mistake. My DAG failed on the very first node. I realized that refreshing the Dataform tables could be done more simply with Dataform's own release and scheduling features, making Airflow an unnecessary overkill for that step.
• Troubleshooting in Cloud Composer was difficult. The process of uploading the DAG file and waiting for it to update was slow and tedious.

The Pivot to Docker:

• I decided to research a way to have more control over the process by running it locally. This led me to a new friend: Docker.
• I dove into learning Docker, setting up image files, and creating a local Airflow instance. This time, I scaled back my goal. Instead of automating the entire workflow at once, I focused on a single step: getting my Python script to run automatically.
• However, I am currently stuck. I'm facing a nightmare of dependency issues and Python interpreter problems. Troubleshooting these types of issues has proven to be incredibly difficult.

My effort is ongoing, and I'm hopeful that I'll be able to successfully automate my workflow soon. This journey has been a testament to the steep learning curve of MLOps.

Interview

This week, I had a technical interview for a Product Analyst position. My experience has been primarily in data analysis for business and e-commerce, so this role was a new challenge. To prepare, I spent a lot of time studying the basics of the position.

I quickly realized that a fundamental skill for this job is A/B testing. I dove deep into how A/B tests are carried out, from the core concepts of p-values, t-tests, and z-tests to the practical application of these methods in a business setting.

My most valuable resource was a paper titled "Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing," which offers insights from leaders at Google, LinkedIn, and Microsoft. Here are the most important takeaways from my study:

1. A/B Testing is the "Gold Standard" for Causality

It's easy to confuse correlation with causality. A/B tests solve this by randomly splitting users into different groups—a "Control" group that sees the existing product and a "Treatment" group that sees a new feature. This allows you to reliably establish a cause-and-effect relationship between a change and its impact.

2. It's Not Just About Big Wins

While A/B tests can uncover unexpected breakthroughs (like a simple UI tweak that earns millions), the most consistent progress comes from many small, incremental changes. Companies like Google and Bing achieve their goals by running thousands of these experiments, each one contributing to a larger win.

3. Define Your Metrics and Guardrails

Before running a test, you must define an Overall Evaluation Criterion (OEC)—a single, clear metric that measures the experiment's objective. Equally important are Guardrail Metrics, which ensure a new feature doesn't negatively impact other critical areas, like site performance or error rates.

4. Acknowledge Human Fallibility

A crucial lesson from these industry leaders is that our intuition is often wrong. Companies have found that a high percentage (70-90%) of new ideas and experiments do not actually improve key metrics. This is why a data-driven approach is essential, replacing the "Highest Paid Person's Opinion" (HiPPO) with concrete evidence.

5. Speed Matters

The research repeatedly shows that even minor performance improvements can have a huge impact. For example, a 100-millisecond slowdown on a website can lead to a significant drop in sales and engagement.

In summary, my biggest takeaway is that A/B testing is a powerful tool for driving innovation, but it requires careful design, an organizational commitment to data, and a willingness to be proven wrong. I feel much more confident about my understanding of this critical part of product analysis now.

People who inspired me this week

1. Economic Loss After GenZ Protest | Nepal’s Financial Challenges
2. Gen-Z Revolution vs Nepal Government
3. FINALLY POLICE STATION RESTORED GARYOYOOO

That’s all for this week. See you next week!