Problem
- One Selenium script mixed data fetching, image rendering and Slack sending, making changes and re-runs hard
- Dozens of operational metrics made it hard for the business to read the key trends quickly
Company
CX Real-time Uploader & Ops Dashboard
Operational reporting re-architected from one monolithic script into Airflow DAGs
Period | 2026.02
A real-time uploader with fetching, rendering and sending tangled in one script
-
Metrics
BeforeDozens of metrics pulled and compared each time
AfterDecisions from 4 core charts
-
Structure
BeforeOne script mixing fetching, rendering and sending
AfterTask-level Airflow DAGs published every 10 minutes
My Role
- Split the logic into modules and Airflow DAGs; rendered 4 order charts, city panels and delivery-partner toggle alerts in parallel
- Defined artifact lifecycle and re-run scenarios with Google Drive temp/queue folders and failure cleanup
- Cleaned up BW history schema and region data; designed Superset virtual datasets, filters and charts
Result
- Automatic publishing every 10 minutes
- Dozens of metrics reduced to 4 core charts so the business could decide faster
Technical Decisions
Split into modules and Airflow DAGs
Logic tangled in one script was hard to change and re-run; tasks now fail and retry independently
Running in Production
- Google Drive temp/queue folders and cleanup steps define artifact lifecycle and re-run scenarios
