⚡

Projects

A selection of data engineering & ML projects that I've built — from scalable pipelines to intelligent systems.

Paintings & Museum Analytics — SQL, PySpark & Pandas

Solved 20+ competitive data modeling challenges using SQL, PySpark, and Pandas. Built JOIN-heavy queries, window functions, and aggregation pipelines on a multi-table museum dataset — reducing query execution time by 35% through indexing and DataFrame caching.

SQLPySparkPandasPython

Data Engineering Toolkit — SQL vs PySpark vs Pandas

A comprehensive side-by-side guide covering SELECT, filter, aggregation, string ops, joins, window functions, and data cleaning across SQL, PySpark, and Pandas. Used by data engineering teams as an onboarding reference.

SQLPySparkPandasPython

Azure End-to-End Data Pipeline — IPL Analytics

Built a production-grade Azure Medallion Architecture pipeline ingesting IPL cricket data from GitHub via Azure Data Factory into Blob Storage, transforming it through Bronze → Silver → Gold layers in Databricks using PySpark and Delta Lake, then surfacing insights in SQL Analytics.

Azure Data FactoryDatabricksPySparkDelta Lake+1

Real-Time Streaming Pipeline with Kafka & Spark

Designed and deployed a real-time event streaming pipeline using Apache Kafka for message brokering and Spark Structured Streaming for micro-batch processing. Streams transaction events into Delta Lake with exactly-once semantics and sub-second latency.

Apache KafkaSpark StreamingDelta LakePython+1

ML Churn Prediction API with FastAPI & PySpark

Built an end-to-end ML pipeline for customer churn prediction: PySpark for feature engineering on 1M+ rows, scikit-learn for model training (XGBoost, 89% accuracy), and a FastAPI REST API serving real-time predictions with sub-100ms response time.

PySparkFastAPIscikit-learnPython+1

IPL Data Science — Exploratory Analysis & Predictions

Deep exploratory data analysis on IPL cricket dataset (2008–2023) covering 900+ matches. Built interactive visualisations, identified key performance drivers, and trained predictive models for match outcome and player performance ratings.

PythonPandasMatplotlibscikit-learn+1