Nitin Magdum

Nitin Magdum

Data Engineer

--:--:--

...

๐Ÿ’ผ

Work History

I'm a Data Engineer that loves building scalable data systems and pipelines that power products impacting millions.

๐Ÿ’ผ

Celebal Technologies

Senior Data Engineer

Jun 2025 โ€” Present
  • Utilized PySpark to process and analyze large datasets efficiently, enabling faster data processing and improved scalability.
  • Developed an automated data pipeline to streamline data ingestion from multiple sources.
  • Performed extensive data cleaning to ensure accuracy and integrity, addressing missing values and inconsistencies.
  • Utilized statistical methods to analyze datasets, identifying trends and patterns that inform business strategies.
  • Implemented a real-time analytics engine to provide instant insights and improve decision-making.
  • Enhanced ETL processes for faster data transformation, reducing processing time by 40%.
  • Built a monitoring system to track data pipeline performance and ensure data integrity in real-time.
  • Developed scalable data pipelines using Databricks to manage and analyze massive datasets, enhancing data-driven decision-making in real-time.
  • Designed and implemented end-to-end data pipelines in Databricks using PySpark and Delta Lake, following Medallion Architecture to structure raw, refined, and curated data layers.
  • Developed batch and Spark Structured Streaming jobs integrated with Databricks Workflows, enabling automated and reliable data processing.
  • Built and maintained a scalable data warehouse, delivering high-quality, analytics-ready data to downstream teams.
๐Ÿš€

Ingenius Technologies

Data Engineer

May 2023 โ€” Jun 2025
  • Utilized PySpark to process and analyze large datasets efficiently, enabling faster data processing and improved scalability.
  • Developed an automated data pipeline to streamline data ingestion from multiple sources.
  • Performed extensive data cleaning to ensure accuracy and integrity, addressing missing values and inconsistencies.
  • Utilized statistical methods to analyze datasets, identifying trends and patterns that inform business strategies.
  • Implemented a real-time analytics engine to provide instant insights and improve decision-making.
  • Enhanced ETL processes for faster data transformation, reducing processing time by 40%.
  • Built a monitoring system to track data pipeline performance and ensure data integrity in real-time.
  • Developed scalable data pipelines using Databricks to manage and analyze massive datasets, enhancing data-driven decision-making in real-time.
  • Built and managed scalable data pipelines end-to-end using Databricks with PySpark, leveraging Delta Lake for ACID-compliant data storage and efficient big data processingโ€”improving overall pipeline performance by 40%.
  • Worked extensively with Unity Catalog to centralize data governance, manage access controls, and ensure consistent data lineage across Databricks workspaces.
  • Developed and orchestrated ETL/ELT workflows using Databricks Workflows and Jobs, integrating with Azure Data Factory (ADF) for enterprise-grade scheduling and monitoring.
  • Implemented auto-scaling and cluster optimization strategies in Databricks, reducing compute costs by $5K/month while maintaining performance at scale.
  • Integrated Spark Structured Streaming into Databricks pipelines for real-time ingestion and processing of incremental data loads.
  • Developed batch ETL pipelines in Databricks using PySpark and Delta Lake, implementing CDC and SCD Types 0, 1, and 2 to load and maintain a curated data warehouse for accurate historical and real-time reporting.
  • Automated pipeline deployment using notebooks and parameterized workflows, improving release cycle speed and reducing manual intervention by 70%.
  • Built scalable batch data pipelines in Databricks using PySpark, Delta Lake, and Medallion Architecture (Bronze, Silver, Gold), orchestrated through Databricks Workflows for automated and reliable processing.
  • Enforced data quality and governance practices using Delta Lake features like time travel, schema enforcement, and audit logs via Databricks SQL Analytics, ensuring robust data management and compliance.
โš™๏ธ

Agiliad Technologies

Data Engineer

Oct 2022 โ€” Mar 2023
  • Leveraged Pandas for data manipulation and transformation, streamlining workflows and enhancing data analysis capabilities.
  • Developed complex SQL queries to extract, transform, and load data from multiple relational databases, ensuring optimized performance and accuracy.
  • Implemented Azure Data Factory (ADF) pipelines to orchestrate data movement and transformation across various cloud and on-premise sources.
  • Utilized Azure Synapse Analytics for large-scale data warehousing, enabling seamless integration with Power BI for advanced reporting and analytics.
๐ŸŽ“

Education

Strong foundation in Engineering since childhood, excelling academically to build solutions that matter.

๐ŸŽ“

DKTE Society's Textile and Engineering Institute, Ichalkaranji

B.Tech in Electronics Engineering

2019 โ€” 2022
  • Applied Mathematics III, Electronic Devices and Circuits, Signals and Systems, Control Systems, Microprocessors, Electromagnetic Fields.
  • Applied Mathematics IV, Digital Signal Processing, Communication Systems, Linear Integrated Circuits, Power Electronics, Network Analysis.
  • VLSI Design, Embedded Systems, Microcontrollers, Data Communication, Project Management.
  • Industrial Electronics, Software Engineering, Communication Networks, Control System Design, Technical Communication.
  • Advanced Digital Signal Processing, Wireless Communication, Robotics, Power System Engineering, Research Methodology, Machine Learning.
  • Project Work, Professional Ethics, Deep Learning, Seminar, Entrepreneurship Development.
๐Ÿ“œ

DKTE Society's Yashwantrao Chavan Polytechnic, Ichalkaranji

Diploma in Electrical Engineering

2015 โ€” 2018
  • Covers foundational concepts in mathematics, physics, and chemistry, along with engineering drawing and basic electrical engineering.
  • Focuses on advanced mathematics and materials for electrical engineering, with practical skills in electronics and workshop practices.
  • Explores advanced mathematics, electrical machines, control systems, and both analog and digital electronics.
  • Introduces advanced mathematics and a deeper understanding of electrical machines, power electronics, and power systems.
  • Emphasizes advanced mathematics, electrical drives, system design, and renewable energy technologies.
  • Focuses on applied mathematics in industrial contexts, advanced control systems, and project work.