๐ผ
Work History
I'm a Data Engineer that loves building scalable data systems and pipelines that power products impacting millions.
๐ผJun 2025 โ Present
Celebal Technologies
Senior Data Engineer
- Utilized PySpark to process and analyze large datasets efficiently, enabling faster data processing and improved scalability.
- Developed an automated data pipeline to streamline data ingestion from multiple sources.
- Performed extensive data cleaning to ensure accuracy and integrity, addressing missing values and inconsistencies.
- Utilized statistical methods to analyze datasets, identifying trends and patterns that inform business strategies.
- Implemented a real-time analytics engine to provide instant insights and improve decision-making.
- Enhanced ETL processes for faster data transformation, reducing processing time by 40%.
- Built a monitoring system to track data pipeline performance and ensure data integrity in real-time.
- Developed scalable data pipelines using Databricks to manage and analyze massive datasets, enhancing data-driven decision-making in real-time.
- Designed and implemented end-to-end data pipelines in Databricks using PySpark and Delta Lake, following Medallion Architecture to structure raw, refined, and curated data layers.
- Developed batch and Spark Structured Streaming jobs integrated with Databricks Workflows, enabling automated and reliable data processing.
- Built and maintained a scalable data warehouse, delivering high-quality, analytics-ready data to downstream teams.
๐May 2023 โ Jun 2025
Ingenius Technologies
Data Engineer
- Utilized PySpark to process and analyze large datasets efficiently, enabling faster data processing and improved scalability.
- Developed an automated data pipeline to streamline data ingestion from multiple sources.
- Performed extensive data cleaning to ensure accuracy and integrity, addressing missing values and inconsistencies.
- Utilized statistical methods to analyze datasets, identifying trends and patterns that inform business strategies.
- Implemented a real-time analytics engine to provide instant insights and improve decision-making.
- Enhanced ETL processes for faster data transformation, reducing processing time by 40%.
- Built a monitoring system to track data pipeline performance and ensure data integrity in real-time.
- Developed scalable data pipelines using Databricks to manage and analyze massive datasets, enhancing data-driven decision-making in real-time.
- Built and managed scalable data pipelines end-to-end using Databricks with PySpark, leveraging Delta Lake for ACID-compliant data storage and efficient big data processingโimproving overall pipeline performance by 40%.
- Worked extensively with Unity Catalog to centralize data governance, manage access controls, and ensure consistent data lineage across Databricks workspaces.
- Developed and orchestrated ETL/ELT workflows using Databricks Workflows and Jobs, integrating with Azure Data Factory (ADF) for enterprise-grade scheduling and monitoring.
- Implemented auto-scaling and cluster optimization strategies in Databricks, reducing compute costs by $5K/month while maintaining performance at scale.
- Integrated Spark Structured Streaming into Databricks pipelines for real-time ingestion and processing of incremental data loads.
- Developed batch ETL pipelines in Databricks using PySpark and Delta Lake, implementing CDC and SCD Types 0, 1, and 2 to load and maintain a curated data warehouse for accurate historical and real-time reporting.
- Automated pipeline deployment using notebooks and parameterized workflows, improving release cycle speed and reducing manual intervention by 70%.
- Built scalable batch data pipelines in Databricks using PySpark, Delta Lake, and Medallion Architecture (Bronze, Silver, Gold), orchestrated through Databricks Workflows for automated and reliable processing.
- Enforced data quality and governance practices using Delta Lake features like time travel, schema enforcement, and audit logs via Databricks SQL Analytics, ensuring robust data management and compliance.
โ๏ธOct 2022 โ Mar 2023
Agiliad Technologies
Data Engineer
- Leveraged Pandas for data manipulation and transformation, streamlining workflows and enhancing data analysis capabilities.
- Developed complex SQL queries to extract, transform, and load data from multiple relational databases, ensuring optimized performance and accuracy.
- Implemented Azure Data Factory (ADF) pipelines to orchestrate data movement and transformation across various cloud and on-premise sources.
- Utilized Azure Synapse Analytics for large-scale data warehousing, enabling seamless integration with Power BI for advanced reporting and analytics.
๐
Education
Strong foundation in Engineering since childhood, excelling academically to build solutions that matter.
๐2019 โ 2022
DKTE Society's Textile and Engineering Institute, Ichalkaranji
B.Tech in Electronics Engineering
- Applied Mathematics III, Electronic Devices and Circuits, Signals and Systems, Control Systems, Microprocessors, Electromagnetic Fields.
- Applied Mathematics IV, Digital Signal Processing, Communication Systems, Linear Integrated Circuits, Power Electronics, Network Analysis.
- VLSI Design, Embedded Systems, Microcontrollers, Data Communication, Project Management.
- Industrial Electronics, Software Engineering, Communication Networks, Control System Design, Technical Communication.
- Advanced Digital Signal Processing, Wireless Communication, Robotics, Power System Engineering, Research Methodology, Machine Learning.
- Project Work, Professional Ethics, Deep Learning, Seminar, Entrepreneurship Development.
๐2015 โ 2018
DKTE Society's Yashwantrao Chavan Polytechnic, Ichalkaranji
Diploma in Electrical Engineering
- Covers foundational concepts in mathematics, physics, and chemistry, along with engineering drawing and basic electrical engineering.
- Focuses on advanced mathematics and materials for electrical engineering, with practical skills in electronics and workshop practices.
- Explores advanced mathematics, electrical machines, control systems, and both analog and digital electronics.
- Introduces advanced mathematics and a deeper understanding of electrical machines, power electronics, and power systems.
- Emphasizes advanced mathematics, electrical drives, system design, and renewable energy technologies.
- Focuses on applied mathematics in industrial contexts, advanced control systems, and project work.