About
Results-driven Data Engineer with 2 years of experience designing, implementing, and optimizing robust ELT pipelines and data warehousing solutions. Proven ability to leverage PySpark, SQL, and AWS services (S3, Redshift, Airflow) to process millions of records daily, deliver critical business intelligence, and drive data-driven efficiency for organizational empowerment.
Work
NucleusTeq
|Data Engineer II
Indore, Madhya Pradesh, India
→
Summary
As Data Engineer II, implemented and optimized ELT pipelines and Databricks workflows to process millions of records daily, delivering critical enterprise tables for business intelligence and maintaining robust data flow.
Highlights
Engineered and deployed ELT pipelines, ingesting over 1 million records daily from diverse sources (SOAP APIs, S3 feeds, JDBC) into AWS S3 using XML and Parquet formats.
Developed and optimized Databricks workflows to transform raw data into a structured Bronze, Silver, and Gold medallion architecture within the Databricks catalog, enhancing data processing efficiency.
Consolidated and integrated data from multiple sources utilizing PySpark and SQL to create and deliver critical enterprise tables, supporting robust reporting and analytics for business stakeholders.
Proactively monitored and maintained Databricks jobs, swiftly resolving data flow issues to ensure consistent data availability and integrity.
NucleusTeq
|Data Engineer I
Indore, Madhya Pradesh, India
→
Summary
As Data Engineer I, designed and maintained scalable ELT pipelines using Apache Airflow, automating data ingestion and transformation into Amazon Redshift while improving query performance and reducing data latency.
Highlights
Designed and maintained scalable ELT pipelines utilizing Apache Airflow to automate the ingestion and transformation of diverse datasets from various sources into Amazon Redshift.
Developed complex SQL queries and optimized Redshift stored procedures, significantly improving query performance and reducing data latency for critical reporting and analytics.
Leveraged AWS services including S3, Athena, and Redshift to construct and monitor robust, cloud-based data workflows, ensuring high reliability and availability.
Collaborated effectively with cross-functional teams to align data strategies with overarching business objectives, ensuring seamless integration and efficient data workflow execution.
Education
Lakshmi Narain College of Technology University
→
Master
Computer Application
Government Holkar Science College
→
Bachelor
Computer Science
Languages
English
Certificates
AWS Cloud Practitioner
Issued By
AWS
Cloud Foundation
Issued By
Boomi
Skills
Data Engineering
Data Pipelines, ETL, Big Data, Data Warehousing, Cloud.
Programming Languages
Python, SQL, Java, OOPs.
Big Data Technologies
PySpark, Spark SQL, Databricks.
Cloud Platforms & Services
AWS S3, Amazon Redshift, Apache Airflow, AWS Athena.
Version Control & Tools
Git, GitHub, VS Code, Jira, Confluence.