Data Engineer with 2+ years of expertise in designing, building, and monitoring scalable, cloud-native ELT/ETL pipelines and robust data warehouse solutions. Leveraging AWS (S3, Glue, Redshift, Airflow), Apache Spark, Python, and SQL, I have a proven track record in optimizing data workflows for performance, cost-efficiency, and enabling data-driven decision-making, capable of reliably handling large data volumes.
Data Engineer
Hyderabad, Telangana, India
→
Summary
Currently serving as a Data Engineer, I design, deploy, and optimize scalable ELT pipelines and data warehouse solutions to support critical business intelligence and data-driven decision-making.
Highlights
Analyzed complex source data in-depth to define business logic, column relationships, and data dependencies, ensuring precise data mapping and robust pipeline development.
Engineered and deployed end-to-end ELT pipelines using AWS Managed Airflow, Glue, S3, and dbt Core, processing over 2 million records daily for efficient data ingestion and cost-effective transformation into Amazon Redshift.
Optimized data loading processes by implementing incremental logic in AWS Glue using CDC timestamps, resulting in a 30% reduction in compute costs and accelerating processing time by 66% using incremental dbt models.
Automated data pipeline orchestration with Airflow, designing and implementing scheduling capabilities that reduced manual intervention by 80% and streamlined workflow efficiency.
Constructed comprehensive metadata tables to monitor pipeline health, facilitate rapid failure recovery, and ensure end-to-end data integrity.
Mentored 40 junior engineers in SQL and AWS cloud fundamentals, enhancing foundational skills through structured training, interactive exercises, and rigorous evaluations.
Software Development Intern
Remote, N/A, India
→
Summary
As a Software Development Intern, I contributed to the development of data processing tools and a web application for optimizing trade logistics.
Highlights
Scraped and processed HSN data from customs tariff websites using WebHarvey and Excel, ensuring 100% data accuracy for trade analysis.
Engineered an algorithm utilizing the Haversine Distance formula to accurately identify the nearest seaport for any given location, optimizing logistics planning.
Developed a dynamic web application using HTML, CSS, Bootstrap, and Google Maps API to visually represent optimal trade routes.
Integrated the core algorithm with a Flask RESTful API, establishing seamless frontend-backend communication via JavaScript Fetch API.
Student Mentor
→
Summary
Provided mentorship and guidance to first-year students, fostering their academic and personal development within the IIT KGP community.
Highlights
Mentored four first-year students for two years, offering comprehensive academic guidance and personal support to aid their transition and growth.
→
B. Tech and M. Tech (Dual Degree)
Chemical Engineering
Grade: CGPA: 8.14
Awarded By
IIT Kharagpur
Led a 200+ member team and managed a 2.6 Lakh INR budget to secure 3rd place in IIT KGP's Illumination contest, showcasing strong leadership and project execution skills.
Awarded By
Prince Education, Sikar
Achieved 3rd rank among over 40,000+ candidates in the Prince Talent Reward Exam, demonstrating exceptional academic aptitude.
Awarded By
Karnataka Colleges Foundation
Secured All India Rank 252 out of 70,000+ candidates in the highly competitive COMEDK UGET examination.
Issued By
AWS Training and Certification
Issued By
HackerRank
Python, SQL, C++, JavaScript, HTML, CSS.
dbt, Airflow, PostgreSQL, MySQL, ETL Pipelines, Data Modelling, Data Warehousing.
S3, Redshift, Glue, EC2, Lambda, RDS, DynamoDB, CloudWatch, IAM, SNS, SES.
Apache Spark, Databricks, Delta Lake, DLT, HDFS, MapReduce.
→
Summary
Led a large team and managed a significant budget to conceptualize, design, and execute a creative project for the annual Illumination contest at IIT Kharagpur, demonstrating strong leadership and project management capabilities.