Data Engineering Lead Analyst

Position Overview

Evernorth Core Platform Engineering team is looking for a Data Engineer to design, develop, and implement robust data pipelines for Packaged Business Capabilities (PBCs). You'll work closely with data scientists, analysts, and other engineers to ensure high data quality, performance, and availability.

The ideal candidate brings hands-on experience working with big data tools and platforms such as Apache Spark and Databricks, has a strong command of Python, and is comfortable managing ETL workflows, DAGs, and orchestration using tools like Jenkins or Airflow. Experience in cloud-native environments—particularly AWS and AWS Glue—is essential.

Key  Responsibilities: 

< >Design, implement, and optimize data pipelines using Apache Spark and Databricks to ingest, process, and transform large-scale structured and unstructured datasets.Develop, schedule, and monitor ETL workflows and DAGs using orchestration tools such as Airflow, Jenkins, or similar.Build and maintain data models, tables, and schemas optimized for analytics and reporting in cloud-based data warehouses.Write clean, reusable, and modular code in Python, following best practices and standards.Use AWS Glue to manage metadata, data cataloging, and ETL transformation jobs.Collaborate with analytics and product teams to understand data needs and translate them into engineering solutions.Ensure data quality, reliability, and observability through unit testing, logging, alerting, and documentation.Support version control and CI/CD practices for data pipelines.Participate in code reviews and contribute to team knowledge sharing. 

Required Qualifications

< >3–5 years of experience in data engineering or related field.Strong experience working with Apache Spark in distributed data processing.Hands-on experience with Databricks and working in collaborative notebooks.Proficient in Python for data processing and scripting.Experience building and managing ETL pipelines and DAG workflows using orchestration tools (e.g., Jenkins, Airflow, Luigi).Solid understanding of cloud services, particularly AWS Glue, S3, Lambda, and Athena.Experience working with structured and semi-structured data (e.g., JSON, Parquet, Delta Lake).Familiarity with CI/CD practices and tools (Git, Jenkins, etc.).Comfortable with SQL and relational database systems (e.g., PostgreSQL).Excellent problem-solving skills and attention to detail.Strong communication and collaboration abilities. 

Desired Experience:  

< >Experience with Delta Lake, Unity Catalog, or Medallion architecture on Databricks.Experience with monitoring and logging tools (e.g., CloudWatch).Familiarity with data governance, access controls, and compliance standards.Prior experience working in Agile/Scrum development teams.  

Location & Hours of Work

< >Full-time position, working 40 hours per week. Expected overlap with US hours as appropriate.Primarily based in the Innovation Hub in Hyderabad, India, with flexibility to work remotely as required. 

Equal Opportunity Statement

Evernorth is an Equal Opportunity Employer actively encouraging and supporting organization-wide involvement of staff in diversity, equity, and inclusion efforts to educate, inform and advance both internal practices and external work with diverse client populations.

About Evernorth Health Services