Databricks / AWS Lead Software Engineer

Location: Columbus, OH, United States

Job Family: Software Engineering

About the Role

At JP Morgan Chase, we are at the forefront of financial services innovation, and our HR Data team in Columbus, OH plays a pivotal role in leveraging data to drive talent strategies and operational excellence. As a Databricks / AWS Lead Software Engineer, you will lead the development of cutting-edge data solutions that empower HR decision-making across our global organization. This role involves architecting scalable platforms on Databricks and AWS to process vast datasets related to employee analytics, diversity metrics, and compliance reporting, ensuring alignment with the firm's rigorous regulatory standards in the financial industry. You will collaborate with data scientists, HR business partners, and compliance teams to deliver insights that support JP Morgan's commitment to inclusive growth and risk management. In this leadership position, you will spearhead initiatives to build robust data pipelines using Spark, Delta Lake, and AWS services like S3 and EMR, optimizing for performance and cost in a cloud-native environment. Your expertise will be crucial in integrating machine learning workflows on Databricks to forecast workforce trends and enhance talent acquisition processes. By fostering a culture of innovation, you will mentor a team of engineers, conduct code reviews, and implement best practices that uphold JP Morgan's high standards for data security and governance, particularly in handling sensitive financial and personal information under frameworks like SOX and GDPR. Joining JP Morgan Chase means contributing to a world-class institution where your technical leadership can make a tangible impact on our 250,000+ employees. This role offers the opportunity to work on transformative projects in a dynamic, hybrid environment in Columbus, OH, with access to top-tier resources and career growth paths. If you are passionate about data engineering in finance and ready to lead the next wave of HR data innovation, we invite you to bring your skills to our team.

Key Responsibilities

• Lead the design, development, and implementation of scalable data solutions using Databricks and AWS for the HR Data team
• Collaborate with cross-functional teams to integrate data pipelines that support HR analytics and reporting in a financial services context
• Mentor junior engineers and drive best practices in data engineering, ensuring compliance with JP Morgan's security and governance standards
• Optimize data workflows for performance, cost-efficiency, and reliability on AWS infrastructure
• Develop and maintain ETL processes using Spark and Delta Lake to handle large volumes of HR data
• Innovate with machine learning models on Databricks to derive insights for talent management and workforce planning
• Conduct code reviews, implement testing strategies, and ensure high-quality deliverables
• Stay abreast of emerging technologies and recommend integrations that enhance JP Morgan's data ecosystem
• Troubleshoot and resolve complex issues in production environments, minimizing downtime for critical HR systems
• Contribute to strategic planning for data architecture, aligning with business objectives in the financial sector

Required Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field; advanced degree preferred
• 7+ years of experience in software engineering with a focus on data platforms and cloud technologies
• Proven expertise in Databricks, including Delta Lake, Spark, and MLflow for building scalable data solutions
• Strong experience with AWS services such as S3, EMR, Lambda, and Glue for data processing and storage
• Demonstrated leadership in leading engineering teams and delivering data-driven projects in a financial services environment
• Experience with agile methodologies and CI/CD pipelines for software development
• Ability to obtain necessary security clearances for handling sensitive financial data

Preferred Qualifications

• Master's degree in Data Science or related field
• Experience in the financial services industry, particularly with HR data analytics or compliance
• Certifications such as AWS Certified Solutions Architect or Databricks Certified Data Engineer
• Knowledge of regulatory frameworks like GDPR, SOX, and financial data privacy standards
• Prior work at a large-scale financial institution like JP Morgan Chase

Required Skills

• Proficiency in Python, Scala, or Java for data processing
• Expertise in Databricks platform, including notebooks and clusters
• Advanced knowledge of AWS cloud services and infrastructure as code (IaC)
• Experience with Apache Spark for big data analytics
• Strong understanding of data modeling, warehousing, and ETL/ELT processes
• Familiarity with SQL and NoSQL databases (e.g., PostgreSQL, DynamoDB)
• Skills in version control systems like Git and CI/CD tools (e.g., Jenkins, GitHub Actions)
• Knowledge of containerization and orchestration (Docker, Kubernetes)
• Leadership and communication skills for team collaboration
• Problem-solving abilities in high-stakes financial environments
• Understanding of data security, encryption, and compliance in finance
• Analytical mindset for optimizing data pipelines
• Agile project management and scrum methodologies
• Experience with monitoring tools (e.g., CloudWatch, Databricks Jobs)

Benefits

• Competitive base salary and performance-based annual bonuses
• Comprehensive health, dental, and vision insurance plans
• 401(k) retirement savings plan with company matching contributions
• Generous paid time off, including vacation, sick days, and parental leave
• Professional development opportunities, including tuition reimbursement and access to internal training programs
• Employee stock purchase plan and financial wellness resources
• On-site fitness centers and wellness programs at JP Morgan Chase facilities
• Flexible work arrangements, including hybrid options in Columbus, OH

JP Morgan Chase is an equal opportunity employer.