Lead Data Engineer - Python, Pyspark & AWS

Location: GLASGOW, LANARKSHIRE, United Kingdom

Job Family: Data Engineering

About the Role

At JP Morgan Chase, we are at the forefront of financial innovation, leveraging cutting-edge technology to power global banking, investment, and asset management services. As a Lead Data Engineer - Python, PySpark & AWS in our Glasgow team, you will play a pivotal role in driving impactful cloud data solutions that support critical business functions such as risk assessment, fraud detection, and regulatory reporting. Joining our collaborative and innovative team, you will advance your engineering career by architecting scalable data infrastructures that handle petabyte-scale financial datasets, ensuring reliability and efficiency in a fast-paced, high-stakes environment. This position offers the opportunity to contribute to transformative projects that directly influence JP Morgan's position as a leader in the financial services industry. In this leadership role, you will design and implement robust data pipelines using Python and PySpark on AWS, integrating services like S3 for storage, EMR for processing, and Glue for orchestration. You will collaborate closely with data scientists, quantitative analysts, and compliance teams to translate complex financial requirements into actionable data strategies, while upholding stringent security protocols to safeguard client information. Responsibilities include mentoring junior engineers, optimizing performance for real-time trading analytics, and innovating with emerging cloud technologies to enhance operational resilience. Your work will directly impact key initiatives, such as improving portfolio optimization models and streamlining compliance workflows, in alignment with JP Morgan's commitment to technological excellence. We value engineers who thrive in dynamic, team-oriented settings and are passionate about solving intricate challenges in finance. This role not only provides exposure to advanced tools and methodologies but also fosters professional growth through JP Morgan's renowned development programs. If you are ready to lead data engineering efforts that drive business value and innovation at one of the world's largest financial institutions, join us in Glasgow to shape the future of data-driven finance.

Key Responsibilities

• Design, build, and optimize scalable data pipelines using Python, PySpark, and AWS to support financial analytics and reporting
• Lead the development of cloud-native data solutions that enhance risk management and compliance in JP Morgan Chase's operations
• Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to deliver impactful insights
• Implement data governance and security best practices to protect sensitive financial information
• Mentor junior engineers and drive innovation in data engineering practices within the team
• Troubleshoot and resolve complex data processing issues in high-volume, real-time financial systems
• Integrate AWS services to enable efficient data ingestion, transformation, and storage for trading and portfolio analysis
• Contribute to agile sprints, ensuring timely delivery of data engineering projects aligned with business objectives
• Monitor and optimize data infrastructure for cost-efficiency and performance in a dynamic financial landscape
• Stay abreast of emerging technologies to recommend improvements in JP Morgan's data ecosystem

Required Qualifications

• Bachelor's degree in Computer Science, Engineering, or a related field
• 5+ years of experience in data engineering roles, with a focus on cloud-based data solutions
• Proficiency in Python and PySpark for large-scale data processing
• Hands-on experience with AWS services including S3, EMR, Glue, and Lambda
• Strong understanding of data pipelines, ETL processes, and data warehousing in financial environments
• Experience working with sensitive financial data and adhering to regulatory standards like GDPR and SOX
• Demonstrated ability to lead small teams in agile development environments

Preferred Qualifications

• Master's degree in Data Science or related discipline
• Experience in the financial services industry, particularly in banking or investment management
• Certification in AWS (e.g., AWS Certified Data Analytics - Specialty)
• Knowledge of machine learning frameworks integrated with PySpark
• Prior exposure to big data technologies like Hadoop or Kafka in a production setting

Required Skills

• Python programming expertise
• PySpark for distributed data processing
• AWS cloud platform proficiency (S3, EMR, Glue)
• ETL pipeline development
• SQL and NoSQL database management
• Data modeling and schema design
• Agile methodologies and version control (Git)
• Problem-solving in high-stakes financial environments
• Leadership and team collaboration
• Communication for stakeholder engagement
• Knowledge of financial regulations and data privacy
• Big data tools (Hadoop, Spark)
• Scripting and automation skills
• Analytical thinking for data optimization
• Adaptability to evolving tech landscapes

Benefits

• Competitive base salary and performance-based annual bonuses
• Comprehensive health, dental, and vision insurance coverage
• Generous 401(k) matching and pension contributions
• Paid time off including vacation, sick leave, and parental leave
• Professional development opportunities with tuition reimbursement and access to internal training programs
• Employee stock purchase plan and financial wellness resources
• Flexible hybrid work arrangements and wellness stipends
• Global mobility support for career advancement within JP Morgan Chase

JP Morgan Chase is an equal opportunity employer.