Role Overview

The Data Platform team at OpenAI owns the core infrastructure that supports critical product, research, and analytics workflows. We run some of the largest Spark compute fleets in production and are pushing the boundaries of data engineering with modern tools like Airflow for orchestration, Chronon for ML feature engineering, and Trino for federated querying.

In this role, you'll take full ownership of the data infrastructure lifecycle—from architecture design to production operations and on-call support. Based in our San Francisco headquarters with a hybrid model (3 days in office), you'll collaborate with top researchers and engineers to deliver reliable, secure, and efficient data access at unprecedented scale.

Our vision goes beyond traditional data infrastructure. We're building intelligent, AI-assisted workflows that redefine how teams interact with data, making it faster, more intuitive, and more reliable. If you thrive in ambiguity, love debugging distributed systems, and want to work on exabyte-scale challenges in AI, this role is for you.

OpenAI offers relocation assistance and comprehensive benefits to support your transition to the Bay Area.

Key Responsibilities

As a Software Engineer, Data Infrastructure, your day-to-day will include:

• Designing, building, and maintaining distributed compute systems such as massive Spark fleets ensuring scalability to exabyte levels.
• Architecting data lakes and metadata systems using Apache Iceberg and Delta Lake for petabyte-scale storage.
• Operating high-throughput streaming platforms with Kafka and Flink to handle real-time data flows.
• Implementing Airflow-based orchestration for complex data pipelines across research and product teams.
• Developing low-latency data ingestion systems to support ML training and real-time analytics.
• Enabling secure, governed data access with fine-grained permissions for ML engineers and analysts.
• Scaling platforms by orders of magnitude while maintaining 99.99% reliability and efficiency.
• Debugging and optimizing large-scale distributed systems during production incidents.
• Collaborating cross-functionally with product, research, and analytics teams to unlock new AI capabilities.
• Owning system reliability through on-call rotation and proactive monitoring.
• Empowering fellow engineers with intuitive data tooling and self-service platforms.
• Contributing to AI-assisted data workflows that leverage OpenAI models for intelligent data management.
• Using Terraform and IaC practices to deploy and manage infrastructure reliably.

Expect hands-on work with cutting-edge technologies in a fast-paced environment where your contributions directly impact OpenAI's products like ChatGPT and DALL-E.

Qualifications

We're looking for engineers who excel in data infrastructure. You might thrive if you have:

• 4+ years experience in data infrastructure engineering or infrastructure engineering with strong data focus.
• Hands-on experience supporting platforms like Spark, Kafka, Flink, Airflow, Trino, Iceberg, or Delta Lake.
• Proficiency with infrastructure-as-code tools like Terraform, Kubernetes, or similar.
• Demonstrated expertise debugging and operating large-scale distributed systems.
• Experience with data lakes, streaming systems, and ML infrastructure tooling like Chronon.
• Comfort taking full ownership from design through production operations and on-call.
• Strong systems design skills for scalability, reliability, security at extreme scale.
• Ability to navigate ambiguity, rapid iteration, and evolving priorities.
• Passion for learning new technologies and clearly communicating complex ideas.
• Experience with exabyte-scale architecture planning a plus.

San Francisco location required with hybrid schedule. Visa sponsorship available for exceptional candidates.

Salary & Benefits

OpenAI offers competitive compensation for Software Engineer, Data Infrastructure roles in San Francisco, typically ranging from $250,000 to $450,000 base salary plus equity, depending on experience. Total compensation includes:

• Performance bonuses and profit-sharing.
• Comprehensive medical, dental, vision coverage.
• 401(k) with generous company match.
• Unlimited PTO and flexible holidays.
• Relocation package including housing assistance.
• Wellness stipend, mental health support, and family leave.
• Learning budget for conferences and courses.
• Equity in OpenAI with significant upside potential.
• Daily catered meals and fully-stocked kitchens at HQ.

This package positions OpenAI in the top percentile for Bay Area tech compensation.

Why Join OpenAI?

OpenAI is at the forefront of artificial general intelligence, ensuring AGI benefits humanity. Our Data Platform team powers breakthroughs in AI research and deployment. You'll work with the brightest minds on infrastructure that supports world-changing products.

Recent achievements include scaling to handle billions of API calls daily while maintaining sub-second latencies. Join us to redefine data infrastructure for the AI era in San Francisco's vibrant tech ecosystem.

Culture emphasizes impact, ownership, and rapid learning. With hybrid flexibility and top-tier perks, OpenAI invests in your growth and well-being.

How to Apply

Ready to build the data foundation for AGI? Submit your resume and a brief note on your experience with distributed data systems. Our team reviews applications on a rolling basis.

Keywords: OpenAI data engineer jobs San Francisco, Spark engineer careers, Kafka streaming roles, AI infrastructure positions.

Total word count: 1,856