Role Overview

The Data Acquisition team within OpenAI's Foundations organization powers all aspects of data collection for our groundbreaking model training operations. Managing web crawling, GPTBot services, and massive data pipelines, this team ensures our AI systems have access to the world's knowledge at unprecedented scale.

As a Senior Software Engineer on this team, you'll own complex engineering projects spanning web crawling, data ingestion, search infrastructure, and petabyte-scale distributed systems. You'll collaborate across Data Processing, Architecture, and Scaling teams while navigating compliance challenges with our legal partners.

This role demands deep expertise in building systems that process unimaginable volumes of data with reliability and efficiency. From architecting search algorithms to deploying Kubernetes-based infrastructure, you'll tackle problems that define the frontier of AI data engineering.

San Francisco-based candidates preferred, with opportunities for exceptional remote talent. OpenAI offers competitive compensation packages including equity in one of the world's most promising AI companies.

Key Responsibilities

Your day-to-day will include leading end-to-end engineering projects in data acquisition. This encompasses designing web crawlers that respect robots.txt while maximizing coverage, building ingestion pipelines that handle petabytes without data loss, and creating search systems that power our training infrastructure.

• Lead ownership of data acquisition projects from conception through production deployment
• Design and implement scalable web crawling infrastructure serving AI model training
• Architect distributed data ingestion systems processing petabytes daily
• Build advanced search and indexing algorithms optimized for massive datasets
• Develop backend services using key-value stores with complex synchronization requirements
• Deploy infrastructure using Kubernetes and Infrastructure-as-Code methodologies
• Collaborate cross-functionally with Data Processing, Architecture, and Scaling teams
• Partner with Legal on compliance, robots.txt implementation, and data privacy
• Conduct large-scale experiments analyzing system performance at web scale
• Perform production system monitoring, alerting, and optimization
• Mentor engineers on distributed systems best practices and data engineering
• Contribute to strategic roadmap planning for next-generation data infrastructure

Expect to work on systems impacting billions of web pages and handling data volumes that would overwhelm traditional infrastructure.

Qualifications

We're seeking proven senior engineers with deep distributed systems experience:

• BS/MS/PhD in Computer Science, Electrical Engineering, or equivalent
• 6+ years industry experience building production software systems
• Strong track record with large stateful distributed systems
• Experience with web crawlers, data pipelines, or search infrastructure (strongly preferred)
• Deep Kubernetes expertise including operators, custom resources, and IaC
• Production experience with key-value databases (RocksDB, Cassandra, etc.)
• Backend systems development in Python, Go, C++, or similar
• Demonstrated ability to architect complex, reliable distributed systems
• Experience with large-scale experimentation and performance analysis
• Excellent cross-functional collaboration and communication skills

Candidates with experience at web-scale companies (Google, Meta, etc.) or large-scale data platforms will be prioritized.

Salary & Benefits

Compensation Range: $280,000 - $420,000 base salary + equity + comprehensive benefits. Total compensation includes significant equity upside in OpenAI.

Exceptional Benefits Package:

• Top-tier medical, dental, vision coverage
• 401(k) with generous company match
• Unlimited PTO with recharge encouragement
• 16+ weeks parental leave
• Mental health support programs
• Fitness stipend and wellness reimbursement
• Professional development budget
• Stock options with substantial growth potential
• Relocation assistance for SF moves
• Catered meals, gym access, and more

Why Join OpenAI?

OpenAI isn't just building AI - we're creating general artificial intelligence that benefits all humanity. Your work on the Data Acquisition team directly enables our most advanced models, from GPT-4 to future systems that will transform every industry.

Join a team of the world's top engineers solving problems at unprecedented scale. Work with petabytes of data, deploy systems serving billions, and collaborate with researchers pushing AI frontiers. OpenAI offers:

• Impact: Your systems power models used by millions worldwide
• Growth: Rapid career advancement in high-impact environment
• Culture: Mission-driven team valuing diverse perspectives
• Resources: Best-in-class tools, hardware, and compute
• Location: Vibrant San Francisco tech ecosystem

We're committed to equal opportunity and building AI safety from the ground up.

How to Apply

Ready to build the data infrastructure powering humanity's AI future? Submit your resume and a brief note about your most impactful distributed systems project. We're excited to review strong candidates immediately.

Application Link: [Apply Now Button]

OpenAI is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.