Role Overview

The Data Acquisition team within OpenAI's Foundations organization is at the heart of our model training operations. We're responsible for collecting massive datasets through web crawling, GPTBot services, and advanced data ingestion pipelines. As a Software Engineer on this team, you'll own critical projects that ensure seamless data flow to our AI research teams.

Located in San Francisco, this role involves close collaboration with Data Processing, Architecture, and Scaling teams. You'll tackle challenges like building systems that handle petabytes of data daily while maintaining strict compliance with data privacy regulations. This is not just engineering—it's mission-critical work that directly impacts OpenAI's ability to develop safe, beneficial AGI.

With 4+ years of experience required, this position demands expertise in distributed systems, Kubernetes, and large-scale data processing. If you thrive in fast-paced environments and love solving complex scalability problems, this OpenAI Software Engineer role is your opportunity to shape the future of AI.

Key Responsibilities

OpenAI Software Engineers in Data Acquisition lead from the front. Your day-to-day will include:

• Owning end-to-end engineering projects in web crawling, data ingestion, and search infrastructure
• Designing and deploying distributed systems capable of processing petabytes of unstructured data
• Collaborating cross-functionally with Data Processing, Architecture, Scaling, and Legal teams
• Architecting sophisticated data indexing algorithms and search capabilities
• Building resilient backend services using key-value databases and synchronization protocols
• Managing deployments in Kubernetes with Infrastructure-as-Code practices
• Conducting data-driven experiments to optimize system performance at scale
• Ensuring compliance with global data privacy regulations through close legal partnership
• Performing system health monitoring and proactive optimization
• Contributing to OpenAI's web crawling infrastructure including GPTBot operations
• Implementing high-throughput data ingestion pipelines
• Documenting architectures and mentoring junior engineers
• Staying ahead of emerging technologies in distributed systems and data engineering

These responsibilities position you at the intersection of systems engineering, data science, and AI research—perfect for engineers who want maximum impact.

Qualifications

To succeed in this OpenAI Data Acquisition Engineer role, you'll need:

• BS/MS/PhD in Computer Science, Electrical Engineering, or related technical field
• 4+ years of professional software engineering experience
• Proven track record building large-scale distributed systems
• Deep expertise in data processing pipelines and petabyte-scale infrastructure
• Hands-on Kubernetes experience with Infrastructure-as-Code (Terraform, etc.)
• Experience with web crawlers or large-scale data collection systems (strongly preferred)
• Strong systems programming skills (Python, Go, C++, Rust)
• Backend development experience with key-value stores (Cassandra, Redis, DynamoDB)
• Demonstrated ability to handle ambiguity and rapidly changing priorities
• Excellent written and verbal communication skills
• Passion for AI safety and beneficial AGI development

Candidates with experience in search infrastructure, data privacy compliance, or large web crawling projects will receive strong consideration.

Salary & Benefits

OpenAI offers competitive compensation for San Francisco Software Engineers in Data Acquisition. Total compensation typically ranges from $220,000 - $350,000 annually, including base salary, equity, and bonuses. Exact figures depend on experience and qualifications.

Comprehensive benefits include:

• Top-tier medical, dental, vision coverage
• 401(k) with generous company match
• Unlimited PTO with encouragement to recharge
• Parental leave (16 weeks fully paid)
• Professional development budget
• Wellness programs and mental health support
• Commuter benefits and free lunches
• Equity in OpenAI with significant upside potential

This package reflects OpenAI's commitment to attracting world-class talent to solve humanity's greatest challenges.

Why Join OpenAI?

OpenAI isn't just another tech company—it's the leading organization developing safe artificial general intelligence. Your work on the Data Acquisition team directly enables breakthroughs like GPT-4 and beyond. Here's why engineers choose OpenAI:

• Mission-Driven Impact: Every system you build powers AI that benefits humanity
• Cutting-Edge Challenges: Solve problems at unprecedented scale (petabytes daily)
• Elite Team: Collaborate with PhDs and industry veterans from Google, Meta, DeepMind
• San Francisco HQ: Vibrant tech ecosystem with top talent concentration
• Rapid Growth: Join during our most exciting phase of expansion
• Culture of Excellence: High ownership, intellectual rigor, and genuine care for safety

OpenAI values diverse perspectives and is committed to equal opportunity employment. We conduct background checks consistent with applicable law, including San Francisco Fair Chance Ordinance.

How to Apply

Ready to build the data infrastructure powering AGI? Here's your path to joining OpenAI's Data Acquisition team:

1. Review Requirements: Ensure you meet the 4+ years experience and distributed systems expertise
2. Prepare Application: Submit resume, GitHub/portfolio, and cover letter explaining your fit
3. Technical Screening: Coding assessment focused on systems design and data engineering
4. Technical Interviews: 4-5 rounds covering distributed systems, Kubernetes, data pipelines
5. Team Matching: Meet potential teammates and discuss project fit
6. Offer: Competitive compensation package with equity

Apply now—OpenAI Data Acquisition Engineer positions fill quickly. Shape the future of AI from San Francisco.

Keywords: OpenAI software engineer jobs San Francisco, data acquisition engineer careers, distributed systems engineer OpenAI, Kubernetes engineer AI, web crawling engineer jobs