Role Overview

OpenAI's Inference team is at the forefront of deploying our flagship models like GPT series, 4o Image Generation, and Whisper across diverse platforms. As a Software Engineer on the Multi Modal Inference team, you'll develop high-performance infrastructure for serving real-time audio, image, and multimodal workloads at massive scale. This small, fast-moving team collaborates directly with world-class researchers and product teams to bring experimental AI capabilities into production.

Multimodal inference represents the future of AI interaction—think generating speech from text, understanding complex images, and enabling natural human-AI conversations that transcend text-only interfaces. Your work will ensure these models are reliable, performant, and accessible to millions of developers worldwide. Based in San Francisco, you'll thrive in an environment that values innovation, rapid iteration, and cross-functional collaboration.

This isn't just engineering; it's pioneering the infrastructure that makes advanced AI ubiquitous. With OpenAI's mission to benefit all of humanity, your contributions will have global impact.

Key Responsibilities

In this high-impact role, you'll tackle challenging problems at the intersection of ML systems, distributed computing, and real-time performance. Key responsibilities include:

• Designing and implementing inference infrastructure optimized for large-scale multimodal models handling image, audio, and text inputs.
• Optimizing end-to-end systems for ultra-low latency and high-throughput delivery of complex multimodal outputs.
• Building tools and workflows that seamlessly transition experimental research prototypes into robust production services.
• Collaborating intimately with AI researchers training frontier models and product engineers defining novel user interactions.
• Driving system-level improvements in GPU utilization, tensor parallelism, pipeline parallelism, and custom hardware abstractions.
• Developing scalable data pipelines for heterogeneous model architectures and diverse input/output formats.
• Implementing monitoring, alerting, and auto-scaling systems for production inference clusters.
• Contributing to open-source inference tooling and developer platforms that power the AI ecosystem.
• Troubleshooting live production issues across distributed GPU clusters under heavy load.
• Partnering with infrastructure teams to deploy on cutting-edge hardware like H100s, TPUs, and custom accelerators.
• Creating APIs and SDKs that enable developers to easily integrate multimodal capabilities into applications.
• Staying ahead of evolving research by incorporating new techniques like speculative decoding and mixture-of-experts routing.
• Conducting performance profiling and benchmarking across diverse model sizes and workloads.

Expect to own projects end-to-end, from initial design through production deployment and ongoing optimization.

Qualifications

We're seeking engineers who excel in ambiguous, fast-paced environments and have a track record of shipping production ML systems. You might thrive if you:

• Have 5+ years building and scaling inference systems for LLMs, vision models, or multimodal AI.
• Deep expertise with GPU-based ML workloads, understanding memory bandwidth, compute utilization, and KV cache dynamics.
• Experience optimizing complex data pipelines for images, audio spectrograms, and tokenized multimodal inputs.
• Familiarity with state-of-the-art inference frameworks (vLLM, TensorRT-LLM, SGLang, custom serving stacks).
• Strong systems programming skills in Python/C++/CUDA for performance-critical components.
• Proven ability to collaborate with research scientists on bleeding-edge model architectures.
• Comfort owning distributed systems spanning networking, storage, and orchestration layers.
• Experience with containerization (Docker/Kubernetes) and cloud infrastructure (AWS/GCP).
• Bonus: Production experience with diffusion models, audio codecs, or real-time speech systems.

Salary & Benefits

OpenAI offers competitive compensation packages for Software Engineers in Inference roles, typically ranging from $250,000 to $450,000 base salary plus significant equity. Total compensation can exceed $700K+ for top performers, including bonuses and stock options in our rapidly growing company.

Comprehensive benefits include:

• Premium health, dental, vision coverage with low employee premiums
• 401(k) with generous company match
• Unlimited PTO and flexible work policies
• Parental leave (16+ weeks) and fertility benefits
• Professional growth stipend ($3K+/year)
• Wellness programs including mental health support
• Daily catered meals and fully stocked kitchens
• Commuter benefits and gym memberships
• Employee resource groups and volunteer opportunities

This package reflects OpenAI's commitment to attracting world-class talent building safe AGI.

Why Join OpenAI?

OpenAI isn't just another tech company—it's the leading force in artificial general intelligence research and deployment. Our models power applications used by millions daily, from ChatGPT to DALL·E and beyond. Joining the Inference team means:

• Working with the smartest researchers and engineers in AI
• Access to unlimited compute resources and latest hardware
• Direct impact on products serving 100M+ weekly users
• Equity in a company valued at $150B+ with explosive growth
• Culture emphasizing safety, rapid iteration, and mission alignment
• San Francisco location in vibrant SoMa tech hub

We're building AI that benefits humanity, and your expertise in multimodal inference will accelerate that mission.

How to Apply

Ready to power the next era of multimodal AI? Submit your resume and a brief note on your most impactful inference project. Our hiring process includes:

1. 30-minute recruiter screen
2. Technical deep-dive with engineering team
3. Systems design interview focused on ML infra
4. Live coding challenge with multimodal scenarios
5. Final interviews with research and leadership

OpenAI is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. Apply today to join the AI revolution!