Senior Software Engineer, Inference Platform at MongoDB - Palo Alto, California

Role Overview

MongoDB, the modern developer data platform, is hiring a Senior Software Engineer, Inference Platform to build the next-generation AI inference infrastructure powering MongoDB Atlas. Join our Search and AI Platform team in Palo Alto, California to create real-time, low-latency inference systems for semantic search, vector embeddings, and AI-native developer experiences.

From our recent Voyage.ai acquisition, you'll collaborate with top ML researchers to productionize embedding models, rerankers, and hybrid retrieval systems directly integrated with MongoDB's NoSQL database. This hybrid role focuses on scalable, multi-tenant infrastructure using tools like vLLM, Kubernetes, and cloud-native architectures.

Own core components of the inference platform that serve thousands of developers building AI applications on MongoDB Atlas – the world's most popular developer data platform supporting AWS, Google Cloud, and Azure.

Key Responsibilities at MongoDB

• Design and implement multi-tenant inference platform components integrated directly with MongoDB Atlas for semantic search and hybrid retrieval.
• Partner with AI engineers and ML researchers to productionize embedding models and rerankers for both batch and real-time inference use cases.
• Build platform features including latency-aware routing, model versioning, health monitoring, and comprehensive observability.
• Optimize performance, autoscaling, GPU utilization, and resource efficiency in cloud-native environments.
• Collaborate across product, infrastructure, and ML teams to meet Atlas-scale reliability and latency requirements.
• Gain hands-on experience with vLLM inference engine and Kubernetes container orchestration.
• Drive integration between inference systems and MongoDB's vector search capabilities like Faiss, HNSW, and ScaNN.
• Contribute to developer-first experiences enabling AI-native applications on the MongoDB platform.
• Ensure seamless scalability for 75% of Fortune 100 companies using MongoDB Atlas.

Qualifications & Requirements

• 5+ years building backend or infrastructure systems at scale in high-performance environments.
• Expertise in Go, Rust, Python, or C++ with focus on performance-critical systems and reliability.
• Deep experience with cloud-native architectures, distributed systems, and multi-tenant service design.
• Familiarity with ML model serving, inference runtimes, and vector databases even without direct model deployment.
• Bonus: Knowledge of vector search algorithms (Faiss, HNSW, ScaNN) and hybrid retrieval systems.
• Proven ability to collaborate across functional teams including ML researchers, backend engineers, and platform teams.
• Passion for building systems integrated into MongoDB Atlas used by thousands of developers worldwide.

Nice to Have: Production ML workload integration, RAG systems, or open-source contributions to ML serving infrastructure.

Salary & Benefits

Competitive compensation for Senior Software Engineer, Inference Platform roles in Palo Alto, California typically ranges from $180,000 - $260,000 USD base salary plus equity, bonuses, and comprehensive benefits. Total compensation varies based on experience.

• Competitive Equity Packages with significant upside potential.
• MongoDB University for continuous learning and certification.
• Global Family Leave and generous parental leave policies.
• Wellness Programs including gym reimbursements and mental health support.
• Fertility Assistance and family planning benefits.
• Employee Affinity Groups for diversity and inclusion.
• Hybrid Work Model in beautiful Palo Alto offices.
• Comprehensive Health Insurance with global coverage.

Why Join MongoDB?

Be at the forefront of the AI era database revolution with MongoDB – powering 60,000 customers including 75% of Fortune 100. Build the AI foundation of the world's most popular developer data platform.

• Work with Voyage.ai ML researchers bringing cutting-edge ideas to production scale.
• Tackle hard problems in real-time inference, observability, and distributed infrastructure.
• Thrive in a culture of mentorship, ownership, and technical excellence.
• Join a global team innovating at market speed across multi-cloud environments.

MongoDB Atlas is the only globally distributed, multi-cloud database – and you're building its AI-native future.

How to Apply

Ready to power AI inference at MongoDB scale? Apply now for the Senior Software Engineer, Inference Platform position in Palo Alto, California. Limited hybrid roles available – don't miss your chance to shape the future of developer data platforms.

Submit your resume highlighting MongoDB, NoSQL, distributed systems, and ML infrastructure experience. Join us in redefining databases for the AI era!