Role Overview

Join Datadog's cutting-edge Annotation & Evaluation team as a Senior Software Engineer (MLOps) in Paris or Sophia Antipolis, France. We're building an internal AI platform that powers model training, evaluation, and deployment at scale. This role sits at the intersection of applied ML, data engineering, and platform infrastructure, ensuring our AI models are reliable, safe, and production-ready across Datadog's cloud observability and security platform.

From interactive labeling pipelines to automated evaluation environments, you'll create the feedback loops that enable engineers and scientists to continuously improve LLMs and AI agents. Work with production-like telemetry, develop benchmarking tools, and drive trust & safety observability for Datadog's AI products. This is a hybrid role emphasizing our vibrant office culture for collaboration and innovation.

Key Responsibilities at Datadog

• Design automated evaluation systems for AI models including LLMs and agents using real production telemetry and scenarios.
• Lead benchmark suite development with evaluation pipelines and model comparison tools featuring trust & safety metrics.
• Build labeling system integrations like Label Studio and coordinate annotation workflows across internal/external teams.
• Collaborate with AI teams to enable fast iteration, reproducible experiments, and interpretable evaluations.
• Develop observability data pipelines that feed model metrics, results, and alerts into Datadog's stack.
• Drive safe deployment practices through bias detection, hallucination monitoring, and human-in-the-loop systems.
• Scale evaluation infrastructure to support enterprise-grade AI deployments across cloud environments.
• Implement performance diagnostics and trust metrics for continuous model improvement.
• Champion MLOps best practices within Datadog's engineering culture.

Qualifications & Requirements

• 6+ years software engineering experience, including 2+ years with ML/AI products or platforms.
• Hands-on experience building backend/data infrastructure for model evaluation, benchmarking, or labeling.
• Full-stack comfort level: from orchestrating data pipelines to building UI dashboards for evaluation.
• Solid grasp of ML concepts – performance metrics, prompt evaluation, trust & safety challenges.
• Fluency in Python or Go + cloud-native development patterns (Kubernetes, Docker, etc.).
• Bonus: Open-source eval frameworks (lm-eval-harness), vector DBs, human feedback loops.

Datadog celebrates diverse backgrounds. Passion for technology and growth mindset? We encourage you to apply even if you don't check every box.

Salary & Benefits

Competitive Senior MLOps Engineer salary in Paris/Sophia Antipolis ranging €85,000-€120,000 based on experience, plus equity. Full benefits package includes:

• New hire RSUs + ESPP stock programs
• Global family health insurance
• Free mental health benefits (employees + dependents 6+)
• Professional development stipend + career pathing
• Mentor/buddy programs + Community Guilds
• Inclusion Talks + internal networking
• Fitness/commuter reimbursements
• Hybrid work flexibility with office perks

Why Join Datadog?

Datadog (NASDAQ: DDOG) powers cloud observability and security for thousands of engineering teams worldwide. As a rare profitable growth SaaS leader, we break down silos and solve cloud complexity. Join our people-first culture where engineers build for engineers.

Work on AI/ML infrastructure that impacts millions of users. Benefit from continuous learning, inclusive Guilds, and global mental health support. Our hybrid model balances collaboration with work-life harmony. Paris and Sophia Antipolis offices offer vibrant tech ecosystems and European quality of life.

Be part of #DatadogLife – solving tough problems, taking smart risks, celebrating wins. Follow @datadoghq for culture insights.

How to Apply

Ready to scale AI evaluation at Datadog? Apply now for Senior Software Engineer (MLOps) – Annotation & Evaluation in France. Submit your resume highlighting ML infrastructure experience. We'll review applications quickly and connect for next steps. Don't miss this chance to shape AI reliability at a cloud observability leader!

Datadog is an Equal Opportunity Employer committed to diversity.