Job Description

Locations

• Sunnyvale, California, United States 94085

Salary

Skills Required

• computer visionintermediate
• large language modelsintermediate
• deep learningintermediate
• machine learningintermediate
• multi-modal LLM (MM-LLM)intermediate
• ML algorithmsintermediate
• data processingintermediate
• model optimization techniquesintermediate
• evaluation approaches for vision-language modelsintermediate
• evaluating ML algorithmsintermediate
• assessing model qualityintermediate
• failure analysisintermediate
• developing performance metricsintermediate
• data cleaningintermediate
• data transformationintermediate
• data curationintermediate
• model optimizationintermediate
• collaboration with multi-functional teamsintermediate
• cross-functional collaborationintermediate
• communication of resultsintermediate

Required Qualifications

• BS and a minimum of 3 years relevant industry experience (experience, 3 years)
• Proven background in data science, machine learning, computer vision and statistical data analysis. (experience)
• Advanced programming skills in data manipulation & processing (SQL & Python preferred). (experience)
• Demonstrated experience in in-depth analysis of machine learning model failures. (experience)
• Experience crafting, conducting, analyzing, and interpreting experiments and investigations. (experience)
• Expertise in data wrangling and developing data visualizations & reporting with toolings such as Tableau, Superset, AWS etc. (experience)

Preferred Qualifications

• Experience working with multi-modal foundation models such as GPT-4o, Gemini 2.5, Claudi 3/4, LLaVA, Flamingo, etc. (experience)
• Familiar with machine learning interpretability method and standard processes. (experience)
• Exposure to evaluating vision-language models in production or research settings. (experience)
• Experience handling complex programs and collaborating across engineering, product, and data teams. (experience)
• Detail-oriented to keep track of and understand the workings of sophisticated algorithms. (experience)
• Strong attention to detail in working with large datasets and complex ML systems. (experience)
• Curious, self-motivated, and able to drive improvements to model evaluation pipelines and annotation programs. (experience)
• Outstanding communication skills – both written and verbal – with experience presenting to leadership. (experience)

Responsibilities

• Our organization supports a diverse array of programs passionate about evaluating ML algorithms and assessing model quality at scale, across domains like computer vision, audio, and multi-modal systems. You will collaborate with multi-functional teams, including domain experts and engineering leads, and adapt methodologies as new insights emerge.
• In this role you will:
• - Evaluate ML & MM-LLM Models: Analyze and validate computer vision, multi-modal, and large language models to ensure they meet accuracy, robustness, and usability standards.
• - Develop Metrics: Design and implement metrics to measure the efficiency and accuracy of models.
• - Failure Analysis: Conduct in-depth analysis on model failures across CV and MM-LLM pipelines to surface root causes and improvement areas.
• - Data Processing: Clean, transform, and curate large-scale datasets for model evaluation and benchmarking.
• - Model Optimization: Apply innovative techniques to optimize models for scalability and real-world deployment.
• - Collaborate multi-functionally: Work closely with cross-functional teams, including software engineers, product managers, and other data scientists, to integrate models into production.
• - Communicate Results: Present findings clearly and effectively to collaborators across levels of technical understanding.

Tags & Categories