Machine Learning - Data Scientist
Apple
Machine Learning - Data Scientist
Job Description
Do you have a passion for computer vision and solving deep learning problems? The Video Engineering Data Analytics and Quality group is seeking an expert in evaluating machine learning and deep learning models, including foundation models and multimodal systems. This role will play a critical part in crafting robust evaluation frameworks, using both traditional statistical methods and modern techniques like LLM-as-a-Judge! The ideal candidate combines strong analytical thinking, expertise in Python, and advanced knowledge of statistical methodologies and data quality standards. This role involves collaboration with teams at Apple passionate about developing foundation models, including ML engineers, data scientists, and ML Infrastructure engineers to deliver amazing user experiences! Develop robust methodologies to assess the performance of foundation models (e.g., LLMs, vision-language models, etc.) across diverse tasks. Leverage LLMs as judges to perform subjective and open-ended model evaluations (e.g., for summarization, reasoning, or multimodal generation tasks). Build, curate, and lead evaluation datasets and benchmarks. Advanced proficiency in at least one scripting language, preferably Python. Collaborate with research, engineering, and product teams to define evaluation goals aligned with user experience and product quality. Conduct failure analysis and uncover edge cases to improve model robustness. Contribute to our tools and infrastructure to automate and scale evaluation processes.
Locations
- Sunnyvale, California, United States 94085
Salary
Estimated Salary Rangemedium confidence
25,000,000 - 60,000,000 INR / yearly
Source: ai estimated
* This is an estimated range based on market data and may vary based on experience and qualifications.
Skills Required
- passion for computer visionintermediate
- solving deep learning problemsintermediate
- evaluating machine learning modelsintermediate
- evaluating deep learning modelsintermediate
- evaluating foundation modelsintermediate
- evaluating multimodal systemsintermediate
- crafting robust evaluation frameworksintermediate
- traditional statistical methodsintermediate
- modern techniques like LLM-as-a-Judgeintermediate
- strong analytical thinkingintermediate
- expertise in Pythonintermediate
- advanced knowledge of statistical methodologiesintermediate
- advanced knowledge of data quality standardsintermediate
- collaboration with teamsintermediate
- developing foundation modelsintermediate
- develop robust methodologiesintermediate
- assess the performance of foundation modelsintermediate
- leverage LLMs as judgesintermediate
- perform subjective model evaluationsintermediate
- perform open-ended model evaluationsintermediate
- build evaluation datasetsintermediate
- curate evaluation datasetsintermediate
- lead evaluation datasetsintermediate
- build benchmarksintermediate
- advanced proficiency in scripting languageintermediate
- advanced proficiency in Pythonintermediate
- collaborate with research teamsintermediate
- collaborate with engineering teamsintermediate
- collaborate with product teamsintermediate
- define evaluation goalsintermediate
- conduct failure analysisintermediate
- uncover edge casesintermediate
- improve model robustnessintermediate
- contribute to toolsintermediate
- contribute to infrastructureintermediate
- automate evaluation processesintermediate
- scale evaluation processesintermediate
Required Qualifications
- BS and a minimum of 10 years relevant industry experience. (experience, 10 years)
- Strong experience in evaluating supervised, unsupervised, and deep learning models. (experience)
- Hands-on experience evaluating LLMs (e.g., GPT, Claude, PaLM) and using them as scoring/judging mechanisms. (experience)
- Familiarity with multimodal models (e.g., image + text, video + audio) and related evaluation challenges. (experience)
- Proficiency in Python and libraries such as NumPy, pandas, scikit-learn, PyTorch, or TensorFlow. (experience)
- Solid understanding of statistical testing, sampling, confidence intervals, and metrics (e.g., precision/recall, BLEU, ROUGE, FID, etc.). (experience)
- Strong documentation skills, including the ability to write technical reports and present to non-technical audiences. (experience)
Preferred Qualifications
- Experience working with open-source evaluation tools like OpenEval, ELO-based ranking, or LLM-as-a-Judge frameworks. (experience)
- Familiarity with prompt engineering, few-shot or zero-shot evaluation techniques. (experience)
- Experience evaluating generative models (e.g., text generation, image generation). (experience)
- Prior contributions to ML benchmarks or public evaluations. (experience)
- Strong interpersonal skills. (experience)
Responsibilities
- Develop robust methodologies to assess the performance of foundation models (e.g., LLMs, vision-language models, etc.) across diverse tasks. Leverage LLMs as judges to perform subjective and open-ended model evaluations (e.g., for summarization, reasoning, or multimodal generation tasks). Build, curate, and lead evaluation datasets and benchmarks. Advanced proficiency in at least one scripting language, preferably Python. Collaborate with research, engineering, and product teams to define evaluation goals aligned with user experience and product quality. Conduct failure analysis and uncover edge cases to improve model robustness. Contribute to our tools and infrastructure to automate and scale evaluation processes.
Tags & Categories
Answer 10 quick questions to check your fit for Machine Learning - Data Scientist @ Apple.
10 Questions
~2 Minutes
Instant Score
Related Books and Jobs
Machine Learning - Data Scientist
Apple
Machine Learning - Data Scientist
Job Description
Do you have a passion for computer vision and solving deep learning problems? The Video Engineering Data Analytics and Quality group is seeking an expert in evaluating machine learning and deep learning models, including foundation models and multimodal systems. This role will play a critical part in crafting robust evaluation frameworks, using both traditional statistical methods and modern techniques like LLM-as-a-Judge! The ideal candidate combines strong analytical thinking, expertise in Python, and advanced knowledge of statistical methodologies and data quality standards. This role involves collaboration with teams at Apple passionate about developing foundation models, including ML engineers, data scientists, and ML Infrastructure engineers to deliver amazing user experiences! Develop robust methodologies to assess the performance of foundation models (e.g., LLMs, vision-language models, etc.) across diverse tasks. Leverage LLMs as judges to perform subjective and open-ended model evaluations (e.g., for summarization, reasoning, or multimodal generation tasks). Build, curate, and lead evaluation datasets and benchmarks. Advanced proficiency in at least one scripting language, preferably Python. Collaborate with research, engineering, and product teams to define evaluation goals aligned with user experience and product quality. Conduct failure analysis and uncover edge cases to improve model robustness. Contribute to our tools and infrastructure to automate and scale evaluation processes.
Locations
- Sunnyvale, California, United States 94085
Salary
Estimated Salary Rangemedium confidence
25,000,000 - 60,000,000 INR / yearly
Source: ai estimated
* This is an estimated range based on market data and may vary based on experience and qualifications.
Skills Required
- passion for computer visionintermediate
- solving deep learning problemsintermediate
- evaluating machine learning modelsintermediate
- evaluating deep learning modelsintermediate
- evaluating foundation modelsintermediate
- evaluating multimodal systemsintermediate
- crafting robust evaluation frameworksintermediate
- traditional statistical methodsintermediate
- modern techniques like LLM-as-a-Judgeintermediate
- strong analytical thinkingintermediate
- expertise in Pythonintermediate
- advanced knowledge of statistical methodologiesintermediate
- advanced knowledge of data quality standardsintermediate
- collaboration with teamsintermediate
- developing foundation modelsintermediate
- develop robust methodologiesintermediate
- assess the performance of foundation modelsintermediate
- leverage LLMs as judgesintermediate
- perform subjective model evaluationsintermediate
- perform open-ended model evaluationsintermediate
- build evaluation datasetsintermediate
- curate evaluation datasetsintermediate
- lead evaluation datasetsintermediate
- build benchmarksintermediate
- advanced proficiency in scripting languageintermediate
- advanced proficiency in Pythonintermediate
- collaborate with research teamsintermediate
- collaborate with engineering teamsintermediate
- collaborate with product teamsintermediate
- define evaluation goalsintermediate
- conduct failure analysisintermediate
- uncover edge casesintermediate
- improve model robustnessintermediate
- contribute to toolsintermediate
- contribute to infrastructureintermediate
- automate evaluation processesintermediate
- scale evaluation processesintermediate
Required Qualifications
- BS and a minimum of 10 years relevant industry experience. (experience, 10 years)
- Strong experience in evaluating supervised, unsupervised, and deep learning models. (experience)
- Hands-on experience evaluating LLMs (e.g., GPT, Claude, PaLM) and using them as scoring/judging mechanisms. (experience)
- Familiarity with multimodal models (e.g., image + text, video + audio) and related evaluation challenges. (experience)
- Proficiency in Python and libraries such as NumPy, pandas, scikit-learn, PyTorch, or TensorFlow. (experience)
- Solid understanding of statistical testing, sampling, confidence intervals, and metrics (e.g., precision/recall, BLEU, ROUGE, FID, etc.). (experience)
- Strong documentation skills, including the ability to write technical reports and present to non-technical audiences. (experience)
Preferred Qualifications
- Experience working with open-source evaluation tools like OpenEval, ELO-based ranking, or LLM-as-a-Judge frameworks. (experience)
- Familiarity with prompt engineering, few-shot or zero-shot evaluation techniques. (experience)
- Experience evaluating generative models (e.g., text generation, image generation). (experience)
- Prior contributions to ML benchmarks or public evaluations. (experience)
- Strong interpersonal skills. (experience)
Responsibilities
- Develop robust methodologies to assess the performance of foundation models (e.g., LLMs, vision-language models, etc.) across diverse tasks. Leverage LLMs as judges to perform subjective and open-ended model evaluations (e.g., for summarization, reasoning, or multimodal generation tasks). Build, curate, and lead evaluation datasets and benchmarks. Advanced proficiency in at least one scripting language, preferably Python. Collaborate with research, engineering, and product teams to define evaluation goals aligned with user experience and product quality. Conduct failure analysis and uncover edge cases to improve model robustness. Contribute to our tools and infrastructure to automate and scale evaluation processes.
Tags & Categories
Answer 10 quick questions to check your fit for Machine Learning - Data Scientist @ Apple.
10 Questions
~2 Minutes
Instant Score
Related Books and Jobs