MDM Engineer
Amgen
MDM Engineer
Job Description
ABOUT AMGEN
What you will do
- Develop distributed data pipelines using PySpark on Databricks for ingesting, transforming, and publishing master data
- Write optimized SQL for large-scale data processing, including complex joins, window functions, and CTEs for MDM logic
- Implement match/merge algorithms and survivorship rules using Informatica MDM or Reltio APIs
- Build and maintain Delta Lake tables with schema evolution and versioning for master data domains
- Use AWS services like S3, Glue, Lambda, and Step Functions for orchestrating MDM workflows
- Automate data quality checks using IDQ or custom PySpark validators with rule-based profiling
- Integrate external enrichment sources (e.g., D&B, LexisNexis) via REST APIs and batch pipelines
- Design and deploy CI/CD pipelines using GitHub Actions or Jenkins for Databricks notebooks and jobs
- Monitor pipeline health using Databricks Jobs API, CloudWatch, and custom logging frameworks
- Implement fine-grained access control using Unity Catalog and attribute-based policies for MDM datasets
- Use MLflow for tracking model-based entity resolution experiments if ML-based matching is applied
- Collaborate with data stewards to expose curated MDM views via REST endpoints or Delta Sharing
What we expect of you
- 8 to 13 years of experience in Business, Engineering, IT or related field
Must-Have Skills
- Advanced proficiency in PySpark for distributed data processing and transformation
- Strong SQL skills for complex data modeling, cleansing, and aggregation logic
- Hands-on experience with Databricks including Delta Lake, notebooks, and job orchestration
- Deep understanding of MDM concepts including match/merge, survivorship, and golden record creation
- Experience with MDM platforms like Informatica MDM or Reltio, including REST API integration
- Proficiency in AWS services such as S3, Glue, Lambda, Step Functions, and IAM
- Familiarity with data quality frameworks and tools like Informatica IDQ or custom rule engines
- Experience building CI/CD pipelines for data workflows using GitHub Actions, Jenkins, or similar
- Knowledge of schema evolution, versioning, and metadata management in data lakes
- Ability to implement lineage and observability using Unity Catalog or third-party tools
- Comfort with Unix shell scripting or Python for orchestration and automation
- Hands on experience on RESTful APIs for ingesting external data sources and enrichment feeds
- Experience with Tableau or PowerBI for reporting MDM insights
- Exposure to Agile practices and tools (JIRA, Confluence)
- Prior experience in Pharma/Life Sciences
- Understanding of compliance and regulatory considerations in master data
Compensation
8–12
Locations
- Hyderabad, India
Salary
Estimated Salary Rangehigh confidence
50,000 - 80,000 USD / yearly
Source: xAI estimated
* This is an estimated range based on market data and may vary based on experience and qualifications.
Skills Required
- Advanced proficiency in PySpark for distributed data processing and transformationintermediate
- Strong SQL skills for complex data modeling, cleansing, and aggregation logicintermediate
- Hands-on experience with Databricks including Delta Lake, notebooks, and job orchestrationintermediate
- Deep understanding of MDM concepts including match/merge, survivorship, and golden record creationintermediate
- Experience with MDM platforms like Informatica MDM or Reltio, including REST API integrationintermediate
- Proficiency in AWS services such as S3, Glue, Lambda, Step Functions, and IAMintermediate
- Familiarity with data quality frameworks and tools like Informatica IDQ or custom rule enginesintermediate
- Experience building CI/CD pipelines for data workflows using GitHub Actions, Jenkins, or similarintermediate
- Knowledge of schema evolution, versioning, and metadata management in data lakesintermediate
- Ability to implement lineage and observability using Unity Catalog or third-party toolsintermediate
- Comfort with Unix shell scripting or Python for orchestration and automationintermediate
- Hands on experience on RESTful APIs for ingesting external data sources and enrichment feedsintermediate
- Experience with Tableau or PowerBI for reporting MDM insightsintermediate
- Exposure to Agile practices and tools (JIRA, Confluence)intermediate
- Prior experience in Pharma/Life Sciencesintermediate
- Understanding of compliance and regulatory considerations in master dataintermediate
Required Qualifications
- 8 to 13 years of experience in Business, Engineering, IT or related field (experience)
Responsibilities
- Develop distributed data pipelines using PySpark on Databricks for ingesting, transforming, and publishing master data
- Write optimized SQL for large-scale data processing, including complex joins, window functions, and CTEs for MDM logic
- Implement match/merge algorithms and survivorship rules using Informatica MDM or Reltio APIs
- Build and maintain Delta Lake tables with schema evolution and versioning for master data domains
- Use AWS services like S3, Glue, Lambda, and Step Functions for orchestrating MDM workflows
- Automate data quality checks using IDQ or custom PySpark validators with rule-based profiling
- Integrate external enrichment sources (e.g., D&B, LexisNexis) via REST APIs and batch pipelines
- Design and deploy CI/CD pipelines using GitHub Actions or Jenkins for Databricks notebooks and jobs
- Monitor pipeline health using Databricks Jobs API, CloudWatch, and custom logging frameworks
- Implement fine-grained access control using Unity Catalog and attribute-based policies for MDM datasets
- Use MLflow for tracking model-based entity resolution experiments if ML-based matching is applied
- Collaborate with data stewards to expose curated MDM views via REST endpoints or Delta Sharing
Tags & Categories
Answer 10 quick questions to check your fit for MDM Engineer @ Amgen.
10 Questions
~2 Minutes
Instant Score
Related Books and Jobs
MDM Engineer
Amgen
MDM Engineer
Job Description
ABOUT AMGEN
What you will do
- Develop distributed data pipelines using PySpark on Databricks for ingesting, transforming, and publishing master data
- Write optimized SQL for large-scale data processing, including complex joins, window functions, and CTEs for MDM logic
- Implement match/merge algorithms and survivorship rules using Informatica MDM or Reltio APIs
- Build and maintain Delta Lake tables with schema evolution and versioning for master data domains
- Use AWS services like S3, Glue, Lambda, and Step Functions for orchestrating MDM workflows
- Automate data quality checks using IDQ or custom PySpark validators with rule-based profiling
- Integrate external enrichment sources (e.g., D&B, LexisNexis) via REST APIs and batch pipelines
- Design and deploy CI/CD pipelines using GitHub Actions or Jenkins for Databricks notebooks and jobs
- Monitor pipeline health using Databricks Jobs API, CloudWatch, and custom logging frameworks
- Implement fine-grained access control using Unity Catalog and attribute-based policies for MDM datasets
- Use MLflow for tracking model-based entity resolution experiments if ML-based matching is applied
- Collaborate with data stewards to expose curated MDM views via REST endpoints or Delta Sharing
What we expect of you
- 8 to 13 years of experience in Business, Engineering, IT or related field
Must-Have Skills
- Advanced proficiency in PySpark for distributed data processing and transformation
- Strong SQL skills for complex data modeling, cleansing, and aggregation logic
- Hands-on experience with Databricks including Delta Lake, notebooks, and job orchestration
- Deep understanding of MDM concepts including match/merge, survivorship, and golden record creation
- Experience with MDM platforms like Informatica MDM or Reltio, including REST API integration
- Proficiency in AWS services such as S3, Glue, Lambda, Step Functions, and IAM
- Familiarity with data quality frameworks and tools like Informatica IDQ or custom rule engines
- Experience building CI/CD pipelines for data workflows using GitHub Actions, Jenkins, or similar
- Knowledge of schema evolution, versioning, and metadata management in data lakes
- Ability to implement lineage and observability using Unity Catalog or third-party tools
- Comfort with Unix shell scripting or Python for orchestration and automation
- Hands on experience on RESTful APIs for ingesting external data sources and enrichment feeds
- Experience with Tableau or PowerBI for reporting MDM insights
- Exposure to Agile practices and tools (JIRA, Confluence)
- Prior experience in Pharma/Life Sciences
- Understanding of compliance and regulatory considerations in master data
Compensation
8–12
Locations
- Hyderabad, India
Salary
Estimated Salary Rangehigh confidence
50,000 - 80,000 USD / yearly
Source: xAI estimated
* This is an estimated range based on market data and may vary based on experience and qualifications.
Skills Required
- Advanced proficiency in PySpark for distributed data processing and transformationintermediate
- Strong SQL skills for complex data modeling, cleansing, and aggregation logicintermediate
- Hands-on experience with Databricks including Delta Lake, notebooks, and job orchestrationintermediate
- Deep understanding of MDM concepts including match/merge, survivorship, and golden record creationintermediate
- Experience with MDM platforms like Informatica MDM or Reltio, including REST API integrationintermediate
- Proficiency in AWS services such as S3, Glue, Lambda, Step Functions, and IAMintermediate
- Familiarity with data quality frameworks and tools like Informatica IDQ or custom rule enginesintermediate
- Experience building CI/CD pipelines for data workflows using GitHub Actions, Jenkins, or similarintermediate
- Knowledge of schema evolution, versioning, and metadata management in data lakesintermediate
- Ability to implement lineage and observability using Unity Catalog or third-party toolsintermediate
- Comfort with Unix shell scripting or Python for orchestration and automationintermediate
- Hands on experience on RESTful APIs for ingesting external data sources and enrichment feedsintermediate
- Experience with Tableau or PowerBI for reporting MDM insightsintermediate
- Exposure to Agile practices and tools (JIRA, Confluence)intermediate
- Prior experience in Pharma/Life Sciencesintermediate
- Understanding of compliance and regulatory considerations in master dataintermediate
Required Qualifications
- 8 to 13 years of experience in Business, Engineering, IT or related field (experience)
Responsibilities
- Develop distributed data pipelines using PySpark on Databricks for ingesting, transforming, and publishing master data
- Write optimized SQL for large-scale data processing, including complex joins, window functions, and CTEs for MDM logic
- Implement match/merge algorithms and survivorship rules using Informatica MDM or Reltio APIs
- Build and maintain Delta Lake tables with schema evolution and versioning for master data domains
- Use AWS services like S3, Glue, Lambda, and Step Functions for orchestrating MDM workflows
- Automate data quality checks using IDQ or custom PySpark validators with rule-based profiling
- Integrate external enrichment sources (e.g., D&B, LexisNexis) via REST APIs and batch pipelines
- Design and deploy CI/CD pipelines using GitHub Actions or Jenkins for Databricks notebooks and jobs
- Monitor pipeline health using Databricks Jobs API, CloudWatch, and custom logging frameworks
- Implement fine-grained access control using Unity Catalog and attribute-based policies for MDM datasets
- Use MLflow for tracking model-based entity resolution experiments if ML-based matching is applied
- Collaborate with data stewards to expose curated MDM views via REST endpoints or Delta Sharing
Tags & Categories
Answer 10 quick questions to check your fit for MDM Engineer @ Amgen.
10 Questions
~2 Minutes
Instant Score
Related Books and Jobs