Principal Software Engineer, Catalog & Real-Time Serving Systems
Instacart
Principal Software Engineer, Catalog & Real-Time Serving Systems
Job Description
Responsibilities
- Provide architectural leadership for Catalog, streaming, and data-intensive systems, emphasizing ML serving infrastructure and best practices, and drive the technical roadmap.
- Design, build, and scale reliable, efficient, and adaptable solutions to address changing business and ML needs.
- Lead the development and optimization of ML serving endpoints, ensuring high availability, low latency, robust performance, and implement fail-fast input validations and track metrics using Datadog.
- Centralize ML serving logic and decouple it from product applications to improve debugging, manageability, and system performance.
- Drive and contribute to company-wide transformational initiatives, impacting key business metrics like revenue, personalization, and operational efficiency, and influence the direction of ML infrastructure including real-time inferencing.
- Serve as a subject matter expert for Catalog, streaming, data-intensive, and ML serving technologies, providing guidance and mentorship to engineering and data science teams.
- Identify and implement innovative solutions to optimize system performance, reduce costs, and improve data processing and ML serving latency.
- Collaborate with cross-functional teams, including Product, Retailer, IC App, Ads, ML Infrastructure, and Data Science, to deliver integrated ML-driven solutions, and lead incident response and resolution for high-severity issues.
Required Qualifications
- Extensive experience in software engineering, with a focus on distributed systems, streaming processing (e.g., Flink), data intensive applications, and particularly, Machine Learning serving and deployment.
- Proven track record of designing, implementing, and scaling large-scale, high-performance systems, including ML serving infrastructure.
- Deep understanding of database technologies, data modeling, data pipelines, and ML model deployment patterns.
- Strong architectural skills and the ability to design and evaluate complex technical solutions across diverse technology domains, including Catalog, Streaming, and Machine Learning.
- Excellent problem-solving and debugging skills, with specific experience in addressing issues related to ML model serving, data quality, and infrastructure stability.
- Strong communication and collaboration skills, with the ability to effectively work across teams, influence stakeholders, and mentor junior engineers.
- Experience with cloud platforms and related technologies, including ML serving platforms (e.g., Sagemaker).
- Ability to quantify and demonstrate the impact of technical contributions on business results (e.g., revenue, efficiency, cost savings, and ML model performance).
- Familiarity with challenges related to ML lifecycle, data flow, and best practices
Preferred Qualifications
- Experience working with large-scale catalog systems or similar data-intensive platforms.
- Significant experience in designing and implementing high-throughput, low-latency ML serving systems.
- Contributions to open-source projects or technical publications related to distributed systems, data engineering, or Machine Learning serving.
- Experience in a high-growth, fast-paced environment, particularly in the context of scaling ML initiatives.
Required Skills
- distributed systems
- streaming processing (e.g., Flink)
- data intensive applications
- Machine Learning serving and deployment
- database technologies
- data modeling
- data pipelines
- ML model deployment patterns
- architectural skills
- problem-solving
- debugging skills
- communication
- collaboration
- cloud platforms
- ML serving platforms (e.g., Sagemaker)
- Datadog
Benefits
- highly market-competitive compensation
- new hire equity grant as well as annual refresh grants
- benefits offerings
Locations
- United States, United States (Remote)
Salary
Estimated Salary Rangemedium confidence
320,000 - 500,000 USD / yearly
Source: ai estimated
* This is an estimated range based on market data and may vary based on experience and qualifications.
Skills Required
- distributed systemsintermediate
- streaming processing (e.g., Flink)intermediate
- data intensive applicationsintermediate
- Machine Learning serving and deploymentintermediate
- database technologiesintermediate
- data modelingintermediate
- data pipelinesintermediate
- ML model deployment patternsintermediate
- architectural skillsintermediate
- problem-solvingintermediate
- debugging skillsintermediate
- communicationintermediate
- collaborationintermediate
- cloud platformsintermediate
- ML serving platforms (e.g., Sagemaker)intermediate
- Datadogintermediate
Required Qualifications
- Extensive experience in software engineering, with a focus on distributed systems, streaming processing (e.g., Flink), data intensive applications, and particularly, Machine Learning serving and deployment. (experience)
- Proven track record of designing, implementing, and scaling large-scale, high-performance systems, including ML serving infrastructure. (experience)
- Deep understanding of database technologies, data modeling, data pipelines, and ML model deployment patterns. (experience)
- Strong architectural skills and the ability to design and evaluate complex technical solutions across diverse technology domains, including Catalog, Streaming, and Machine Learning. (experience)
- Excellent problem-solving and debugging skills, with specific experience in addressing issues related to ML model serving, data quality, and infrastructure stability. (experience)
- Strong communication and collaboration skills, with the ability to effectively work across teams, influence stakeholders, and mentor junior engineers. (experience)
- Experience with cloud platforms and related technologies, including ML serving platforms (e.g., Sagemaker). (experience)
- Ability to quantify and demonstrate the impact of technical contributions on business results (e.g., revenue, efficiency, cost savings, and ML model performance). (experience)
- Familiarity with challenges related to ML lifecycle, data flow, and best practices (experience)
Preferred Qualifications
- Experience working with large-scale catalog systems or similar data-intensive platforms. (experience)
- Significant experience in designing and implementing high-throughput, low-latency ML serving systems. (experience)
- Contributions to open-source projects or technical publications related to distributed systems, data engineering, or Machine Learning serving. (experience)
- Experience in a high-growth, fast-paced environment, particularly in the context of scaling ML initiatives. (experience)
Responsibilities
- Provide architectural leadership for Catalog, streaming, and data-intensive systems, emphasizing ML serving infrastructure and best practices, and drive the technical roadmap.
- Design, build, and scale reliable, efficient, and adaptable solutions to address changing business and ML needs.
- Lead the development and optimization of ML serving endpoints, ensuring high availability, low latency, robust performance, and implement fail-fast input validations and track metrics using Datadog.
- Centralize ML serving logic and decouple it from product applications to improve debugging, manageability, and system performance.
- Drive and contribute to company-wide transformational initiatives, impacting key business metrics like revenue, personalization, and operational efficiency, and influence the direction of ML infrastructure including real-time inferencing.
- Serve as a subject matter expert for Catalog, streaming, data-intensive, and ML serving technologies, providing guidance and mentorship to engineering and data science teams.
- Identify and implement innovative solutions to optimize system performance, reduce costs, and improve data processing and ML serving latency.
- Collaborate with cross-functional teams, including Product, Retailer, IC App, Ads, ML Infrastructure, and Data Science, to deliver integrated ML-driven solutions, and lead incident response and resolution for high-severity issues.
Benefits
- general: highly market-competitive compensation
- general: new hire equity grant as well as annual refresh grants
- general: benefits offerings
Tags & Categories
Principal Software Engineer, Catalog & Real-Time Serving Systems
Instacart
Principal Software Engineer, Catalog & Real-Time Serving Systems
Job Description
Responsibilities
- Provide architectural leadership for Catalog, streaming, and data-intensive systems, emphasizing ML serving infrastructure and best practices, and drive the technical roadmap.
- Design, build, and scale reliable, efficient, and adaptable solutions to address changing business and ML needs.
- Lead the development and optimization of ML serving endpoints, ensuring high availability, low latency, robust performance, and implement fail-fast input validations and track metrics using Datadog.
- Centralize ML serving logic and decouple it from product applications to improve debugging, manageability, and system performance.
- Drive and contribute to company-wide transformational initiatives, impacting key business metrics like revenue, personalization, and operational efficiency, and influence the direction of ML infrastructure including real-time inferencing.
- Serve as a subject matter expert for Catalog, streaming, data-intensive, and ML serving technologies, providing guidance and mentorship to engineering and data science teams.
- Identify and implement innovative solutions to optimize system performance, reduce costs, and improve data processing and ML serving latency.
- Collaborate with cross-functional teams, including Product, Retailer, IC App, Ads, ML Infrastructure, and Data Science, to deliver integrated ML-driven solutions, and lead incident response and resolution for high-severity issues.
Required Qualifications
- Extensive experience in software engineering, with a focus on distributed systems, streaming processing (e.g., Flink), data intensive applications, and particularly, Machine Learning serving and deployment.
- Proven track record of designing, implementing, and scaling large-scale, high-performance systems, including ML serving infrastructure.
- Deep understanding of database technologies, data modeling, data pipelines, and ML model deployment patterns.
- Strong architectural skills and the ability to design and evaluate complex technical solutions across diverse technology domains, including Catalog, Streaming, and Machine Learning.
- Excellent problem-solving and debugging skills, with specific experience in addressing issues related to ML model serving, data quality, and infrastructure stability.
- Strong communication and collaboration skills, with the ability to effectively work across teams, influence stakeholders, and mentor junior engineers.
- Experience with cloud platforms and related technologies, including ML serving platforms (e.g., Sagemaker).
- Ability to quantify and demonstrate the impact of technical contributions on business results (e.g., revenue, efficiency, cost savings, and ML model performance).
- Familiarity with challenges related to ML lifecycle, data flow, and best practices
Preferred Qualifications
- Experience working with large-scale catalog systems or similar data-intensive platforms.
- Significant experience in designing and implementing high-throughput, low-latency ML serving systems.
- Contributions to open-source projects or technical publications related to distributed systems, data engineering, or Machine Learning serving.
- Experience in a high-growth, fast-paced environment, particularly in the context of scaling ML initiatives.
Required Skills
- distributed systems
- streaming processing (e.g., Flink)
- data intensive applications
- Machine Learning serving and deployment
- database technologies
- data modeling
- data pipelines
- ML model deployment patterns
- architectural skills
- problem-solving
- debugging skills
- communication
- collaboration
- cloud platforms
- ML serving platforms (e.g., Sagemaker)
- Datadog
Benefits
- highly market-competitive compensation
- new hire equity grant as well as annual refresh grants
- benefits offerings
Locations
- United States, United States (Remote)
Salary
Estimated Salary Rangemedium confidence
320,000 - 500,000 USD / yearly
Source: ai estimated
* This is an estimated range based on market data and may vary based on experience and qualifications.
Skills Required
- distributed systemsintermediate
- streaming processing (e.g., Flink)intermediate
- data intensive applicationsintermediate
- Machine Learning serving and deploymentintermediate
- database technologiesintermediate
- data modelingintermediate
- data pipelinesintermediate
- ML model deployment patternsintermediate
- architectural skillsintermediate
- problem-solvingintermediate
- debugging skillsintermediate
- communicationintermediate
- collaborationintermediate
- cloud platformsintermediate
- ML serving platforms (e.g., Sagemaker)intermediate
- Datadogintermediate
Required Qualifications
- Extensive experience in software engineering, with a focus on distributed systems, streaming processing (e.g., Flink), data intensive applications, and particularly, Machine Learning serving and deployment. (experience)
- Proven track record of designing, implementing, and scaling large-scale, high-performance systems, including ML serving infrastructure. (experience)
- Deep understanding of database technologies, data modeling, data pipelines, and ML model deployment patterns. (experience)
- Strong architectural skills and the ability to design and evaluate complex technical solutions across diverse technology domains, including Catalog, Streaming, and Machine Learning. (experience)
- Excellent problem-solving and debugging skills, with specific experience in addressing issues related to ML model serving, data quality, and infrastructure stability. (experience)
- Strong communication and collaboration skills, with the ability to effectively work across teams, influence stakeholders, and mentor junior engineers. (experience)
- Experience with cloud platforms and related technologies, including ML serving platforms (e.g., Sagemaker). (experience)
- Ability to quantify and demonstrate the impact of technical contributions on business results (e.g., revenue, efficiency, cost savings, and ML model performance). (experience)
- Familiarity with challenges related to ML lifecycle, data flow, and best practices (experience)
Preferred Qualifications
- Experience working with large-scale catalog systems or similar data-intensive platforms. (experience)
- Significant experience in designing and implementing high-throughput, low-latency ML serving systems. (experience)
- Contributions to open-source projects or technical publications related to distributed systems, data engineering, or Machine Learning serving. (experience)
- Experience in a high-growth, fast-paced environment, particularly in the context of scaling ML initiatives. (experience)
Responsibilities
- Provide architectural leadership for Catalog, streaming, and data-intensive systems, emphasizing ML serving infrastructure and best practices, and drive the technical roadmap.
- Design, build, and scale reliable, efficient, and adaptable solutions to address changing business and ML needs.
- Lead the development and optimization of ML serving endpoints, ensuring high availability, low latency, robust performance, and implement fail-fast input validations and track metrics using Datadog.
- Centralize ML serving logic and decouple it from product applications to improve debugging, manageability, and system performance.
- Drive and contribute to company-wide transformational initiatives, impacting key business metrics like revenue, personalization, and operational efficiency, and influence the direction of ML infrastructure including real-time inferencing.
- Serve as a subject matter expert for Catalog, streaming, data-intensive, and ML serving technologies, providing guidance and mentorship to engineering and data science teams.
- Identify and implement innovative solutions to optimize system performance, reduce costs, and improve data processing and ML serving latency.
- Collaborate with cross-functional teams, including Product, Retailer, IC App, Ads, ML Infrastructure, and Data Science, to deliver integrated ML-driven solutions, and lead incident response and resolution for high-severity issues.
Benefits
- general: highly market-competitive compensation
- general: new hire equity grant as well as annual refresh grants
- general: benefits offerings
Tags & Categories