Senior Software Engineer II - Ads Quality
About this role
Building the infrastructure that powers ads quality at Instacart involves designing and deploying machine learning systems that optimize the ads serving funnel from retrieval to pricing. Engineering professionals in this group work alongside software and machine learning teams to develop foundational solutions, including out-of-the-box large language models and bespoke foundation models. Responsibilities also include eliminating bottlenecks across training and serving pipelines to improve stability, latency, and cost efficiency.
This role exists to support the Ads Quality team in delivering precise response prediction, advertiser optimization, and advanced auction processes using terabytes of data. Engineers shape the technical direction for data, feature processing, and model training and serving solutions, specifically focusing on a new real-time sequential foundation model for ads. Ideal candidates bring solid experience in building production-grade ML infrastructure and thrive in collaborative, cross-functional environments.
What you will do
- Design, develop, and deploy machine learning infrastructure to tackle practical challenges in a complex marketplace while advancing the training and serving platform.
- Work with the company-wide ML Foundation team to develop solutions that meet unique ads requirements.
- Optimize training and serving funnel bottlenecks by profiling real systems to drive wins in throughput, cost, and latency.
- Identify system instabilities and shortcomings to deliver high availability and robust operating procedures.
- Build real-time feature processing infrastructure, including low-latency pipelines that compute, join, and serve fresh features on the critical path.
- Lead the infrastructure foundation for real-time sequential foundation models, covering online feature stores, streaming update paths, and training pipelines.
- Collaborate with product managers, data scientists, and machine learning engineers to co-design systems where constraints are solved together.
- Mentor engineers, set technical direction, and conduct thorough code reviews to ensure high maintainability.
What you need
- Hold a graduate degree in computer science, artificial intelligence, machine learning, operations research, or equivalent self-study and experience.
- Bring at least 3 years of industry experience building ML infrastructure or solving real-world problems with large datasets.
- Demonstrate deep expertise in building, scaling, and managing ML infrastructure for training or serving in production.
- Possess strong programming skills in software engineering languages such as Python and Go.
- Have experience in data manipulation using tools like SQL, Spark, and Pandas, alongside machine learning frameworks including Torch, TensorFlow, and XGBoost.
- Exhibit strong analytical, problem-solving, and cross-functional communication abilities.
Nice to have
- 5+ years of industry experience tech leading a team to build ML infrastructure or using machine learning to solve real-world problems with large datasets
- Demonstrated experience solving complex model training and serving efficiency challenges
- Hands-on experience with real-time and online feature processing and serving systems like Kafka and Flink
- Experience with sequential modeling, transformer architecture, generative retrieval, or large-scale online recommendation systems
- Familiarity with digital advertising platforms, LLM integrations, and prompt engineering
Benefits
- New hire equity grant
- Annual equity refresh grants
- Highly market-competitive compensation
Skills and tech stack
Questions candidates ask
What is the remote work policy?
Instacart operates as a Flex First team, giving employees the flexibility to choose where they do their best work whether from home, an office, or a coffee shop, while staying connected through regular in-person events.
What is the salary range for the role?
For candidates based in California, New York, Connecticut, and New Jersey, the base pay range is $230,000 to $242,500 USD. Offers may vary based on factors such as candidate experience and permanent work location.
What kind of models and technology will I be working with?
You will own technical direction for data, feature processing, and model training solutions, ranging from utilizing out-of-the-box LLMs to building bespoke Foundation Models. A major upcoming focus is building the infrastructure for a real-time sequential foundation model for ads.