Senior | Staff Software Engineer - AI / ML
About this role
Snorkel AI builds frontier AI data labs, generating datasets, benchmarks, evaluations, and custom solutions for real-world artificial intelligence systems. As an early member of the machine learning and research engineering team, the incoming engineer will focus on making long-running agent rollouts and evaluations faster, cheaper, and more rigorous. This position exists to shape the discipline's direction and technical standards as the company scales following its recent Series E funding round.
The successful candidate will study how frontier-grade data is generated and evaluated, form hypotheses, validate them against real production data, and ship the winning solutions at scale. This involves building efficient agentic evaluations, AI model routers, fine-tuned small models, and predictive difficulty systems. Ideal applicants combine strong software engineering fundamentals with deep statistical grounding and practical experience managing production workloads.
What you will do
- Design and implement methods to reduce the cost of long-horizon agent evaluation through adaptive sampling, caching, and model cascades.
- Route evaluation and judge calls to the most cost-effective model that clears the quality bar.
- Fine-tune and serve open-weight models using parameter-efficient methods when they match frontier quality.
- Build predictive models that estimate task difficulty for frontier systems prior to running rollouts.
- Create golden datasets and quantify the accuracy and calibration of LLM-as-judge systems.
- Translate research prototypes into reusable components deployed across various projects.
What you need
- Bring five or more years of experience building production machine learning or software systems with end-to-end ownership.
- Demonstrate hands-on experience running language model or machine learning workloads in production environments.
- Possess a deep grounding in statistics and experimentation, including hypothesis testing and sampling.
- Maintain strong Python and software engineering fundamentals covering testing, code review, and system design.
- Show experience designing evaluations and interpreting results rigorously.
- Exhibit a history of identifying high-impact problems independently and communicating clearly with cross-functional partners.
Nice to have
- Fine-tuning and serving open-weight models with judgment on smaller model efficacy
- Building evaluation platforms, model gateways, or routing systems
- Experience with agentic workloads, benchmarks, or reinforcement learning environments
- A record of taking research into production through publications, open-source work, or shipped features
- MS or PhD in Computer Science, Machine Learning, Statistics, or a related field
Skills and tech stack
Questions candidates ask
What is the work location policy for this role?
This position is based in San Francisco, CA with a hybrid work arrangement.
What is the salary range for the position?
The salary range for this role is $208,000 to $315,000 USD. Actual compensation is determined based on factors including skills, qualifications, experience, and geographic location.
What kind of technical problems will I work on?
You will work on efficient agentic evaluations, AI model routing, fine-tuned small models, predictive difficulty systems, and measurement for AI data. Every problem on this team is an open machine learning or language model problem.