Latest AI JobsBrowse Jobs
AfterQuery logo

Research Scientist – Frontier Evaluations

AfterQuerySan FranciscoResearch
Apply on company site
Salary
$210K – $450K
Job type
Full-time
Experience
Senior
Posted
10 Sept 2026
Status
Open · checked 7d ago

About this role

San Francisco is the location for the Research Scientist – Frontier Evaluations position at AfterQuery, a full-time role focused on designing and publishing rigorous benchmarks for advanced artificial intelligence systems. This opportunity suits experienced researchers with a publication record who want to build evaluation infrastructure for foundation model development.

Role Overview

AfterQuery is hiring Research Scientists in San Francisco to create and release evaluations covering agentic, coding, and safety domains, alongside expert applications in fields such as healthcare, STEM, and finance. Selected candidates take ownership of the evaluation lifecycle from initial scoping to public release while collaborating with internal and external experts to address critical model capability gaps.

Key Responsibilities

Drive the complete lifecycle of frontier artificial intelligence benchmarks, covering design, validation, deployment, and ongoing refinement.
Coordinate alongside researchers and domain specialists to build assessments targeting significant model failures and coverage limitations.
Evaluate model performance and failure characteristics using structured experiments and statistical techniques.
Construct reproducible testing systems, grading frameworks, and benchmark infrastructure.
Partner with research teams on model post-training to quantify capability improvements.
Share findings via technical publications, articles, and benchmark reports.

Requirements

Demonstrated history of authoring research papers or publishing benchmarks.
Solid scientific writing capabilities and clear technical communication.
Adherence to strict experimental standards, incorporating baselines, contamination controls, ablations, and statistical verification.
Capacity to transition vague evaluation objectives into structured, reproducible public releases.
Expertise focused on safety, coding, and agentic evaluations, or applied machine learning within specialized domains.

Nice to Have

A PhD in a relevant technical discipline.
Prior research papers accepted at prominent peer-reviewed journals or conferences.
Enthusiasm for multidisciplinary investigation and blending techniques across artificial intelligence, engineering, and specialized sectors.

Skills and Tools

Agentic Evaluations
Coding Evaluations
Safety Evaluations
Applied Machine Learning
Benchmark Infrastructure
Statistical Methods

Salary and Benefits

Compensation range of $210K to $450K, plus equity and bonus offerings.
Health coverage including medical, dental, and vision insurance.
Retirement planning via a 401(k) plan featuring an employer match.
Daily meals provided through an UberEats stipend.
Monthly wellness stipend.

About the Company

AfterQuery operates as an applied research lab in San Francisco that curates data solutions to support foundation model development. Backed by investors such as Altos Ventures, BoxGroup, and Y Combinator, the organization supplies data to organizations building foundational artificial intelligence systems.

How to Apply

Interested candidates should apply on the official company careers page.

Skills and tech stack

Machine Learning

Questions candidates ask

Where is the job located?

The position is based in San Francisco.

What is the salary and compensation structure?

The salary ranges from $210K to $450K, with additional offerings in equity and bonuses.

What background is preferred for this role?

A PhD in a related technical field and research publications at leading conferences or peer-reviewed journals are preferred.

What benefits are provided to eligible employees?

Benefits include medical, vision, and dental health insurance, a 401(k) with employer match, a daily UberEats stipend for meals, and a monthly wellness stipend.

How should candidates submit their application?

Candidates must apply on the official company careers page.

Ready to apply?

The application is on the official AfterQuery careers page.

Apply on company site