Latest AI JobsBrowse Jobs
Anthropic logo

Research Engineer, Model Evaluations

AnthropicRemote-Friendly (Travel-Required) | San Francisco, CA | New York City, NYAI Research & Engineering
Apply on company site
Salary
Not listed
Job type
Full-time
Work mode
Remote
Posted
28 Apr 2026
Status
Open · checked 7d ago

About this role

Anthropic is currently seeking a Research Engineer for Model Evaluations to design critical assessments that measure and validate the capabilities of Claude. This remote-friendly position, with options in San Francisco or New York, is ideal for technical professionals passionate about building reliable artificial intelligence systems.

Role Overview

In this capacity, the professional will turn ambiguous ideas of machine intelligence into clear, defensible metrics that leadership, researchers, and the public can trust. The position involves designing evaluations across Claude's personality and capabilities, creating distributed infrastructure to run these assessments at scale, and partnering with researchers throughout training cycles to interpret live results.

Key Responsibilities

Design and execute evaluations measuring reasoning, safety properties, knowledge, and agentic behavior, while producing informative visualizations.
Build, harden, and scale distributed evaluation execution platforms to run reliably against checkpoints during reinforcement learning training runs.
Own monitoring dashboards for model health, reducing latency, improving signal-to-noise ratios, and preventing regressions.
Debug anomalous evaluation results quickly during active training runs to determine root causes.
Improve internal tooling, libraries, and workflows utilized by researchers for iterating on evaluations.
Run experiments testing how scaffolding, sampling, and prompting choices impact performance benchmarks.

Requirements

Strong programming skills in Python, including experience with production or research infrastructure.
Demonstrated experience building or operating reliable distributed systems, data pipelines, or large-scale infrastructure.
Clear verbal and written communication abilities for explaining technical findings to non-specialists.
Comfort operating in a production-support or on-call capacity during live training cycles.
A Bachelor's degree or equivalent combination of relevant education, training, and professional experience.
A dedicated interest in the societal impacts of artificial intelligence and safety.

Nice to Have

Hands-on experience utilizing large language models such as Claude for prompting, sampling, and scaffolding.
Background in statistics, experimental design, and data visualization with a track record of building trusted dashboards.
Experience developing evaluation metrics for language models or working with experiment-tracking and monitoring systems.
Experience with large-scale dataset sourcing, curation, and processing alongside ML training infrastructure.

Skills and Tools

Python
Distributed Systems
Data Pipelines
Machine Learning Infrastructure
Data Visualization and Dashboards
Experiment Tracking and Observability Tools

Salary and Benefits

Annual base salary ranging from $500,000 to $850,000 USD.
Comprehensive benefits package including competitive compensation, optional equity donation matching, and generous parental leave.
Flexible working hours and generous vacation policies.
Visa sponsorship provided where applicable.

About the Company

Anthropic is a public benefit corporation headquartered in San Francisco, working as a single cohesive team on large-scale research efforts. The organization focuses on creating reliable, interpretable, and steerable artificial intelligence systems that remain beneficial for society. The workplace operates under a location-based hybrid policy where staff spend at least 25 percent of their time in office.

How to Apply

Interested candidates should review the official Anthropic careers page to submit an application and learn more about current opportunities.

Skills and tech stack

Python

Questions candidates ask

What is the salary range for the Research Engineer role?

The annual base salary for this position ranges from $500,000 to $850,000 USD.

Where is the position located?

This role is remote-friendly with required travel, featuring office locations in San Francisco, CA, and New York City, NY.

Does Anthropic offer visa sponsorship?

Yes, Anthropic sponsors visas for eligible candidates and retains an immigration lawyer to assist with the process when an offer is made.

What are the primary programming requirements?

Candidates must have strong Python programming skills, including experience with research or production infrastructure.

What is the hybrid work policy?

Staff are generally expected to work from an office at least 25 percent of the time, though specific role requirements may vary.

Ready to apply?

The application is on the official Anthropic careers page.

Apply on company site