Member of Technical Staff, Evals

HandshakeSan Francisco, CAHAI Research

Apply on company site

Opens the official Handshake careers page

Salary

$200K – $350K

Job type

Full-time

Posted

29 Sept 2026

Status

Open · checked 7d ago

About this role

Role Overview

The Member of Technical Staff, Evals position at Handshake involves designing how advanced artificial intelligence systems are measured, analyzed, and enhanced. Based in San Francisco, California, this full-time opportunity within the HAI Research department is ideal for researchers and builders who want to advance the AI economy by transforming human expertise into actionable data.

Key Responsibilities

Create evaluation frameworks, methodologies, and benchmarks for frontier large language models, multimodal models, AI agents, and reinforcement-learning environments.

Build feedback systems such as programmatic verifiers, reward models, and graders to improve and measure model behaviors.

Investigate how to make evaluations reliable, difficult, representative, and resilient against reward hacking or shortcutting.

Construct systems for superior human data, covering expert task design, data-quality signals, data-attribution techniques, and annotation methodologies.

Execute rapid, rigorous iteration cycles by prototyping, evaluating, interpreting outcomes, analyzing failure points, and converting insights into new benchmarks.

Share findings publicly through research, technical writing, open-source software, and benchmarks.

Requirements

Hold a PhD in machine learning, artificial intelligence, computer science, data science, or a related discipline, or possess equivalent industry research experience.

Record of publications at premier artificial intelligence and machine learning conferences such as NeurIPS, ICML, ICLR, and COLM.

Demonstrated builder mindset with experience tinkering with agents and delivering quality software, datasets, or benchmarks, evidenced by a strong GitHub profile or product portfolio.

Proficiency in Python, along with experience creating scalable software and working with agents.

Deep understanding of frontier artificial intelligence, including evaluation techniques, benchmarks, post-training recipes, agent harnesses, and data shapes.

Ability to thrive with high ownership in a fast-paced, ambiguous setting.

Skills and Tools

Python

AI Benchmarks and Evaluation Frameworks

Agent Harnesses and Post-Training Recipes

Reward Models and Programmatic Verifiers

Scalable Software Development

Salary and Benefits

Annual salary ranging from $200,000 to $350,000 USD.

Equity ownership in a rapidly expanding company.

Comprehensive health coverage including medical, dental, and vision, plus mental health support and a $500 wellness stipend.

Financial security through a 401(k) match, competitive pay, and financial coaching.

Family benefits featuring paid parental leave, fertility support, and parental coaching.

Professional development supported by a $2,000 learning stipend and ongoing growth programs.

Office perks in San Francisco including free lunch, gym access, and commuting support.

Time off policies featuring flexible paid time off, 15 holidays, and 2 flex days, alongside team outings, referral bonuses, and a $2,000 learning stipend.

About the Company

Handshake focuses on organizing expert human knowledge to propel the artificial intelligence economy forward. Collaborating directly with frontier laboratories on critical data, evaluation, and post-training challenges, the organization builds systems that convert expert insights into valuable data. Team members work alongside engineers and researchers from prominent labs and organizations, helping scale a data business that grew significantly within its first year.

How to Apply

Interested candidates should visit the official Handshake careers page on joinhandshake.com to submit an application for the Member of Technical Staff, Evals role.

Skills and tech stack

Python LLMs Reinforcement Learning

Questions candidates ask

What is the salary range for the Member of Technical Staff, Evals role?

The annual salary for this position ranges from $200,000 to $350,000 USD.

Where is the job located?

The role is preferentially located in San Francisco, CA, though exceptional candidates in other locations are also considered.

What qualifications are required?

Candidates need a PhD in ML, AI, computer science, data science, or related fields (or equivalent industry research experience), publications at top AI venues, strong Python skills, experience with agents and scalable software, and deep knowledge of frontier AI evaluations.

What benefits are provided for full-time US employees?

Perks include equity, a 401(k) match, medical/dental/vision coverage, mental health support, a $500 wellness stipend, a $2,000 learning stipend, paid parental leave, flexible PTO, 15 holidays plus 2 flex days, and free lunch and gym access at the San Francisco office.

How can I apply for this position?

You can apply directly through the official Handshake careers page at joinhandshake.com.

Ready to apply?

The application is on the official Handshake careers page.

Apply on company site