AI Safety Policy Evaluator, Violence & Threats | Seattle Onsite

HandshakeSeattle, WAHAI Delivery Ops

Apply on company site

Opens the official Handshake careers page

Salary

$55 - $120 USD per hour

Job type

Full-time

Posted

2 Sept 2026

Status

Open · checked 7d ago

About this role

Role Overview

The AI Safety Policy Evaluator, Violence & Threats at Handshake is an onsite, full-time position based in Seattle, Washington, designed for professionals with a strong background in violent fiction, military response, crisis intervention, or trust and safety. This role involves reviewing user prompts, conversation history, and AI model responses to determine how artificial intelligence systems handle complex topics involving violence, weapons, and threats.

Key Responsibilities

Analyze user queries and AI model outputs that involve violence, weapons, threats, and dark fiction within the full context of the conversation.

Differentiate between safe contexts—such as fictional narratives, historical accounts, educational texts, and self-defense—and prompts that request real-world harm or actionable instructions.

Determine whether an AI model response provides genuine, high-risk real-world capabilities regardless of how the user framed the request.

Identify indicators of emotional distress, anger, or dark humor to separate them from credible threats.

Write clear rationales citing policy language and contextual details for ambiguous cases.

Create and test adversarial or borderline prompts to discover where models establish safety boundaries.

Report policy gaps, emerging edge cases, and contradictions to project leads and policy teams.

Participate in team calibration meetings to maintain consistent evaluation standards.

Requirements

Demonstrated depth in violent fiction or media, military or emergency response, crisis or threat assessment, trust and safety, content moderation, or related policy work.

Strong analytical writing skills capable of explaining nuanced judgment calls clearly and precisely.

Ability to separate personal viewpoints from established customer policies during repetitive tasks involving sensitive material.

Capacity to maintain focus, accuracy, and sound judgment while reviewing graphic content on a regular basis.

Nice to Have

Published or produced creative work featuring violence, such as novels, screenplays, comics, games, or fan fiction.

Professional experience in military, law enforcement, corrections, private security, or armed disciplines.

Training or background in crisis intervention, threat assessment, forensic psychology, or violence prevention.

Direct experience with firearms, martial arts, or other weapon disciplines as a practitioner or instructor.

Prior professional experience in AI evaluation, red teaming, data annotation, RLHF, or content moderation.

Familiarity evaluating responses from popular language models like ChatGPT, Claude, or Gemini.

Experience participating in calibration sessions, inter-rater agreement processes, or adjudication workflows.

Skills and Tools

Large Language Models (LLM)

Content policy evaluation and application

Threat assessment and crisis analysis

Adversarial prompt engineering

Salary and Benefits

The compensation for this role ranges from $55 to $120 per hour, with placement determined by individual experience. Employment is handled through TCWGlobal on a W-2 basis supporting Handshake AI. Eligible employees can access medical, dental, and vision insurance, flexible spending accounts, pretax commuter benefits, and a 401(k) retirement plan with an employer match. Additional offerings include voluntary term life, critical illness, and accident insurance, plus wellness and pet-related perks. Paid time off accrues at one hour per 40 hours worked up to annual caps, accompanied by statutory sick leave and 13 paid holidays.

About the Company

Handshake powers a massive career network connecting 25 million job seekers, over one million employers, and 1,600 educational institutions. Handshake AI collaborates directly with frontier AI lab researchers to design evaluations, publish benchmarks, and enhance artificial intelligence models using human expertise.

How to Apply

Interested candidates should visit the official company careers page to submit an application and review further instructions.

Skills and tech stack

LLM

Questions candidates ask

Where is this job located?

The role is onsite at Handshake's office in Seattle, Washington, operating Monday through Friday.

What is the pay rate for the AI Safety Policy Evaluator position?

The compensation ranges from $55 to $120 per hour, depending on the candidate's experience.

Is software engineering experience required?

No, software engineering or LLM product experience alone is not sufficient; the role requires non-engineering content-policy evaluation expertise in areas like violent fiction, military response, crisis assessment, or trust and safety.

What employment type is this role?

This is a full-time, W-2 assignment through TCWGlobal working 40 hours per week on a non-exempt basis.

Does this role involve viewing graphic content?

Yes, the position involves regular and deliberate engagement with graphic material, including depictions of violence, weapons, torture, abuse, threats, self-harm, and death, supported by structured evaluation frameworks and mental health resources.

Ready to apply?

The application is on the official Handshake careers page.

Apply on company site