Latest AI JobsBrowse Jobs
Scale AI logo

Engineering Manager, Agent Oversight

Scale AISan Francisco, CA; New York, NYApplications Platform Engineering
Apply on company site
Salary
Not listed
Job type
Full-time
Posted
31 Oct 2025
Status
Open · checked 7d ago

About this role

Scale AI is hiring an Engineering Manager for Agent Oversight in San Francisco or New York to lead the team building platforms for monitoring and evaluating agentic applications. This opportunity suits an experienced engineering leader with a background in production machine learning and team management.

Role Overview

As the Engineering Manager for Agent Oversight, you will lead a multi-disciplinary team of software and machine learning engineers at Scale AI. You will own the platform roadmap spanning deployment, monitoring, evaluation, and machine learning-driven improvement of agentic applications. Your team will build the infrastructure that powers agentic AI in production for enterprise and government customers, helping to establish industry standards for agent oversight.

Key Responsibilities

Lead a multi-disciplinary team of software and machine learning engineers to drive technical delivery across the Scale Generative AI Platform.
Own the platform roadmap across deployment, monitoring, evaluation, and machine learning-driven improvement of agentic applications.
Collaborate cross-functionally with customers, forward deployed teams, product management, and internal engineering to translate requirements into platform capabilities.
Build and ship features end-to-end, ranging from system design through debugging and rigorous testing.
Drive high-velocity experimentation to validate and improve platform capabilities based on real customer usage.
Establish technical direction, culture, and processes for a fast-growing team.
Mentor and develop engineers and machine learning scientists while influencing organizational scaling.

Requirements

7+ years of engineering experience, including 2+ years directly managing engineers or machine learning engineers responsible for a production machine learning or LLM-powered system.
Hands-on familiarity with agent architectures such as tool use, planning, and multi-agent orchestration.
Ability to review machine learning experiment design or evaluation methodology to engage effectively with machine learning engineers and scientists.
Track record owning the full lifecycle of platform-level infrastructure from initial design through scaling across multiple teams.
Experience collaborating with product managers, forward deployed engineering teams, and customers to translate real-world requirements into prioritization decisions.
Proven history of building and growing high-performing engineering teams through hiring, retention, and output improvements.

Nice to Have

Experience building or overseeing evaluation, monitoring, or observability systems for machine learning and LLM-powered products in production.
Strong grasp of the full machine learning and agent development lifecycle from experimentation to production deployment and iteration.
Deep understanding of modern LLMs and agentic system design, including prompt- and system-level optimization and integration with external tools and APIs.
Published research, open-source contributions, or patents in agentic systems, LLMs, or applied machine learning.
Ability to operate in ambiguous problem spaces while balancing research-driven approaches with pragmatic product constraints.

Skills and Tools

LLM
Machine Learning Infrastructure
Agent Architectures
Platform Engineering

Salary and Benefits

Base salary range for this full-time position in San Francisco and New York is $248,400 to $310,500 USD, with exact placement determined by work location, skills, experience, and qualifications. Eligible roles include equity-based compensation subject to Board of Director approval. Comprehensive benefits include medical, dental, and vision coverage, retirement benefits, learning and development stipends, commuter stipends, and generous paid time off.

About the Company

Scale AI develops reliable artificial intelligence systems for the world's most important decisions. As a leading AI data foundry, the company provides high-quality data and full-stack technologies that power advanced models and fuel breakthroughs in generative AI, defense, and autonomous vehicles. Scale partners with leading enterprises and governments to bring dependable artificial intelligence into production.

How to Apply

Candidates should apply directly on the official Scale AI careers page.

Skills and tech stack

LLM

Questions candidates ask

Where is this position located?

This position is based in San Francisco, CA or New York, NY.

What is the salary range for this role?

The base salary range for this position in San Francisco and New York is $248,400 to $310,500 USD.

What experience is required for this engineering manager role?

Candidates need 7+ years of engineering experience, including 2+ years directly managing engineers or machine learning engineers responsible for a production machine learning or LLM-powered system.

What benefits are provided by Scale AI?

Scale AI offers comprehensive health, dental, and vision coverage, retirement benefits, learning and development stipends, commuter stipends, and generous paid time off.

Is remote work permitted for this role?

This is a US-based team with members located across New York and San Francisco.

Ready to apply?

The application is on the official Scale AI careers page.

Apply on company site