Latest AI JobsBrowse Jobs
Fireworks AI logo

Member of Technical Staff, Evals Platform

Fireworks AISan MateoEngineering
Apply on company site
Salary
$210K - $320K USD
Job type
Full-time
Work mode
Remote
Posted
30 Oct 2025
Status
Open · checked 7d ago

About this role

Fireworks AI is seeking a Member of Technical Staff for the Evals and Post-Training Product team to help developers improve models on the platform. This engineering role is based in San Mateo or available remotely, perfectly suited for professionals skilled in scalable system design and data science.

Role Overview

This position sits at the intersection of scalable system design, deep data science, and model quality. The professional chosen for this role will build the infrastructure and workflows that seamlessly connect evaluation and post-training. The current evaluation setup needs to evolve to support enhanced programmatic access and scale across backend systems, sandbox infrastructure, and user-facing surfaces.

Key Responsibilities

Scale Eval Infrastructure by designing systems that eliminate single-host coordination bottlenecks, resolve log-syncing latency, and establish seamless programmatic access.
Act as a metrics obsessive by tracking state-of-the-art benchmarks, reading research papers, resolving data discrepancies, and rigorously reproducing published results internally.
Pioneer and construct brand-new benchmarks to measure model performance across complex, emerging, and domain-specific use cases.
Own fine-tuning product experiences by building and enhancing workflows for SFT, RFT, and related model-improvement capabilities.
Collaborate closely with customers and internal stakeholders to understand evaluation and fine-tuning needs, triage issues, and turn bespoke workflows into reusable solutions.

Requirements

Possess 1 to 7 years of software engineering or data science experience, with hiring open at multiple levels.
Demonstrate strong system design skills to architect scalable, programmatic systems and modernize legacy infrastructure.
Bring hands-on experience working with sandbox infrastructure and secure environments for code execution and testing.
Apply an analytical and data science mindset with deep knowledge of LLM evaluations, metric design, and data-driven model improvement.
Understand the end-to-end generative AI lifecycle, ranging from base model prompting to dataset curation, fine-tuning, and production agents.

Nice to Have

Bring 3 or more years of software engineering or applied data science experience.
Familiarity with the Harbor framework or similar container registry and orchestration tools.
Interest in public writing and analysis, such as publishing technical blogs or whitepapers detailing model performance insights.
Knowledge of inference and hardware constraints, including GPU limits and inference optimization techniques.
Startup DNA characterized by comfort in fast-paced environments and taking end-to-end feature ownership.

Skills and Tools

PyTorch
Large Language Models (LLMs)
Scalable System Design
Sandbox Infrastructure
Data Science and Analytics
Container Registry and Orchestration Tools
Fine-Tuning Workflows (SFT, RFT)

Salary and Benefits

Annual salary range of USD 210,000 to USD 320,000.

About the Company

Fireworks AI is a platform for specialized intelligence that empowers companies to build, train, and serve AI models tailored to specific data, workflows, and products. Founded by the team behind PyTorch and backed by prominent investors like AMD, Benchmark, Index Ventures, Lightspeed, NVIDIA, Sequoia Capital, and TCV, the Series D company is valued at $17.5 billion. Fireworks AI powers production artificial intelligence with numerous state-of-the-art open models spanning text, image, embedding, audio, and multimodal workloads.

How to Apply

Interested candidates should visit the official company careers page at fireworks.ai to submit an application.

Skills and tech stack

PyTorchLLM

Questions candidates ask

What is the salary range for the Member of Technical Staff position?

The annual salary for this role ranges from USD 210,000 to USD 320,000.

Is this position remote?

Yes, this role offers remote flexibility while also being based out of San Mateo.

How many years of experience are required?

The position requires between 1 to 7 years of software engineering or data science experience, as the company hires at multiple levels.

What department does this role belong to?

This role is part of the Engineering department.

Who backs Fireworks AI?

Fireworks AI is backed by AMD, Atreides, Benchmark Capital, Index Ventures, Lightspeed, NVIDIA, Sequoia Capital, and TCV.

Ready to apply?

The application is on the official Fireworks AI careers page.

Apply on company site