Latest AI JobsBrowse Jobs
code-metal logo

Staff AI Platform Engineer

code-metalBoston HubEngineering
Apply on company site
Salary
$205K – $250K
Job type
Full-time
Work mode
Remote
Experience
Senior · 8+ yrs
Posted
6 Oct 2026
Status
Open · checked 7d ago

About this role

The Staff AI Platform Engineer role is based in the Boston Hub and suits an experienced technical leader focused on building foundational AI infrastructure and production-ready systems. This position involves guiding a four-person team to create the AI enablement stack used across various products.

Role Overview

Code Metal seeks a Staff AI Platform Engineer to act as the technical lead for a new four-person team in the Boston Hub. The position centers on designing, building, and operating production systems that support AI-driven code transpilation and mission planning. The role combines core platform engineering with data science and AI research fundamentals.

Key Responsibilities

Establish technical direction and architecture for the AI platform, author design documents and RFCs, guide build-versus-buy choices, and mentor team members.
Deploy, benchmark, and optimize production inference for open-weight models using TensorRT-LLM, vLLM, and SGLang.
Manage the model gateway providing access to self-hosted and commercial models with unified authentication, routing, failover, quotas, and cost tracking.
Create reusable agent harnesses and orchestration primitives for reliable, verifiable workflows.
Develop context-engineering services handling memory, retrieval, and data discovery.
Implement end-to-end service instrumentation using OpenTelemetry traces and metrics alongside experiment-tracking layers.
Design for product standard readiness from inception, including multi-tenancy, versioned APIs, security, and air-gapped deployments.

Requirements

Production-grade Python proficiency combined with strong platform engineering fundamentals covering API design, distributed systems, containers, Kubernetes, CI/CD, and testing.
Proven track record of shipping production agentic systems with clear understanding of reliability bottlenecks.
Experience in context engineering including retrieval-augmented generation, embeddings, vector or hybrid search, and agent memory.
Background in instrumenting services and operating AI services against service-level objectives.
Solid data science and AI research fundamentals regarding transformers, LLM inference, experiment design, benchmarking, and model evaluation, along with familiarity with PyTorch, Hugging Face, and fine-tuning or serving language models.
Staff-level technical leadership experience encompassing cross-system architecture ownership, RFC authoring, roadmap translation from ambiguous needs, and mentorship.

Nice to Have

Production experience operating an LLM gateway or API gateway handling routing, authentication, rate limiting, quotas, failover, and cost attribution.
Hands-on deployment and tuning of LLM inference engines like vLLM, SGLang, or TensorRT-LLM on GPU infrastructure.
Knowledge of inference optimization techniques such as speculative decoding, prefix caching, tensor/pipeline/expert parallelism, disaggregated prefill and decode, and GPU profiling.
Experience developing evaluation harnesses for LLMs and agents, plus experiment-tracking tools like MLflow or Weights & Biases.
Background in transitioning internal platforms into external products with multi-tenancy, SDKs, versioned APIs, and documentation.
Experience deploying AI systems on-premises, in air-gapped or classified environments, or within regulated sectors like defense and aerospace.

Skills and Tools

Python
PyTorch
Hugging Face
vLLM
SGLang
TensorRT-LLM
OpenTelemetry
Kubernetes
MLflow
Weights & Biases
Retrieval-Augmented Generation (RAG)

Salary and Benefits

Salary range from $205K to $250K with included equity offers
Comprehensive health care plan providing 100% premium coverage for medical, dental, and vision
401(k) retirement plan with a 5% match
Uncapped vacation policy alongside sick leave and public holidays
Flexible hybrid or remote work arrangement options
Relocation assistance available for qualifying employees

About the Company

Code Metal specializes in automated software engineering and verification. Founded in 2023 with offices located in Boston and San Francisco, the organization focuses on constraining AI and applying formal methods to generate provably correct code with auditable proof. Customers include the U.S. Air Force, L3Harris, RTX, and Toshiba, and financial backing is provided by investors such as Accel, Salesforce Ventures, B Capital, Smith Point Capital, J2 Ventures, Shield Capital, Overmatch, and RTX.

How to Apply

Interested candidates should apply directly on the official company careers page.

Skills and tech stack

PythonPyTorchHugging FacevLLMSGLangTensorRT-LLMOpenTelemetryKubernetesMLflowWeights & BiasesLLMLLMsTransformersFine-tuningDistributed Systems

Questions candidates ask

Where is the position located?

The role is located in the Boston Hub, with flexible hybrid or remote work arrangements available.

What is the salary and compensation structure for this role?

The salary ranges from $205K to $250K with additional equity offers.

What level of experience is required?

The position typically requires 8+ years of software engineering experience, including 4+ years building and operating ML or LLM systems in production at a Staff level.

What benefits are included with employment?

Benefits include 100% premium coverage for medical, dental, and vision health care plans, a 401(k) with a 5% match, uncapped vacation, flexible hybrid or remote work options, and relocation assistance for qualifying employees.

Is US citizenship required for the job?

US citizenship may be required for certain project assignments involving security clearance.

Ready to apply?

The application is on the official code-metal careers page.

Apply on company site