Latest AI JobsBrowse Jobs
Fireworks AI logo

Member of Technical Staff, LLM Infrastructure

Fireworks AISan MateoEngineering
Apply on company site
Salary
$175K - $220K USD
Job type
Full-time
Work mode
Remote
Posted
5 Nov 2025
Status
Open · checked 7d ago

About this role

Fireworks AI is seeking a Member of Technical Staff for LLM Infrastructure to design core systems for its generative AI platform, available in San Mateo or remotely. This full-time engineering position suits professionals with at least three years of software engineering experience focused on infrastructure or machine learning systems.

Role Overview

As a Software Engineer on the AI Infrastructure team, the selected professional will design core systems that drive the generative AI platform at Fireworks AI. The mission centers on making the platform reliable and user-friendly by building infrastructure and tools that guarantee reliability, performance, quality, and availability. The role involves close collaboration with the cloud infrastructure, product, and performance teams to deliver infrastructure linking customers with the proprietary inference engine.

Key Responsibilities

Design and develop scalable backend infrastructure supporting distributed training, inference, and data pipelines.
Build and maintain core backend services including the LLM CI/CD pipeline, control plane, and model serving systems.
Support performance optimization, cost efficiency, and reliability enhancements across compute, storage, and networking layers.
Construct frameworks and safeguards to secure industry-leading model quality.
Translate research and product needs into infrastructure solutions through cross-functional teamwork.
Take part in code reviews, technical discussions, and continuous integration and deployment workflows.

Requirements

Bachelor's degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.
Minimum of 3 years of software engineering experience focused on infrastructure or machine learning systems.
Strong programming abilities in Python, Go, or a similar language.
Demonstrated experience in machine learning infrastructure and tooling such as PyTorch, MLflow, Vertex AI, SageMaker, or Kubernetes.
Foundational understanding of LLM concepts including context length, disaggregated prefill, and KV cache memory estimation.

Nice to Have

Five or more years of software engineering experience centered on infrastructure or machine learning systems.
Familiarity with open-source inference engines like vLLM, Sglang, or TRT-LLM.
Past contributions to open-source infrastructure or machine learning projects.
Background in constructing large-scale machine learning and MLOps infrastructure.

Skills and Tools

Python
Go
PyTorch
Kubernetes
MLOps tools (MLflow, Vertex AI, SageMaker)
LLM technologies and inference engines

Salary and Benefits

Annual salary range between $175,000 and $220,000 USD.
Opportunities to solve complex challenges in low-latency inference and scalable model serving.
Collaboration with world-class engineers and AI researchers.

About the Company

Fireworks AI functions as a platform for specialized intelligence, allowing companies to build, train, and serve AI models tailored to proprietary data, workflows, and products. Founded by the team behind PyTorch and backed by prominent investors such as AMD, Benchmark Capital, Index Ventures, Lightspeed, NVIDIA, Sequoia Capital, and TCV, the Series D enterprise is valued at $17.5 billion and delivers production AI using hundreds of open models across multiple workloads.

How to Apply

Interested candidates should visit the official Fireworks AI careers page to submit an application for this position.

Skills and tech stack

PythonPyTorchKubernetesLLMMLOps

Questions candidates ask

What is the salary for the Member of Technical Staff, LLM Infrastructure role?

The salary ranges from $175,000 to $220,000 per year in USD.

Can this position be done remotely?

Yes, this role offers remote work flexibility alongside the San Mateo location option.

What are the minimum experience requirements?

Candidates need at least 3 years of software engineering experience with a focus on infrastructure or machine learning systems, plus a relevant bachelor's degree or equivalent practical experience.

What programming languages are required?

Strong programming skills in Python, Go, or a similar language are required.

What technical tools and frameworks are relevant to the position?

Relevant tools and frameworks include PyTorch, Kubernetes, MLflow, Vertex AI, SageMaker, and open-source inference engines like vLLM, Sglang, or TRT-LLM.

Ready to apply?

The application is on the official Fireworks AI careers page.

Apply on company site