Senior Software Engineer — Infra Agent Systems
About this role
Engineers who thrive on building foundational platforms and solving complex distributed systems problems will find a natural home within the Infra Agent Systems team at Together AI. Operating one of the largest GPU fleets globally, the group develops production AI agents alongside the core platforms that power them to automate infrastructure diagnostics, incident investigation, and operational workflows. Team members design knowledge graphs, search and retrieval systems, and orchestration frameworks that enable autonomous agents to reason, act, and continuously improve real-world AI infrastructure. The role exists to bridge the gap between AI agents and massive-scale hardware operations, ensuring systems are trustworthy enough to manage live production environments. Ideal candidates bring deep backend expertise and a strong desire to shape the future of autonomous cloud operations through robust software engineering.
Working across two main pillars, developers spend half their time building agent systems that interact with infrastructure teams through APIs, CLIs, dashboards, and chat interfaces. The remaining effort focuses on the underlying core platform, including agent orchestration, evaluation loops, and telemetry integration. Success in the position requires taking end-to-end ownership of services from architecture and implementation to deployment and production operations. Professionals who enjoy tackling architectural ambiguities at the intersection of machine learning and large-scale infrastructure will uncover vast opportunities to build, learn, and scale.
What you will do
- Design and construct production-grade AI agent systems capable of diagnosing and remediating infrastructure anomalies across a massive GPU fleet.
- Develop the underlying distributed services, orchestration frameworks, knowledge graphs, and retrieval systems that empower infrastructure agents.
- Create fleet intelligence mechanisms by synthesizing telemetry data, operational knowledge, and historical incidents to enhance agent decision-making.
- Establish robust integrations with observability, incident management, ticketing, and chat platforms using well-designed application programming interfaces.
- Maintain end-to-end ownership of services encompassing system architecture, implementation, rigorous testing, deployment, and operational monitoring.
- Enhance agent performance iteratively by utilizing evaluation frameworks, retrieval tuning, and actionable production feedback loops.
What you need
- Bring at least five years of professional experience building production backend architectures, distributed systems, or core infrastructure platforms.
- Demonstrate strong systems design capabilities with a proven track record of owning major technical systems from initial concept through production launch.
- Possess specialized depth in areas such as AI agent systems, orchestration, tool utilization, evaluation methodologies, or grounding techniques.
- Show working knowledge of knowledge graphs, graph data modeling, search mechanisms, retrieval pipelines, ranking algorithms, or semantic search.
- Exhibit robust backend engineering proficiencies including clean API design, clear service boundaries, data modeling, and complex system integrations.
- Bring hands-on familiarity with Kubernetes, GitOps tools like ArgoCD, infrastructure-as-code practices, and modern cloud platforms.
- Remain comfortable writing software across multiple programming languages including Go, TypeScript, Python, or Rust.
Nice to have
- GPU infrastructure, datacenters, bare-metal systems, hardware failure modes, BMC/IPMI, or cluster schedulers
- Graph databases
- Event-driven systems and messaging platforms such as NATS or Kafka
- Observability platforms such as Prometheus and Grafana
- Building evaluation frameworks or improving the quality and reliability of LLM-powered systems
Benefits
- Competitive compensation
- Startup equity
- Health insurance
- Flexibility in terms of remote work
- Other benefits
Skills and tech stack
Questions candidates ask
What is the remote work policy?
The company offers flexibility in terms of remote work for this position.
What is the salary range for this role?
The US base salary range for this full-time position is $250,000 to $300,000, supplemented by equity and benefits.
What kind of technology stack will I be working with?
Engineers work across languages such as Go, TypeScript, Python, or Rust, alongside Kubernetes, GitOps tools like ArgoCD, and various observability platforms.
What core problems does the team focus on?
The team tackles making AI agents trustworthy enough to operate production infrastructure while building the knowledge, retrieval, and distributed systems that make those agents effective.