Staff Engineer, Distributed Storage and HPC & AI Infrastructure
About this role
Together AI builds a specialized AI Native Cloud platform that delivers high-performance inference, fine-tuning, and massive-scale pre-training for artificial intelligence workloads. The engineering team operates multi-petabyte storage systems designed specifically to handle extreme throughput and low-latency data paths for large-scale GPU clusters. Staff engineers in this group directly tackle the complexities of managing parallel filesystems, object stores, and custom Kubernetes storage operators that automate provisioning and ensure strict multi-tenancy.
Technical leaders joining the organization spend their days scaling high-performance storage architectures, integrating technologies like Vast, Weka, and Ceph, and optimizing end-to-end data paths for heavy machine learning workloads. The role exists to support rapid growth in GPU capacity while maintaining high availability, low latency, and cost efficiency across thousands of nodes. Ideal candidates possess extensive backgrounds in distributed storage engineering, deep familiarity with cloud-native tooling, and a proven track record of designing reliable, high-throughput storage systems for compute-heavy environments.
What you will do
- Architect and implement the technical strategy and storage roadmap for Together AI while scaling the GPU fleet.
- Engineer and scale multi-petabyte AI and machine learning storage systems through automated tiering and cost optimization.
- Develop intelligent caching and tiered storage architectures to achieve extreme IOPS and cluster-wide throughput.
- Tune storage isolation at network layers to support secure, production-grade multi-tenancy for storage clients.
- Code Kubernetes storage operators and controllers to enable automated provisioning and self-service abstractions.
- Engineer end-to-end data paths to achieve high throughput per GPU node and scale storage infrastructure across thousands of nodes.
What you need
- Applicants must have at least eight years of professional experience in storage engineering managing distributed storage at multi-petabyte scale.
- A proven track record deploying and operating high-performance storage for GPU and HPC clusters is required.
- Candidates need deep Kubernetes and cloud-native storage experience in production environments.
- Strong coding skills in Go and Python are necessary for building production-grade systems and tooling.
- A Bachelor or Master degree in Computer Science, Engineering, or equivalent practical experience is mandatory.
- Demonstrated history of technical leadership designing systems that improve performance, reliability, or cost efficiency is expected.
Nice to have
- GPU Direct Storage, NVMe-oF, storage networking, and RDMA implementations
- ML and AI storage patterns including model weights, checkpointing, and dataset caching
- Storage benchmarking and profiling tools such as fio, iperf3, iostat, and blktrace
Skills and tech stack
Questions candidates ask
What kind of storage systems and technologies will I work with?
You will operate, scale, and optimize multi-petabyte storage systems purpose-built for AI training and inference workloads. This involves evaluating and integrating technologies such as Vast, Weka, Ceph, and Lustre while managing high-performance parallel filesystems and object stores.
What are the primary software development expectations for this role?
You will code Kubernetes-native storage operators and self-service platforms to provide automated provisioning, strict multi-tenancy, performance isolation, and quota enforcement. Strong coding skills in Go and Python are required to build production-grade systems and internal tooling.
What performance metrics and scale are expected from the infrastructure?
The team engineers end-to-end data paths to achieve 10+ GB/s per GPU node and scales storage infrastructure across thousands of nodes. You will also maintain high reliability targets, including a history of designing systems with 99.999%+ uptime.