About this role
The Research Engineer at Constellation focuses on bridging the gap between raw multimodal data and the machine learning models that process it. The day-to-day work involves orchestrating and optimizing complex training runs for long-horizon sequences, establishing robust data pipelines to transform growing corpora into usable training material, and developing the research code that enables the broader team to iterate efficiently. This role is crucial because Constellation is focused on building a new class of foundation models by collecting and leveraging the richest multimodal dataset ever assembled, aiming to create AI that deeply understands human experience.
This individual will be responsible for ensuring training infrastructure is efficient, data throughput is maximized, and research ideas are translated into reliable, reproducible pipelines. They will work closely with both engineering and research teams to ensure that the research process is both flexible and grounded in high-quality, traceable systems. The role is ideal for someone who enjoys deep technical problem-solving at the intersection of large-scale distributed systems and cutting-edge multimodal AI.
Candidates who excel in this role are those who thrive in an environment requiring meticulous attention to detail across the entire ML lifecycle, from data ingestion to model evaluation. A strong background in building reusable research infrastructure and a commitment to making complex systems traceable will be key to success here.
What you will do
- Orchestrate and optimize distributed training configurations, including techniques like DDP/FSDP, mixed precision, checkpointing, and recovery for multi-day runs.
- Develop high-throughput data loading mechanisms, focusing on sharding, prefetching, caching, and format choices to maximize GPU saturation.
- Wrangle complex multimodal data by aligning and synchronizing multi-stream time series and managing evolving collection protocols.
- Build pipelines to derive rich, versioned features from raw recordings, ensuring reproducibility based on input data changes.
- Write clean, well-tested, and documented research code, including libraries and data loaders, that the rest of the team can utilize.
- Implement traceability across all runs by versioning code, configuration, datasets, and environments.
- Construct evaluation harnesses that automatically run on new checkpoints to surface regressions early.
What you need
- Possess at least three years of experience building ML systems or research infrastructure, specifically including experience with distributed training runs on over 100 GPUs.
- Demonstrate expert-level proficiency in Python and deep knowledge of PyTorch internals, including DDP, FSDP, mixed precision, gradient accumulation, and profiling tools.
- Show a proven track record of building or maintaining research libraries that other teams depend on, with contributions to packages like torch_geometric or similar tooling.
- Have experience with data pipelines for large, unstructured, and multimodal datasets, familiar with columnar and streaming formats such as Zarr, Parquet, or WebDataset.
- Be proficient with experiment tracking and dataset versioning tools like ClearML, Weights & Biases, or MLflow.
- Exhibit sufficient research fluency to understand research papers, reimplement components, and diagnose issues in loss curves.
- Demonstrate debugging skills capable of resolving issues ranging from corrupted data shards to subtle training divergences.
Nice to have
- Custom kernel work in CUDA or Triton, or compiler-level optimization (torch.compile, TensorRT, ONNX)
- Experience with time-series or multimodal data where synchronization across streams is a prerequisite for analysis and learning
- Low-latency or edge inference, quantization, or distillation
- Comfort dropping into Rust or C++ when Python is the wrong tool
- Interest in neurotech, mental health, or human-AI interaction
Benefits
- Comprehensive, high-quality health, dental, and vision insurance with premiums fully covered
- A renovated, light-filled office with a quirky layout in the heart of San Francisco's Mission District, surrounded by world-class food and coffee
- Relocation assistance for those joining us in SF and workspace setup stipend
- Monthly team dinners and outings, plus twice-yearly off-sites and retreats
- Fully stocked kitchen (snacks!)
- Sponsorship for travel to relevant conferences and regular professional development activities to help you level up
- A minimum of 12 weeks of fully paid parental leave
Skills and tech stack
Questions candidates ask
What is the remote policy for this role?
The posting does not explicitly detail the remote work policy. The role is listed as FullTime and the location is San Francisco.
What is the compensation structure?
The salary range for this Research Engineer role is listed between $180K and $250K, and the offer includes equity.
What is the primary technology stack used?
The core technologies involve expert-level Python and deep knowledge of PyTorch internals, including DDP and FSDP. Data formats mentioned include Zarr, Parquet, Arrow, Lance, WebDataset, and Vortex.
What is the team environment like?
The role requires thriving in a high-bandwidth, collaborative environment where the engineer works closely with both engineering and research teams.
What opportunities are there for professional development?
The company provides sponsorship for travel to relevant conferences and regular professional development activities to help the engineer level up.