About this role
The ideal candidate for this role enjoys owning complex, high-impact production systems within an AI infrastructure context. This position is central to DataSnipper's operations, as the team is responsible for managing every AI call, from inference routing and failover to cost attribution, and ensuring the quality of the AI output. You will be building the core platform that allows product teams to measure AI quality through versioned datasets and experiment tracking. This work directly influences how DataSnipper redefines audit processes by embedding AI deeply into workflows.
This role exists to establish and maintain the foundational infrastructure for scalable, reliable, and traceable AI operations. The engineer will own critical components like the LLM gateway and the evaluation platform, setting the standards for other teams. It is suited for someone who thrives on solving hard, open problems and has the ability to deliver tangible results visible to a large user base.
A successful candidate will possess strong ownership skills, a growth mindset, and the ability to collaborate effectively across product and ML engineering teams. This is a chance to build category-defining infrastructure at scale.
What you will do
- Own the LLM gateway, managing routing, provider failover, rate limits, retries, and cost attribution across various model providers.
- Deploy, version, and deprecate models across different clouds, regions, and environments, including capacity planning managed via infrastructure as code.
- Construct the shared evaluation platform, encompassing versioned datasets, experiment tracking, run and result schemas, and traceability.
- Manage the infrastructure required for asynchronous and long-running AI workloads.
- Oversee observability for AI traffic, monitoring latency, retries, token usage, cost, and errors per team and use case.
- Participate in the on-call rotation to handle incidents and manage follow-ups.
- Implement security and compliance controls such as retention, access control, RBAC, and SSO.
What you need
- Candidates must have at least five years of experience in backend or platform engineering, coupled with strong proficiency in Python.
- Experience building or operating LLM inference infrastructure, specifically a gateway or routing layer that handles multiple providers, failover, rate limiting, and cost attribution is required.
- Proficiency with cloud infrastructure at the infrastructure level and experience with infrastructure as code tools like Terraform is necessary.
- Demonstrated experience running a shared service in production, including managing on-call duties, incidents, postmortems, and SLOs.
- Hands-on experience with observability tools such as OpenTelemetry and Grafana, including service instrumentation and dashboard design.
- Comfort and experience with privacy and compliance matters, including PII handling, anonymization, retention policies, and access control.
- Prior experience building platform or shared-service capabilities that are consumed by multiple internal teams is essential.
Nice to have
- Experience with LLM or agent evaluation
- Temporal or another durable workflow engine
- Self-hosted inference, capacity planning, load testing
- Synthetic data or document anonymisation pipelines
- Document AI: VLMs, OCR, structured extraction and the metrics that go with it
- Domain experience in audit, accounting or fintech
Skills and tech stack
Questions candidates ask
What is the remote policy for this role?
The posting does not specify the remote work policy for this position.
What is the compensation structure?
The posting does not provide specific salary information for this role.
What is the primary tech stack used?
The core technologies mentioned include Python, Terraform for infrastructure as code, and observability tools like OpenTelemetry and Grafana.
What is the team structure like?
This is described as a small team with a large blast radius, and the role involves strong collaboration with product and ML engineers.