Latest AI JobsBrowse Jobs
A

Site Reliability Engineer, AI Observability

Appnovation TechnologiesNew York, Austin, Miami, DallasTechnology
Apply on company site
Salary
Not listed
Job type
Full-time
Experience
Senior
Posted
6 Oct 2026
Status
Open · checked 7d ago

About this role

Appnovation Technologies seeks a Site Reliability Engineer to maintain an observability platform for LLM applications in New York, Austin, Miami, or Dallas. This role suits an infrastructure professional with deep operational expertise in Kubernetes, ClickHouse, and AWS.

Role Overview

Appnovation Technologies requires a Site Reliability Engineer to oversee a shared observability system built on Langfuse for a global life sciences client. The position involves sustaining production environments while establishing new monitoring, alerting, and service level definitions from the ground up.

Key Responsibilities

Troubleshoot and fix issues across Kubernetes, ClickHouse, PostgreSQL, Redis, and ingestion workers.
Manage ClickHouse operations using the operator model, handling replication, Keeper quorum, and storage tiering.
Perform and test data restoration processes to guarantee recoverability.
Execute safe software upgrades with tested rollback procedures.
Implement monitoring, alerting, and service level metrics from scratch.
Maintain infrastructure as code using Kubernetes manifests, Helm, and Argo CD.
Author technical runbooks and standard operating procedures for incident response.
Support internal teams utilizing the platform and handle user onboarding.
Automate repetitive operational tasks.

Requirements

Practical background managing Kubernetes on AWS via Helm values and Argo CD.
Production experience operating ClickHouse with replication, topology management, and backups.
Track record of building monitoring and alerting solutions alongside user-centric service levels.
Experience performing software upgrades and handling schema migrations with defined rollback plans.
Operational knowledge of PostgreSQL and Redis for debugging queue and metadata issues.
Strong technical writing skills for creating actionable runbooks and incident reviews.

Nice to Have

Background in observability engineering covering OpenTelemetry pipelines, Grafana, and alerting.
Integration experience with OIDC or enterprise single sign-on systems.
Familiarity with GitHub Actions for deployment pipelines.
Prior work with LLM monitoring tools like Langfuse, LangSmith, or Arize Phoenix.
Professional background in regulated sectors such as life sciences or pharmaceuticals.

Skills and Tools

Kubernetes
AWS
ClickHouse
PostgreSQL
Redis
Argo CD
Helm
Langfuse

About the Company

Appnovation operates as a global digital partner delivering strategy, experience, design, engineering, and managed services. The organization embraces diversity, equity, and inclusion as an equal opportunity employer.

How to Apply

Candidates should apply on the official company careers page.

Skills and tech stack

KubernetesAWSClickHousePostgreSQLRedisArgo CDHelmLangfuseOpenTelemetryGrafanaGitHub ActionsLangSmithArize PhoenixLLM

Questions candidates ask

What locations are available for this position?

The role is available in New York, Austin, Miami, and Dallas.

What technology stack does the observability platform use?

The platform utilizes Langfuse, Kubernetes on AWS, ClickHouse, PostgreSQL, Redis, and Argo CD.

What are the primary responsibilities regarding monitoring?

The engineer must build monitoring, alerting, and service levels completely from scratch since the platform currently lacks them.

What backup tasks are required?

The engineer must rehearse, time, document, and test data restoration against various failure modes rather than relying solely on successful backup job logs.

Is industry experience required?

Experience in life sciences, pharma, or another regulated industry is preferred rather than strictly mandatory.

Ready to apply?

The application is on the official Appnovation Technologies careers page.

Apply on company site