Latest AI JobsBrowse Jobs
Lightfield logo

Software Engineer, Data Infrastructure (Staff)

LightfieldHQ: San FranciscoEngineering
Apply on company site
Salary
$180K - $300K USD
Job type
Full-time
Posted
3 Aug 2026
Status
Open · checked 7d ago

About this role

This staff-level Software Engineer, Data Infrastructure role at Lightfield involves building and scaling core data systems for an artificial intelligence company based in San Francisco or Cambridge. The position suits an experienced technologist eager to tackle high-volume data challenges and shape foundational architectures close to the product.

Role Overview

Lightfield is scaling its technical operations to support a rapidly expanding customer base and massive database workloads. The company operates a high-throughput environment handling billions of queries and millions of background jobs monthly. This position calls for a staff engineer who specializes in data infrastructure while remaining enthusiastic about backend systems and product-facing data challenges. Team members can operate from the San Francisco headquarters or an infrastructure-focused site in Kendall Square, Cambridge.

Key Responsibilities

Scale the analytics engine powering customer-facing dashboards, managing query performance under row-level security, workload isolation, and read architectures.
Create ingestion paths, event models, schemas, and query patterns moving data from transactional writes into search and history with strong freshness and correctness guarantees.
Develop and evolve schema-flexible, graph-shaped data models to maintain high query speeds as customer-defined objects and relationships grow.
Construct foundational systems for historical reporting, auditability, attribute versioning, and change capture.
Build robust data mechanisms for consumption billing, pipeline generation, and artificial intelligence agent evaluation.
Evaluate the current architecture to determine the optimal timing for introducing advanced analytical, streaming, or workflow systems.
Establish technical direction, boundaries, abstractions, and engineering practices for organizational data systems.

Requirements

Minimum of 7 years of professional experience.
Robust software engineering fundamentals.
Proven ownership of production data systems where factors like replication lag, query plans, backfills, and schema evolution directly impacted users.
Ability to debug across diverse layers of the technical stack.
Sound judgment regarding tactical fixes versus durable architectural investments.
Product-minded perspective valuing how infrastructure choices influence users and engineering velocity.
Clear communication skills, high ownership, and a focus on practical tradeoffs.

Nice to Have

Familiarity with ClickHouse, OLAP technologies, event pipelines, data warehouses, or analytical infrastructure.
Knowledge of Kafka, Flink, Spark, Iceberg, or comparable streaming and lakehouse tools.
Experience operating Postgres at scale and balancing OLTP and OLAP workloads.
Background with APIs, queues, workflow platforms, and distributed systems.
Expertise in observability, incident response, service ownership, and production troubleshooting.
Experience handling data for machine learning or artificial intelligence systems, including enrichment pipelines and evaluation harnesses.
Previous tenure in a high-growth product organization.

Skills and Tools

Spark
Kafka
Postgres
Redis
Typesense
BullMQ
ClickHouse
Flink
Iceberg

Salary and Benefits

Annual salary ranging from $180,000 to $300,000 USD
Meaningful early equity opportunities
Comprehensive health insurance including medical, dental, and vision coverage
Three weeks of paid time off
Eleven paid company holidays plus a winter holiday break
Three months of paid family leave
Wednesdays designated for remote work from home
Team dinners, social events, offsites, and company retreats
401(k) retirement plan
Commuter and lunch stipends

About the Company

Lightfield is redefining customer relationship management by establishing a living business world model driven by emails, meetings, and customer interactions. Backed by prominent investors including Andreessen Horowitz, Coatue, Maverick Ventures, and Greylock, the firm has raised over $70 million. The founding team previously developed Tome, serving over 25 million users, and brings deep expertise from leading technology organizations.

How to Apply

Interested candidates should visit the official company careers page at lightfield.app to submit an application for this role.

Skills and tech stack

SparkKafka

Questions candidates ask

Where is this job located?

This role is based out of the San Francisco headquarters or an infrastructure-focused site in Kendall Square, Cambridge.

What is the salary range for this position?

The annual salary is between $180,000 and $300,000 USD, accompanied by early equity.

How many years of experience are required?

Candidates must have 7+ years of professional experience.

What technologies are used in the tech stack?

The current stack utilizes Postgres, Redis, Typesense, BullMQ, Spark, and Kafka, among other tools.

What benefits are included?

Benefits include medical/dental/vision insurance, 3 weeks of PTO, 3 months of paid family leave, a 401k plan, commuter and lunch stipends, and equity.

Ready to apply?

The application is on the official Lightfield careers page.

Apply on company site