About this role
The Data Engineer at You.com is central to building and scaling the AI Search Infrastructure that supports modern AI systems. The day-to-day work involves developing and maintaining reliable, high-performance data pipelines, supporting both batch and real-time data processing. This individual will collaborate extensively with Finance, Engineering, Product, and Analytics teams to ensure data quality, accessibility, and usability across the entire organization. A key focus is enabling data activation, ensuring high-quality data flows to warehouses and outward to business tools like Salesforce. Furthermore, this role contributes directly to powering next-generation AI-driven applications, including agent-based systems, by establishing robust data foundations and pipelines.
This position exists to solve complex data challenges across search, large language models, and large-scale infrastructure, aiming to make AI systems more reliable and useful. The successful candidate will play a defining role in how modern AI finds and utilizes knowledge by building the necessary data infrastructure. This role is ideal for someone who enjoys solving data challenges comprehensively, managing processes from initial data ingestion through to generating actionable insights and serving AI applications.
Candidates who thrive in cross-functional environments and enjoy end-to-end data problem-solving will find this role suitable. It requires a blend of deep technical skill in data engineering with a strong understanding of business needs related to finance and marketing data utilization.
What you will do
- Build and maintain scalable data pipelines for both batch and streaming data using Databricks, Spark, Kafka, and AWS services.
- Develop pipelines to ingest data from source systems like Salesforce, billing, product events, and API logs into clean analytics layers.
- Design, develop, and optimize ETL/ELT workflows utilizing DBT, PySpark, SQL, and tools such as Fivetran.
- Partner with the Finance team to develop data solutions, metrics, and forecasting models related to revenue accounting and margin reporting.
- Create and manage curated datasets to support analytics, reporting, and go-to-market initiatives.
- Develop and maintain reverse ETL pipelines to synchronize data from the warehouse to downstream tools like Salesforce and HubSpot.
- Support AI/ML and agent-based applications by preparing and serving high-quality datasets for MCP integrations and AI-driven applications.
- Monitor pipeline performance, troubleshoot data issues, and implement data quality checks and alerting mechanisms.
What you need
- Possesses at least six years of experience in data engineering or a closely related field.
- Demonstrates strong, hands-on experience with core technologies including Databricks, AWS services (such as S3, Glue, Athena, EMR), and Kafka.
- Must be proficient in Python (PySpark) and SQL for handling large-scale data processing tasks.
- Requires experience building and maintaining ETL/ELT pipelines, with familiarity with DBT or similar workflow tools being preferred.
- Should have experience with data ingestion tools like Fivetran or comparable solutions.
- Familiarity with reverse ETL workflows and syncing data to platforms such as Salesforce or HubSpot is required.
- Exposure to AI/ML data pipelines, including RAG architectures or vector databases, is beneficial.
- Understanding of data modeling principles and experience working with large-scale batch and streaming datasets is necessary.
Nice to have
- Experience with agent-based systems
- Experience with MCP integrations or LLM-powered applications
- Experience working with Finance and building finance-specific metrics and pipelines
Benefits
- Hubs in San Francisco and New York City offering regular in-
Skills and tech stack
Questions candidates ask
What is the remote policy for this role?
The job is listed as a Hybrid role located in San Francisco. The company offers hubs in San Francisco and New York City with regular in-office opportunities.
What is the salary range for this position?
The salary band for this role is set between $180,000 and $220,000 USD. Compensation is determined by multiple factors assessed during the interview process.
What is the primary tech stack used for data processing?
The core technologies involve Databricks, Spark, Kafka, AWS services, Python (PySpark), and SQL for large-scale data processing.
What kind of cross-functional collaboration is expected?
The Data Engineer will work closely with Finance, Engineering, Product, and Analytics teams to ensure data quality and usability across the organization.