Senior Data Scientist AI Evaluation
About this role
Alpaca seeks a Senior Data Scientist for AI Evaluation to design measurement systems for models and agents operating remotely across the Americas. This remote position suits an experienced quantitative professional who wants to establish a greenfield evaluation practice at a global financial technology infrastructure provider.
Role Overview
Alpaca is hiring a Senior Data Scientist, AI Evaluation to operate as a senior individual contributor establishing the quality bar for models and agents. The person in this role will turn ambiguous quality questions into concrete scoring methods, ground truth definitions, and automated evaluation loops. Working independently from the teams building the systems, this practitioner will leverage an existing data foundation to enhance overall quality and accelerate safe system releases while partnering closely with product, engineering, and analytics engineering teams.
Key Responsibilities
Requirements
Nice to Have
Skills and Tools
Salary and Benefits
About the Company
Alpaca operates as a US-headquartered, global leader in agent-first brokerage infrastructure spanning stocks, ETFs, options, crypto, fixed income, and 24/5 trading. Serving hundreds of financial institutions across 40 countries through institutional-grade APIs, the organization supports broker-dealers, investment advisors, wealth managers, hedge funds, and crypto exchanges representing over 10 million brokerage accounts. The enterprise employs a globally distributed team of over 400 members and is backed by $400 million in funding from prominent global investors.
How to Apply
Candidates should apply directly on the official company careers page.
Skills and tech stack
Questions candidates ask
What is the primary focus of the Senior Data Scientist, AI Evaluation role?
The primary focus is designing how Alpaca measures whether models and agents are accurate, establishing ground truth, scoring methods, and evaluation loops to raise quality and ensure safe rollouts.
Where is the position located?
The position is remote across the Americas.
How many years of experience are required for this role?
The role requires 6 to 10 years of experience in quantitative data science or machine learning, with a specific focus on measurement or evaluation.
What programming languages and technical skills are necessary?
Candidates must be proficient in Python and SQL, possess a strong statistical and machine learning background, and have experience evaluating models in production environments.
What benefits and perks are included with the position?
Benefits include a competitive salary, stock options, health benefits, a one-time USD $500 home-office setup allowance, and a monthly USD $150 stipend via a Brex Card.