Latest AI JobsBrowse Jobs
Mistral AI logo

Research Engineer - Eval Platform

Mistral AIParisScience
Apply on company site
Salary
Not listed
Job type
Full-time
Experience
Senior · 4+ yrs
Posted
8 Oct 2026
Status
Open · checked 7d ago

About this role

Based in Paris, the Research Engineer - Eval Platform position at Mistral AI involves developing the infrastructure used by science teams to measure model quality. This role suits a software professional with a background in building production-grade systems and large-scale machine learning codebases.

Role Overview

Mistral AI is hiring a Research Engineer for the Eval Platform team in Paris. The professional in this role will focus on creating reliable, reproducible, and fast evaluation infrastructure to determine which models, checkpoints, and recipes get deployed.

Key Responsibilities

Design systems that maintain the reproducibility and comparability of evaluation outcomes over time.
Execute large-scale evaluations across GPU clusters, handling tasks from model serving to scoring.
Develop APIs and dashboards that allow researchers to access, explore, and trust evaluation results.
Identify broken or noisy evaluations prior to research decisions.
Assist in evaluating agentic, multi-turn, and tool-using models.
Partner with researchers to translate new evaluation needs into shared and robust tooling.

Requirements

Master's or PhD degree in Computer Science, or equivalent practical experience.
Minimum of four years of experience creating production-grade software, with a focus on large-scale machine learning codebases or distributed systems.
Proficiency in Python alongside software-design practices including testing, code review, and CI/CD.
Practical background running workloads on GPU clusters such as Slurm, Kubernetes, or Ray.
Understanding of LLM inference and evaluation.
Product mindset treating researchers as primary users.
Collaborative and self-driven working style.

Nice to Have

Background in developing or maintaining benchmarks or evaluation harnesses.
Practical use of inference engines like vLLM or SGLang.
Exposure to reinforcement learning or agentic environments.
Knowledge of experimentation statistics including significance testing and variance estimation.
Contributions to open-source machine learning tooling.

Skills and Tools

Python
GPU clusters (Slurm, Kubernetes, Ray)
LLM inference and evaluation
vLLM
SGLang

About the Company

Mistral AI provides full-stack artificial intelligence solutions ranging from developer tools and applications to compute and frontier models. The organization partners with enterprises in high-stakes sectors such as healthcare, defense, finance, manufacturing, and the public sector to co-create tailored AI systems. The workforce operates globally across North America, Europe, Asia, and the Middle East.

How to Apply

Interested candidates should apply directly on the official company careers page.

Skills and tech stack

PythonSlurmKubernetesRayvLLMSGLangLLMDistributed Systems

Questions candidates ask

Where is this job located?

This position is located in Paris.

What level of experience is required?

Candidates need 4+ years of experience building production-grade software, specifically with large-scale ML codebases or distributed systems, alongside a Master's or PhD in Computer Science or equivalent experience.

What technologies and tools are mentioned for the role?

Key tools and technologies include Python, GPU cluster managers like Slurm, Kubernetes, or Ray, and inference engines such as vLLM or SGLang.

Do I need prior experience designing benchmarks?

No, prior benchmark design experience is not required, though having it is listed as a nice-to-have.

How do I apply for the position?

Applicants should submit their application through the official company careers page.

Ready to apply?

The application is on the official Mistral AI careers page.

Apply on company site