About this role
Ensuring AI patient interactions remain safe, accurate, and high quality at scale requires a dedicated evaluations layer. Eucalyptus needs a data scientist to build the services that monitor patient-facing conversations, flag risky content, and integrate AI assistants with in-house patient data models across multiple global markets.
Reporting to the Head of Data, the person in this role will collaborate closely with machine learning engineering, data engineering, and data science teams. Responsibilities include defining metrics with clinicians and operating safety-critical systems in production. Candidates with a careful, safety-first mindset and strong Python fundamentals will thrive in this environment.
This junior position offers substantial early-career ownership within Juniper, a major weight-management program now part of Hims & Hers. Mentorship from experienced leaders provides structured support while building preventive healthcare infrastructure.
What you will do
- Design and build a standardised evaluation framework covering clinical interactions, health coach conversations, AI assistant conversations, PX support, and health insights.
- Build and run services that monitor patient-facing conversations and automatically flag risky content for review and escalation.
- Develop eval datasets, metrics, and thresholds with clinicians, coaches, and support leads by turning quality standards into measurable targets.
- Manage the technical integration between AI assistants and internal data models to make patient and business context available safely and consistently.
- Operate production systems through monitoring, alerting, and iterative improvements as models and prompts evolve.
- Automate guardrail updates using internal and patient feedback gathered from third-party support tools.
- Share results with relevant workstream owners to help improve ongoing patient care and operational workflows.
What you need
- Hold a degree in computer science, engineering, data science, or a related field, or possess equivalent practical experience.
- Demonstrate strong Python skills and the fundamentals needed to build reliable services beyond simple notebooks.
- Bring hands-on experience with large language models through prompt engineering, coursework, internships, or personal projects.
- Apply a careful, safety-first mindset regarding what AI systems communicate to real patients.
- Communicate clearly with both technical peers and clinical stakeholders regarding evaluation results.
- Feel comfortable navigating ambiguity to help define a brand-new evaluations layer from scratch.
Nice to have
- Experience with eval tooling such as promptfoo, Ragas, LangSmith, or in-house harnesses
- Exposure to healthcare, regulated industries, or other high-stakes AI applications
- Familiarity with GCP, SQL, or modern data stacks
Benefits
- Learning budgets, conferences, certifications, peer shadowing, and a global knowledge-sharing culture
- Catered wellness talks, exercise classes, whoops to track wellbeing, free barista coffees, funded social clubs, and quarterly rooftop parties
- Parental and miscarriage leave
- Health, professional development, and personal days
- Employee Assistance Program
Skills and tech stack
Questions candidates ask
What team will I work on and who is the manager?
You will report to the Head of Data and work closely with the ML Engineering, Data Engineering and Data Science teams.
What level of experience is required for this role?
This is a rare junior role offering real ownership of a safety-critical system from day one, with the support of experienced teams around you.
What tools and technologies will I use?
The tech stack includes Python, large language models, evaluation tooling like promptfoo, Ragas, LangSmith, or in-house harnesses, as well as GCP, SQL, or modern data stacks.
What kind of mindset is expected for this position?
Candidates need a careful, safety-first mindset where they care about what an AI system says to a real patient and sweat the edge cases, alongside comfort with ambiguity.
The application is on the official Careers at Eucalyptus careers page.
Apply on company site