Latest AI JobsBrowse Jobs
A

Machine Learning Engineer – ML Evaluation & Experiment Design

anyone-aiArgentina - Fully RemoteSoftware Engineering
Apply on company site
Salary
$65 per hour
Job type
Contract
Work mode
Remote
Experience
Senior · 3+ yrs
Posted
15 Sept 2026
Status
Open · checked 7d ago

About this role

This role suits experienced machine learning engineers who enjoy debugging experiments, understanding why models succeed or fail, identifying problems in datasets and evaluation pipelines, and designing rigorous machine learning experiments. The position is a fully remote, project-based consulting engagement based in Argentina.

Role Overview

Anyone AI is seeking experienced machine learning engineers for a specialized project centered on reviewing and evaluating machine learning challenges utilized in artificial intelligence model training and assessment. The position focuses on analyzing experiments, datasets, metrics, and pipelines to ensure that challenges are technically sound, reproducible, appropriately difficult, and dependent on genuine machine learning reasoning rather than brute-force methods.

Key Responsibilities

Review machine learning challenges to determine technical soundness and solvability.
Evaluate datasets to confirm the presence of meaningful and learnable signals.
Detect unintended shortcuts or artifacts within synthetic datasets.
Assess whether assigned tasks demand proper diagnosis of underlying machine learning issues.
Analyze evaluation metrics and improvement thresholds.
Spot metric gaming, data leakage, and flaws in the evaluation setup.
Verify reproducibility across data, model, and evaluation pipelines.
Check whether the difficulty level of challenges is calibrated correctly.
Deliver clear recommendations for improving, recalibrating, or removing problematic tasks.

Requirements

Three or more years of hands-on applied machine learning experience.
Practical background in machine learning experiment design, model selection, hyperparameter tuning, model evaluation, and data preprocessing and validation.
Clear understanding of train, validation, and test split methodologies.
Ability to spot data leakage, label noise, distribution shift, spurious correlations, feature leakage, and data contamination.
Experience determining if performance gains are statistically meaningful rather than random noise.
Solid grasp of machine learning evaluation metrics and their appropriate use cases.
Background in debugging machine learning workloads across both CPU and GPU environments.
Capacity to analyze technical problems and write clear feedback.

Nice to Have

Experience creating or participating in competitions on platforms like Kaggle or DrivenData.
Background in designing benchmark datasets or machine learning challenges.
Familiarity with data-centric AI or dataset quality.
Experience generating and validating synthetic data.
Knowledge of statistical testing, confidence intervals, and effect sizes.
Experience with machine learning evaluation pipelines, reinforcement learning from human feedback, or AI model evaluation.
History of developing machine learning curricula or technical assessments.
Familiarity with shortcut learning, spurious correlations, Goodhart's law, Simpson's paradox, and metric gaming.

Skills and Tools

Machine Learning Experiment Design
Model Selection
Hyperparameter Tuning
Model Evaluation
Data Preprocessing and Validation
CPU and GPU Workload Debugging
Statistical Significance Testing

Salary and Benefits

Rate of $65 per hour
Contract engagement type

About the Company

Anyone AI recruits specialized machine learning professionals for projects focused on evaluating artificial intelligence challenges, experiment designs, and model training datasets.

How to Apply

Interested candidates should apply directly on the official company careers page.

Skills and tech stack

CPUGPUMachine Learning

Questions candidates ask

What is the location and work type for this position?

The position is fully remote based in Argentina and operates as a contract, part-time, project-based consulting engagement.

What compensation is provided for this role?

The compensation is $65 per hour.

How many years of experience are required?

Candidates must have three or more years of hands-on applied machine learning experience.

What are the primary tasks of the machine learning engineer?

The responsibilities include reviewing machine learning challenges, evaluating datasets for learnable signals, detecting data leakage and metric gaming, verifying pipeline reproducibility, assessing challenge difficulty, and providing written feedback.

How should I submit my application?

Applicants should apply on the official company careers page.

Ready to apply?

The application is on the official anyone-ai careers page.

Apply on company site