SMB AI Power User — Competitive Evaluations
About this role
The Competitive Evaluations team at Meridial builds structured benchmarking workflows to help a major consumer technology company evaluate its bundled AI assistant against standalone market alternatives. Operating as an embedded extension of the client organization, this group conducts head-to-head testing using real-world operational and marketing tasks.
This role exists for a seasoned business owner or operator to use their own day-to-day work tasks as a live test bed for AI performance. The person in this position runs parallel workflows across competing assistants, drafts ad copy, manages customer messages, and documents precise failure modes such as tone misses and formatting errors. An ideal candidate is a hands-on sole proprietor or operations leader who relies on generative AI daily and can write clear, reproducible evaluation reports for engineering teams.
What you will do
- Run head-to-head prompts and multi-turn workflows across the client assistant and competing paid assistants using tasks drawn from your own business operations and marketing.
- Build and maintain a library of realistic small-business test cases spanning ad creative, organic content planning, customer messaging, and competitor research.
- Identify and document failure modes including wrong answers, unnecessary refusals, weak instruction-following, tone misses, and broken formatting.
- Write clear and reproducible reports detailing the prompt used, model, expected versus actual output, severity, and business impact.
- Assess the bundled subscription on value terms by comparing what a business owner receives versus standalone competitor subscriptions.
- Stress-test assistant features directly inside social and commerce surfaces where business owners interact with them.
What you need
- Must be an owner or operator of a small or medium business such as a sole proprietor, founder, or operations lead at a company with roughly 1 to 200 employees.
- Demonstrate daily, habitual use of generative AI tools for both business operations and marketing.
- Maintain an active organic presence for the business personally on Instagram and Facebook including posting, Stories, Reels, and direct message response.
- Possess hands-on paid advertising experience on Meta platforms with a history of personally building, launching, and optimizing campaigns.
- Be based in the United States and authorized to work in the country.
- Have sufficient capacity to complete deliverables according to agreed scope and timeline up to 40 hours per week.
- Demonstrate exceptional written English skills sufficient to describe technical failures for engineering reproduction.
- Feel comfortable working independently against a rubric while maintaining a weekly output cadence.
Nice to have
- At least one current paid AI subscription such as ChatGPT Plus, Claude Pro, or Gemini Advanced.
- Operation of a side business or second venture alongside a primary role.
- Heavy personal and commercial use of Instagram including fluency in Reels, Stories, and trends.
- Simultaneous subscription to two or more AI assistants with an informed view on market positioning.
- Prior experience with model evaluation, red teaming, data annotation, quality assurance, or structured user research.
Benefits
- $40-$50 per hour pay range determined by experience, expertise, and location
Skills and tech stack
Questions candidates ask
What is the remote work policy for this role?
The position is remote-first and world-wide, though candidates must be based in the United States and authorized to work in the country.
What is the compensation for this position?
The pay range is $40 to $50 per hour, with the exact rate determined after evaluating your experience, expertise, and geographic location.
What kind of daily tasks will I perform?
Your day is spent doing what you already do in your business, such as drafting ad copy, planning content, answering customer messages, and reconciling numbers across competing AI assistants.
What are the time commitment expectations?
You must have sufficient capacity to complete deliverables per the agreed scope and timeline, which may include up to 40 hours per week.