All positions
AI Quality Analyst
AI Workforce Remote (Worldwide) Contract
We are seeking AI Quality Analysts to audit human feedback inputs. You will measure dataset agreement levels, run spot checks, and ensure that our workforce deliverables are pristine.
What you'll do
- Perform regular audits and spot checks on annotations, reviews, and prompt datasets
- Calculate and report inter-annotator agreement (IAA) metrics
- Identify guidelines misunderstandings and run retraining sessions for teams
- Maintain the dashboard tracking overall project quality metrics
What we're looking for
- Background in QA, statistics, auditing, or technical content validation
- High attention to detail and strong quantitative assessment skills
- Strong interpersonal skills to deliver constructive feedback to contributors
- Familiarity with data validation logic is highly preferred
Benefits & Perks
- Remote-first contract setup
- Competitive hourly pay
- Influence on quality processes for major AI deployment scales
- Growth pathways into program management and operations leadership
Ready to apply?
Takes about 5 minutes. We review every application personally.
Apply for this position