Outlier is a Scale AI-operated platform where vetted experts write prompts, build grading rubrics and rate model answers to train large language models, earning task-based pay settled weekly.
Used this tool? Rate it
Used this tool? Rate it
Outlier is a platform operated by Scale AI that connects subject-matter experts with leading AI companies, and the human feedback those experts produce is used to improve large language models. Outlier is not software you buy.
Understanding this inversion is the key to reading everything that follows. When you evaluate a normal SaaS product, you ask whether the features justify the price.
There are really two distinct readers arriving at this page, and they want opposite things.
The first is a freelancer or subject-matter expert who has seen advertisements promising flexible remote work training AI and wants to know whether the money is real, whether the platform is legitimate, and what the catch is.
The second is a company that needs training data — an AI lab, a research team, or a product organisation building a domain-specific model. This reader is not going to sign up as a contributor.
This article serves both, but it does not pretend the two views are the same. Where the supply side and the demand side see the same fact differently — and pay rates are exactly such a fact — that difference is stated plainly.
Every large language model that answers helpfully rather than plausibly has been shaped by human judgement after pretraining. That distinction has to come from people.
Outlier is one of the largest pipelines feeding that process. Its contributors are not writing code that ships to users and are not building the model architecture.
Outlier's relationship to its parent matters more than it would for most products, because the parent has been through significant upheaval. Scale AI is a data-infrastructure company that supplies annotation and evaluation services to major AI labs.
For a contributor, that transaction is mostly background noise, though it does signal that the parent is well capitalised.
Outlier's own About page reports more than 500 million US dollars paid out, operations across 50 countries, and a network of more than 100,000 experts, alongside figures for billions of annotations delivered.
The work itself resolves into three recurring activities: writing challenging prompts, creating grading rubrics, and rating and ranking model answers.
The first activity asks contributors to write questions that a model will probably get wrong. This is harder and more interesting than it sounds.
Doing this well requires real domain expertise, which is the entire economic justification for the platform's hiring bar. A chemist knows which reaction mechanisms produce plausible-sounding but wrong intermediate steps.
The second activity is less discussed and arguably more skilled. A rubric is the scoring instrument that lets a grader — human or automated — judge whether a response is good.
Rubric quality compounds through the entire downstream pipeline. This is why the platform treats rubric writing as a distinct, often better-paid competency rather than a clerical step.
The third activity is the volume work: presented with a prompt and two or more model responses, judge which is better and articulate why.
The justification text usually matters as much as the ranking itself. A bare preference tells the training process that response A beat response B; a well-written justification tells it which dimension drove the decision, which is a far richer signal.
Supporting all three is the machinery that decides who sees which task. Contributors declare areas of expertise, complete skill screenings, and pass identity verification before being matched to projects.
Because output quality is the product, the platform maintains ongoing review of contributor work rather than a one-time entrance exam. Submissions are audited, and continued access to a project depends on sustained quality. Community discussion also references time-tracking tooling on certain projects, meaning that on some queues work is monitored more like conventional employment than like piecework.
The clearest fit is a person with genuine specialist knowledge and irregular time.
Graduate stipends are notoriously thin, and academic schedules have gaps that conventional part-time work cannot fill. Task-based AI training fits into evenings and gaps between experiments without requiring a fixed commitment.
Model performance in languages other than English remains substantially weaker, and closing that gap requires native speakers who can also evaluate technical correctness. This is one of the few areas where a contributor has real pricing leverage.
Code is unusually well suited to this work because correctness is often checkable. Engineers assess whether generated code compiles, handles edge cases, and follows idiomatic practice.
On the demand side, an organisation training or fine-tuning a model needs evaluation capacity it cannot staff internally. Buying that capacity through Scale AI's contributor network converts a fixed organisational cost into a variable one.
A specialised subset involves probing models for harmful, biased or unsafe outputs. Anyone considering this category specifically should read that section first, not last.
Start with eligibility, because it is the cheapest thing to check and the most common reason applications fail. You need at least an associate degree to work on Outlier, and certain projects require a bachelor's, master's, or PhD depending on how specialised the work is.
Applications require a valid government ID and mobile phone from your country of residence, a current resume highlighting your expertise, and a LinkedIn profile showing your educational background and work experience.
Onboarding typically takes 30 to 90 minutes and covers account creation, selecting your areas of expertise, completing skill screenings so the platform can match you to projects, and identity verification. Treat the expertise selection deliberately.
Screenings are subject-matter assessments, and they are the gate to the higher-paying queues. They are generally unpaid, which is a real cost worth naming: you invest time before earning anything, with no guarantee of placement.
Rates vary by expertise, project complexity, and location, and the platform states that you will always see the tasking rate before starting any project. This is the single most useful habit to build.
Payments are processed weekly on Tuesdays, covering work completed from the previous Tuesday through Monday at midnight UTC. Payment methods are PayPal, Airtm, or ACH bank transfer.
Do not assume that clearing onboarding produces immediate work. Outlier itself acknowledges that new contributors sometimes do not receive a project right after onboarding, because occasionally there are more qualified people than active tasks available.
The posted rate applies to tasking time. Your effective rate divides total earnings by all hours invested, including reading project instructions, sitting through updates, and redoing flagged work. Track both numbers for two weeks.
Keep independent records: screenshots of completed tasks, your own time log, copies of project instructions, and a running tally of expected earnings. This is inexpensive insurance and it costs nothing while things go well.
The most consistent structural advice from experienced contributors is diversification. Work volume fluctuates, projects end abruptly, and accounts can be closed.
Given reports of balances becoming inaccessible following account closures, the prudent practice is to withdraw as soon as funds are available rather than accumulating a balance on the platform.
Quality scores are cumulative and early mistakes are expensive to recover from. Time spent understanding a project's standards before starting is usually recovered several times over in avoided rework and preserved queue access.
Of the three core activities, rubric writing is the most transferable. It is closely related to evaluation design work that is increasingly in demand at AI companies directly.
Independent contractors are typically responsible for their own tax reporting and social contributions.
If assigned to content moderation or safety red-teaming, take the psychological load seriously. Set limits on daily exposure, use any available opt-out mechanisms, and stop if the work is affecting you. No rate compensates for lasting harm.
The platform works best for people who hold a degree in a field with commercial demand, have hours that do not fit conventional employment, and want supplementary rather than sole income.
The strongest position on the platform belongs to those combining language capability with technical depth, because that intersection is rare and directly tied to a known model weakness.
For a career-changer curious about the AI field, this offers paid exposure to how models are actually evaluated.
If rent depends on this month's earnings, look elsewhere. Availability is intermittent by the platform's own admission, projects end without notice, and account status is not guaranteed. This is structurally unsuitable as a primary income source.
The associate degree floor is a hard gate. Applicants without it should look at annotation platforms with lower barriers rather than spending effort here.
Given documented cases of withheld balances, anyone for whom losing several hundred dollars would be materially damaging should weigh that risk explicitly before investing significant unpaid onboarding time.
Buyers who need large volumes of expert evaluation quickly and lack internal capacity are the natural fit.
Outlier operates as a web application. Contributors log in through the browser to a task interface, with no desktop software to install.
Although the interface is browser-based, the work is poorly suited to phones. Tasks involve reading long model outputs, comparing responses side by side and writing detailed justifications.
A mobile phone from your country of residence is required, but as an identity and account-security instrument rather than a work device. The distinction matters: possessing a phone does not mean you can work from one.
The company reports operating across 50 countries, but eligibility is not purely a matter of national availability. Your visa must permit work, and Outlier places the responsibility for meeting the legal requirements of your country of residence squarely on you.
The three supported payment rails — PayPal, Airtm, and ACH bank transfer — have uneven geographic coverage. ACH is a United States domestic system.
Verification requires submitting government-issued identification and biometric-adjacent imagery. Contributors unwilling or unable to provide this cannot participate; there is no anonymous path onto the platform.
This section works differently from a standard tool listing. There is no subscription to buy — the money flows toward the contributor, not away. What follows is therefore about compensation, presented separately for each audience.
There is no fee to apply or work. Any request for payment in exchange for access to Outlier work should be treated as fraudulent, since platforms of this kind are a recurring target for impersonation scams.
Outlier's own materials cite hourly rates in the region of 15 to 50 US dollars, with variation by expertise, complexity and location.
The company's expert page features contributors describing earnings above 1,000 US dollars in a good week. Read them as an upper bound observed by some contributors, not as an expectation.
The most important caveat is not the rate but what the rate covers. Compensation is tied to tasking time, while onboarding, screenings, instruction review and project training are frequently unpaid or under-compensated.
Settlement is weekly, on Tuesdays, for the prior Tuesday-to-Monday period, via PayPal, Airtm or ACH. Reporting indicates no minimum payout threshold, so small balances are not trapped by a floor.
Scale AI does not publish rate cards for its data services. Enterprise engagements are quoted based on volume, domain, turnaround and quality requirements, and pricing is available only through direct sales contact.
The most frequently compared alternative. Community reporting consistently characterises it as offering steadier work at somewhat lower ceilings, against Outlier's higher peaks and more volatile availability.
A newer entrant in expert-driven model evaluation, carrying notably strong ratings on public review platforms. Its smaller scale cuts both ways — potentially more attentive support, but a narrower range of available projects.
Oriented toward academic and market research studies rather than model training specifically. Participants are compensated for study participation, generally at lower rates than expert AI work but with lower barriers and shorter time commitments.
Both position toward the higher-skill end, matching specialists — particularly engineers — with AI companies for evaluation and training work, often with a more direct contracting relationship than a task marketplace provides.
Established microtask platforms handling large volumes of general annotation. Their qualification barriers are far lower and so are their rates; they are a reasonable fallback for those who do not meet Outlier's degree requirement.
Experienced specialists sometimes bypass platforms entirely, contracting with AI labs directly or through consultancies. This offers materially better rates and clearer legal standing, at the cost of having to find the work yourself.
Alternative data vendors include Appen, Toloka, Surge AI and Labelbox, alongside building an in-house annotation function.
This section carries the information most likely to change a reader's decision, and it is where the evidence has been held to the highest standard. Nothing here rests on impression.
Three clauses in the Terms of Use collectively define the relationship, and prospective contributors should read them before investing time.
First, classification: you accept independent contractor status and assume all liability for proper classification as an independent contractor. Second, ownership: by accepting the Terms of Use you irrevocably assign all right, title and interest throughout the world, including intellectual property rights, in the work product to the company. Third, and most consequentially, termination: the company may deactivate or suspend your account and access to its systems at any time, immediately and without prior notice, for any reason.
That third clause is not unusual boilerplate in isolation, but it is the contractual foundation beneath the most common complaint about the platform. Disputes are further channelled into binding arbitration rather than court, which limits collective legal recourse. None of this is hidden — it is in the published terms — but it is rarely surfaced in recruitment material.
The pattern is corroborated across three independent sources of different types, which is what elevates it above anecdote.
On Trustpilot the platform holds a 3.8 rating across 4,534 reviews, with 59 percent leaving five stars and 22 percent leaving one star. That distribution is itself the finding: not a mediocre average but a sharply bimodal one, where most report a good experience and a substantial minority report a bad one. Trustpilot's own summary attributes the negative sentiment to sudden account bans and removal from ongoing projects without clear explanation, alongside support replies that feel automated.
The community picture matches. The r/outlier_ai community has grown past 84,000 members, and its most frequently used post flairs are suspended by outlier, payments, and confirmed violation: deactivation stands.
Glassdoor provides the third angle, with an overall rating of 3.2 out of 5 across 697 reviews and compensation satisfaction at 3.1 — consistent with a platform that works acceptably for many and poorly for a meaningful minority.
A worker petition hosted on Coworker.org alleges that accounts were closed with no warning, no explanation and no appeal process while earned balances were withheld.
The appropriate epistemic status here is important. These are allegations by affected workers, organised collectively, and not adjudicated findings. What makes them worth reporting is the convergence: the same pattern appears in the Trustpilot summary, in the subreddit's flair taxonomy and in the petition, from populations that do not overlap much. Convergence across independent sources is not proof, but it is far more than an impression.
Between late 2024 and early 2025, Scale AI faced multiple worker lawsuits in quick succession. Press coverage documented a second wage lawsuit filed within a month of the first, with claims centring on below-minimum-wage effective pay and misclassification of workers as contractors rather than employees. These are unproven claims filed by plaintiffs: none of these suits had been adjudicated at the time of writing, and Scale AI has publicly signalled it intends to defend itself against the worker litigation brought against it in this period.
The most serious claim is also the best documented. In January 2025 a class action filed in the Northern District of California alleged that workers were required to write disturbing prompts about violence and abuse without adequate psychological support, with plaintiffs describing resulting mental health consequences and alleging retaliation for seeking counselling.
Scale AI has contested these claims. A company spokesperson stated the intention to defend vigorously and pointed to existing safeguards including opt-out options, content warnings and wellness programmes. These allegations remain unproven, and readers should note both the seriousness of the claims and the fact that they have not been established in court. The practical implication is narrower and safe to state regardless of outcome: anyone considering safety or content-moderation queues should understand what the material may involve and confirm what support is actually available before starting.
Even setting disputes aside, the economics are volatile. The platform acknowledges that qualified contributors sometimes exceed available tasks.
Across review platforms, contributors describe support interactions as templated and difficult to escalate to a human, particularly around account issues.
Screenings, instruction review and project training are largely uncompensated. A contributor may invest several hours before earning anything, with no guarantee of placement. This is a genuine and often unacknowledged cost of entry.
For buyers, the litigation and labour disputes are procurement-relevant, not merely reputational.
It is a legitimate operation. Outlier is run by Scale AI, an established data-infrastructure company with major institutional investors, and the majority of reviewers across public platforms confirm receiving payment — 59 percent of Trustpilot reviewers leave five stars, most citing reliable weekly payment. Legitimacy and reliability are different questions, though.
The company's own materials cite roughly 15 to 50 US dollars per hour, varying by expertise, project complexity and location, with specialised domains such as coding, advanced mathematics, law and medicine at the upper end. Two adjustments matter. First, marketing testimonials describing over 1,000 dollars per week are vendor-selected favourable cases, not typical results.
At minimum an associate degree, with certain projects requiring a bachelor's, master's or PhD depending on specialisation. You will also need a valid government ID and mobile phone from your country of residence, a current resume, and a LinkedIn profile reflecting your education and work history. There is a selective screening process, so meeting the formal minimum does not guarantee acceptance. No prior AI experience is required.
The company reports operating across 50 countries but does not publish a definitive eligibility list, which is a real gap in its published information. The stated requirement is that your visa must permit work and that you are responsible for satisfying the legal requirements of your country of residence. In practice, the reliable test is to start an application and see whether your country is supported.
Payments run weekly on Tuesdays, covering work completed from the previous Tuesday through Monday at midnight UTC, via PayPal, Airtm or ACH bank transfer. Reporting indicates no minimum payout threshold. Because of the cutoff structure, work finished just after a Monday boundary settles nearly two weeks later. Currency conversion and processor fees are your responsibility and can be material for cross-border payments.
The Terms of Use permit deactivation at any time, without prior notice, for any reason — so the contractual answer is that no specific cause is required. Reported triggers include quality-score problems, suspected guideline violations and automated integrity checks, but contributors frequently report receiving only a generic notice without specifics. Reinstatement does happen: the r/outlier_ai community maintains an account restored flair.
The company does. Under the Terms of Use you irrevocably assign all right, title and interest worldwide, including intellectual property rights, in the work product to the company. Prompts you write, rubrics you design and evaluations you produce become company property, and you retain no rights to reuse them. This is standard for annotation work but frequently surprises contributors from creative or academic backgrounds where authorship norms differ.
Identity verification requires selfies and images of your government-issued ID, and that verification data is retained for up to two years after your last interaction with the service. The platform also collects name, address, country of residence, contact details, education and experience information, plus standard technical data such as IP address and device type.
No, and the platform's own structure argues against it. Work availability is intermittent by the company's own admission, projects end without notice, and accounts can be closed at any time under the terms. Community reporting consistently describes feast-or-famine cycles. It functions well as supplementary income for someone with specialist credentials and flexible time, and poorly as a sole income source.
Pricing is not published and is available only through direct sales engagement, quoted on volume, domain, turnaround and quality requirements. Two factors warrant specific diligence. First, ownership: Meta holds roughly a 49 percent non-voting stake in Scale AI, which buyers with strict vendor-neutrality requirements should evaluate directly.
