The Toloka Platform follows a streamlined process from project creation to receiving results:
1. Creating a project
The platform uses a hierarchical structure: Project Space → Project (v1, v2, etc.).
Step 1: Create a project space
A Project Space is your organizational container.
You can have multiple project spaces to organize different projects.
Give it a name, for example: “My ML projects” or “Research experiments”.
Step 2: Create a Project (v1) within the Project space
When you create a project, it automatically becomes version 1 (v1).
A Project represents your annotation task (e.g., “Chatbot Response Evaluation” or “Product Image Classification”).
Give your project a descriptive name that reflects the task—this name will be visible to experts on their task dashboard, so make it clear and informative.
This first version will contain all your project settings and configurations.
Creating additional versions
To create a new version (v2, v3, etc.), you create a new project with the same or similar name.
Each version is a separate project with its own settings and configurations.
When creating a new version, all settings from the previous version are fully copied and become available for editing.
You can experiment with different settings across versions to find what works best.
More details about project setup → Project Configuration
2. Self check: testing your configuration
Before launching, test your configuration yourself. The system provides 10 sample tasks—complete them exactly as experts will to catch issues early.
What to check
interface usability;
task completeness;
LLM QA responses;
guideline clarity.
Important: Complete all 10 tasks fully—this is what you’ll expect from experts.
If you find issues, return to configuration, adjust settings, and run self check again. Iterate until everything works smoothly.
3. Pricing: setting the cost
After self-check, you understand your task well. Now set the price:
Select the experts who will work on your tasks.
Provide time estimate based on your self-check experience.
Set the task price—the payment per expert execution, platform will recommend you the price, you can adjust it.
Platform calculates service fee and LLM QA cost automatically.
Review total price for your dataset, adjust parameters if needed.
You can influence the total cost by adjusting project settings, the task price, expert audience, approximate time to complete a task, or dataset size.
Important: Price is fixed for this project version once you launch.
More details about pricing → Project contract & pricing
4. Launch and receiving results
Once you agree with all requirements and final price, you launch the project. Important: You cannot change settings during annotation. You can cancel annotation and create a new project version if needed.
Results appear as they become ready. You can track progress through the dashboard.
5. Quality feedback and iterations
Important: Checking results against your Quality Requirements is your responsibility. The platform provides LLM QA, but you must verify that results meet your Quality Requirements.
Evaluate results and calibrate with LLM QA assessments. This helps you:
Understand the quality of results.
Enable the agent to suggest improvements to settings.
Improve quality for next launches.
The agent learns from your Quality Feedback and suggests improvements to the project and its settings.