Why this step is critical
Before launching, you must test your configuration yourself. It’s free! This is your opportunity to experience the task exactly as experts will—and catch issues before they affect hundreds or thousands of tasks. The agent doesn’t help here—you work independently to validate the setup. But you are still able to come back to an agent to change project settings or ask any questions.
Why It matters
Catch issues early: Fix problems before they affect hundreds or thousands of tasks.
Improve quality: Better configuration = better results from experts.
Save time and money: Fewer rejected tasks, less need for corrections.
Build confidence: Know your configuration works before launching.
Understand the expert experience: See exactly what experts will see and do.
What you’ll do
The system provides you with 10 sample tasks from your dataset. You’ll:
Complete tasks exactly as experts will.
Experience the expert interface firsthand.
See how LLM QA evaluates your work.
Identify any ambiguities or issues in your configuration.
The process: step by step
Complete a task
Open the first sample task.
Work through it as an expert would.
Fill in all required fields.
Submit your work to LLM QA.
Submit to LLM QA
Click “Submit to LLM QA”.
Your task moves to “Quality Check” status.
LLM QA Check may take up to 1 minute.
Review LLM QA response
After LLM QA Check completes, view the LLM QA evaluation.
See what errors or issues LLM QA found (if any).
Understand how your work was evaluated against Quality Requirements.
Iterate and improve
If LLM QA found issues, understand why.
Correct your guidelines or Quality Requirements if needed.
Try the task again with updated settings.
Repeat until you’re satisfied with the result.
Complete multiple tasks
Work through several tasks (not just one).
This ensures your configuration works consistently.
Verify that you get the results you expect across different examples.
What to check
Interface usability
Is the task interface clear and intuitive?
Can you understand what to do without confusion?
Are all fields accessible and easy to use?
LLM QA responses
How does LLM QA evaluate your work?
Are the checks appropriate and helpful?
Do the quality criteria make sense for your task?
Instruction clarity
Are the guidelines clear enough?
Are there ambiguous cases where you’re not sure what the correct answer should be?
Do examples help clarify expectations?
Task completeness
Can you complete the task as expected?
Are there missing pieces or unclear requirements?
Important: Complete all 10 tasks fully—this is what you’ll expect from experts. Take your time and think through each example carefully, just as you expect experts to do.
If you find issues
Note the specific problems
What’s unclear in the guidelines?
What’s confusing about the interface?
What issues did LLM QA identify?
What doesn’t work as expected?
Return to configuration
Update guidelines to clarify ambiguous points.
Adjust Quality Requirements if LLM QA checks are inappropriate.
Fix interface issues if needed.
Run self-check again
Test with updated configuration.
Verify that issues are resolved.
Continue iterating until everything works smoothly.
After self-check
Once you’re satisfied with the self-check results across multiple tasks, you’re ready to proceed to pricing and launch. If you made changes to the configuration, you can run self-check again to verify the improvements.