Sciloop

AI labs need extremely hard, expert-verified problems and solutions to evaluate and train frontier models, but ordinary crowdsourced data is too easy and gets saturated quickly. Sourcing IMO gold medalists, Codeforces grandmasters, and PhD-level domain experts, then checking and cross-verifying their work, is a slow and specialized task that most labs are not set up to do themselves. Without this, labs lack datasets that sit at the exact threshold where models break, making it hard to measure real progress.

Sciloop runs a two-sided marketplace: expert contributors browse open problem sets on platform.sciloop.dev, solve them in LaTeX-supported write-ups at their own pace, and get paid $225 or more per problem with no deadlines or quotas. On the enterprise side, AI labs specify the domain, difficulty, format and volume they need, and Sciloop handles expert sourcing, quality control, and cross-checking, streaming delivered problem-and-solution pairs continuously as labs update their requirements. The site shows an example active stream of graduate-level physics problems being delivered with multiple expert reviews per item, illustrating the review-and-delivery workflow.