Cortex AI

Frontier labs building embodied AI and robots need large volumes of real-world, real-workplace data to train physical intelligence, but such diverse, richly annotated egocentric and robot trajectory data is hard to obtain at scale. Existing data often lacks the hand and body pose, depth, and subtask annotations needed to make egocentric video directly usable for robotics, and robot-specific trajectory data tied to real embodiments and real workplace conditions is similarly scarce. Without this, labs lack a continuous flow of real-world experience, including recovery data from edge cases, to keep improving deployed models over time.

Cortex AI collects egocentric data from real workplaces, annotating every frame with hand and body pose, depth, and subtask labels so it can be used directly for robotics world models. It also gathers robot trajectories directly in real workplaces through a network of industry partners, producing high-quality, embodiment-specific data for fine-tuning on top of foundation models. The company supports human-in-the-loop rollouts and evaluations, capturing recovery data for edge cases, which creates a data flywheel where every real-world deployment feeds experience back into further training. Together these three offerings (egocentric data, robot data, and human-in-the-loop rollouts/evals) are packaged as the Cortex Harness, described as frontier AI models for continuous real-world robot deployment.