60% success: MBZUAI robot picks fruit with an imagined camera view

Key Points

1. MBZUAI researchers let a robot generate the view its blocked wrist camera would have seen.
2. A Unitree Z1 arm picked a plastic strawberry in 6 of 10 trials using generated images.
3. Field and factory robots could keep working when leaves or mud cover a lens.

The latest:

A robot arm whose wrist camera was unusable still completed 6 of 10 strawberry-picking attempts after a diffusion model generated the missing viewpoint for it, MBZUAI researchers reported. The Abu Dhabi team calls the method imagination at inference, and it needs no extra hardware beyond an external camera already watching the scene. The work was presented at IROS 2026 in Pittsburgh.

Details:

• How it works: Instead of reading a physical wrist-mounted camera, the system takes images from a separate external camera and uses a diffusion model to synthesize what the gripper’s own camera would be showing. That generated frame is passed to the robot’s control system exactly as a live feed would be, so the controller itself requires no modification.
• The model: The team built on ZeroNVS, an existing novel-view-synthesis diffusion model, and fine-tuned it with low-rank adaptation, known as LoRA, to sharpen the generated images. The approach adapts a general-purpose image generator to the narrow task of reconstructing one specific camera angle on a robot arm.
• Simulated results: Across six simulated tasks, the researchers reported that on peg insertion the robot succeeded roughly 30% of the time with the external camera alone, about 72% with a genuine wrist camera, and about 53% with the generated wrist view — recovering much, though not all, of the lost performance.
• Real-world test: Using a Unitree Z1 robot arm to pick a plastic strawberry over 10 trials per setup, the team logged 3 successes with the external camera only, 8 with a real wrist camera, and 6 with the generated view. The researchers describe these physical results as preliminary given the small trial count.
• The problem addressed: Wrist cameras give robots the close-up detail needed for precise grasping, but they sit exactly where leaves, mud or debris land. The method targets that failure point without adding sensors or redesigning the gripper, keeping the fix entirely in software.
• Known limits: The researchers state the diffusion model is slow and cannot yet generate views in real time, a constraint for any live deployment. They also note a real wrist camera is still required during data collection to train the system, so the hardware cannot be removed from the pipeline altogether.
• The team: Anqing Duan, assistant professor of robotics at MBZUAI, led the work with co-authors Haoran Ding, Zezhou Sun, Dezhen Song and Yoshihiko Nakamura. The study was presented at IROS 2026, the IEEE/RSJ International Conference on Intelligent Robots and Systems, held in Pittsburgh.
• Next steps: Planned follow-up work includes testing the method against different occlusion shapes and varying levels of transparency, rather than the single blocking scenario examined so far, and extending the underlying idea to what the researchers call world action models.

Between the lines:

The published figures place the generated view between the two baselines rather than beside the real camera: 53% against 72% in simulation, 6 successes against 8 in the physical test. That positions the method as degradation control for a blocked sensor, not a replacement for one — a reading reinforced by the team’s own note that a physical wrist camera is still needed to train the system.

What’s next

Watch for results from the planned tests on varied occlusion shapes and transparency levels, for any reduction in the diffusion model’s generation time toward real-time operation, and for larger real-world trial counts beyond the 10 runs reported.