On the Day It Gets Smarter, Check How It Stops
■ Behind the improvement, another behavior changed
Overnight, Shiraishi updated Hakobukun's AI to a new version. It was supposed to take corridor corners much more smoothly. And sure enough, Hakobukun came around the corner smoothly. But it stopped 30 cm closer than usual to Oshiro, who had just stepped out from the corner. "Too close!" Oshiro blurted out.
■ A trained AI doesn't change only what you aimed at
A trained AI (a system that learns how to move from many examples) can change other behaviors even when an update targets just one part. When Shiraishi checked the log (the record of how it moved), the stopping distance had changed along with the turning. As Hakobukun said, checking how it stops was not on the update checklist. When something isn't checked, no one notices if it changes.
■ Conclusion: Recheck how it stops for people after every update
Shiraishi added "stopping distance in front of people" to the checklist used for every update. Rolled back to the previous version, Hakobukun stops at its usual distance. The way a robot stops to protect people should be handled by a mechanism separate from learning. On top of that, confirm and record after every update that it still works. Oshiro's "Check it before you change it, okay?" is about that order: verify before it goes out on the floor.
Make the day you improve its movement the day you also check how it stops.
