Ready, aim, release. That sequence for shooting an arrow at a target might be self-evident to adults and even children, but not so for a robot. So researchers at the Italian Institute ofTechnology developed a learning algorithm that’s being used to teach the skill of archery to a humanoid robot known as iCub.
The learning algorithm, called ARCHER (Augmented Reward Chained Regression), was developed and optimized specifically for problems like the archery training, which have a smooth solution space and prior knowledge about the goal to be achieved.
In the case of archery, we know that hitting the center corresponds to the maximum reward we can get. Using this prior information about the task, we can view the position of the arrow’s tip as an augmented reward.
ARCHER uses a chained local regression process that iteratively estimates new policy parameters that have a greater probability of leading to the achievement of the goal of the task, based on the experience so far. An advantage of ARCHER over other learning algorithms is that it makes use of richer feedback information about the result of a rollout.
For the archery training, the ARCHER algorithm is used to modulate and coordinate the motion of the two hands, while an inverse kinematics controller is used for the motion of the arms. After every rollout, the image processing part recognizes automatically where the arrow hits the target, which is then sent as feedback to the ARCHER algorithm.
The image recognition is based on Gaussian Mixture Models for color-based detection of the target and the arrow’s tip.
The experiments are performed on a 53-DOF humanoid robot iCub. The distance between the robot and the target is 3.5 meters, and the height of the robot is 104 cm.
This research will be presented at the Humanoids 2010 conference in December 2010 in the United States.