Skip to Main content Skip to Navigation
Journal articles

Qualitative Adaptive Reward Learning With Success Failure Maps: Applied to Humanoid Robot Walking

Abstract : In the human brain, rewards are encoded in a flexible and adaptive way after each novel stimulus. Neurons of the orbitofrontal cortex are the key reward structure of the brain. Neurobiological studies show that the anterior cingulate cortex of the brain is primarily responsible for avoiding repeated mistakes. According to vigilance threshold, which denotes the tolerance to risks, we can differentiate between a learning mechanism that takes risks and one that averts risks. The tolerance to risk plays an important role in such a learning mechanism. Results have shown the differences in learning capacity between risk-taking and risk-avert behaviors. These neurological properties provide promising inspirations for robot learning based on rewards. In this paper, we propose a learning mechanism that is able to learn from negative and positive feedback with reward coding adaptively. It is composed of two phases: evaluation and decision making. In the evaluation phase, we use a Kohonen self-organizing map technique to represent success and failure. Decision making is based on an early warning mechanism that enables avoiding repeating past mistakes. The behavior to risk is modulated in order to gain experiences for success and for failure. Success map is learned with adaptive reward that qualifies the learned task in order to optimize the efficiency. Our approach is presented with an implementation on the NAO humanoid robot, controlled by a bioinspired neural controller based on a central pattern generator. The learning system adapts the oscillation frequency and the motor neuron gain in pitch and roll in order to walk on flat and sloped terrain, and to switch between them.
Complete list of metadata

Cited literature [64 references]  Display  Hide  Download
Contributor : Claire Dune Connect in order to contact the contributor
Submitted on : Tuesday, February 18, 2020 - 6:07:55 PM
Last modification on : Saturday, June 25, 2022 - 9:13:55 PM


Files produced by the author(s)




John Nassour, Vincent Hugel, Fethi Ben Ouezdou, Gordon Cheng. Qualitative Adaptive Reward Learning With Success Failure Maps: Applied to Humanoid Robot Walking. IEEE Transactions on Neural Networks and Learning Systems, IEEE, 2013, 24 (1), pp.81-93. ⟨10.1109/TNNLS.2012.2224370⟩. ⟨hal-01723809⟩



Record views


Files downloads