Paper Information

Journal:   MODARES MECHANICAL ENGINEERING   JANUARY 2017 , Volume 16 , Number 10 #F0070; Page(s) 241 To 252.
 
Paper: 

ONLINE SOLUTION OF THE HAMILTON–JACOBI–BELLMAN EQUATION FOR NONLINEAR SYSTEMS WITH UNKNOWN DRIFT DYNAMICS USING NEURAL NETWORK

 
 
Author(s):  MOJOODI ARMAN, NARAGHI MAHYAR*, MORADI MOJTABA
 
* DEPARTMENT OF MECHANICAL ENGINEERING, AMIRKABIR UNIVERSITY OF TECHNOLOGY, TEHRAN, IRAN
 
Abstract: 

In this paper a method for online solution of the Hamilton-Jacobi–Bellman (HJB) equation is proposed.
The method is utilized to design an optimal controller for continuous-time nonlinear systems. The main concept in this approach is using experiences to reinforce the controller, which is called Reinforcement Learning (RL). The online solution is based on the actor-critic (AC) structure where two Neural Networks (NNs) approximately solve the HJB equation. Optimal control and optimal value function are approximated by the actor and the critic, respectively. Then, employing gradient descent algorithm, accuracy of the approximation is improved. Since some items like friction and damping are difficult to model and calculate, a neural-robust identifier is used in conjunction with the AC to approximate drift dynamics. Finally, the Actor-Critic-Identifier (ACI) structure is proposed to solve the HJB equation online with no prior knowledge of drift dynamics. The closed-loop stability of the overall system is assured by the Lyapunov theory employing the direct method. Then the effectiveness of the proposed method is illustrated by experiment for DC motor and simulation for a nonlinear system. Results indicate satisfactory performance of the proposed method to solve the Hamilton-Jacobi-Bellman equation.

 
Keyword(s): HJB EQUATION, OPTIMAL CONTROL, NONLINEAR SYSTEM, NEURAL NETWORK, ACTOR-CRITIC
 
References: 
  • ندارد
 
  Persian Abstract Yearly Visit 30
 
Latest on Blog
Enter SID Blog