The infinite horizon optimal tracking problem is solved for a deterministic, control-affine, unknown nonlinear dynamical system. A deep neural network (DNN) updated in real time to approximate the system dynamics. developed framework uses multitimescale concurrent learning-based weight update policy, with which output layer DNN weights are time, but internal features discretely and at slower timescale (i.e., batch-like updates). design of policy motivated by Lyapunov-based analysis, inner...

A real-time Deep Neural Network (DNN) adaptive control architecture is developed for general uncertain nonlinear dynamical systems to track a desired time-varying trajectory. Lyapunov-based method leveraged develop adaptation laws the output-layer weights of DNN model in while data-driven supervised learning algorithm used update inner-layer DNN. Specifically, are estimated using an unsupervised provide responsiveness and guaranteed tracking performance with feedback. The trained collected...

Lyapunov-based real-time update laws are well-known for neural network (NN)-based adaptive controllers that control nonlinear dynamical systems using single-hidden-layer NNs. However, developing weight deep NNs (DNNs) remains an open question. This letter presents the first result with adaptation each layer of a feedforward DNN-based architecture, stability guarantees. Additionally, developed method allows nonsmooth activation functions to be used in DNN facilitate improved transient...

A real-time deep neural network (DNN) adaptive control architecture is developed for uncertain control-affine nonlinear systems to track a time-varying desired trajectory. Lyapunov-based analysis used develop adaptation laws the output-layer weights and constraints inner-layer weight laws. Unlike existing works in DNN-based control, method establishes framework simultaneously update of multiple layers DNN arbitrary depth real-time. The controller enable system trajectory while compensating...

This letter provides an approximate online adaptive solution to the infinite-horizon optimal control problem for control-affine continuous-time nonlinear systems while formalizing system safety using barrier certificates. The use of a function transform certificates formalize behavior. Specifically, function, is transformed aid in developing controller which maintains pre-defined constrained region. To learning value state-space segmented into number user-defined segments. Off-policy...

This article provides an approximate online adaptive solution to the infinite-horizon optimal tracking problem for control-affine continuous-time nonlinear systems with uncertain drift dynamics. A model-based dynamic programming (ADP) approach, which is facilitated using a concurrent learning-based system identifier, approximates value function. To reduce computational complexity of ADP, state space segmented into user-defined segments (i.e., regions). Off-policy trajectories are selected...

This paper provides an approximate online adaptive solution to the infinite-horizon optimal control problem for control-affine continuous-time nonlinear systems. The state-space is segmented into a user-defined number of segments. Off-policy trajectories are selected over each segment facilitate learning value function weight estimates. Sparse neural networks enable framework switching and state space segmentation as well computational benefits due small neurons that active. At sparse...

This paper examines the use of reinforcement learning-based controllers to approximate multiple value functions specific classes subsystems while following a switching sequence. Each subsystem may have varying characteristics, such as different cost or system dynamics. Stability overall sequence is proven using Lyapunov-based analysis techniques. Specifically, methods are developed prove boundedness individual and determine minimum dwell-time condition ensure stability Uniformly ultimately...

A hierarchical reinforcement learning-based control strategy is introduced to facilitate state regulation for a hypersonic vehicle. To account time-varying aerothermoelastic parameters in real-time, switching policy selects subsystem from larger set of potential subsystems. The selection depends on an approximation the optimal value function each subsystem. Integral concurrent learning used approximate parametric uncertainties dynamical system. proven converge neighborhood policy. Uniformly...

Marine robots must maintain precise control and ensure safety during tasks like ocean monitoring, even when encountering unpredictable disturbances that affect performance. Designing algorithms for uncrewed surface vehicles (USVs) requires accounting these to the vehicle it avoids obstacles. While adaptive has addressed USV challenges, real-world applications are limited, certifying amidst unexpected remains difficult. To tackle issues, we employ a model reference controller (MRAC) stabilize...

Abstract Closed-loop functional electrical stimulation (FES) control methods are developed to facilitate motor-assisted cycling as a rehabilitative strategy for individuals with neurological disorders. One challenge this type of design is accounting an input delay called the electromechanical (EMD) that exists between and resultant muscle force. The EMD can cause otherwise stable system become unstable. A real-time deep neural network (DNN)-based motor architecture used estimate nonlinear...

Motivated by the desire to use Coulomb forces as a non-contact means for one satellite exert on another applications such debris removal, two-player non-cooperative zero-sum-game is formulated. Despite uncertainty in dynamics of disabled (DS), including interaction with service (SS), an approximately optimal real-time feedback policy (i.e., approximate dynamic programming (ADP)) derived using reinforcement learning and Bellman error (BE) extrapolation method identify unknown value function...

View Video Presentation: https://doi.org/10.2514/6.2022-0613.vid A model-based approximate dynamic programming (ADP) controller is applied to a hypersonic vehicle (HSV) with time-varying aerothermoelasatic effects for optimal state regulation. To account aerothermoelastic parameter variations, the nominal HSV model discretely switched over time better reflect changes caused by parameters. Lypunov-based analysis leveraged design actor-critic update laws reinforcement learning ADP approach and...

View Video Presentation: https://doi.org/10.2514/6.2021-1131.vid An adaptive controller is developed for a regolith excavation robot to determine the mass of excavated material and account effects gravity friction while on surface other celestial bodies. A data-based integral concurrent learning (ICL) parameter update law accounts estimates unknown mass, gravity, parameters. Lyapunov-based analysis proves that trajectory tracking error estimate errors exponentially converge zero. estimation...

This paper provides an approximate online adaptive solution to the infinite-horizon optimal tracking problem for a set of agents with homogeneous dynamics and common objectives. Model-based reinforcement learning is implemented by simultaneously evaluating Bellman error (BE) at state each agent on nearby off-trajectory points, as needed, throughout space. Each will calculate share their respective BE information centralized estimator, which computes updates shares estimate agents. In doing...

Deploying autonomous systems in safety critical settings necessitates methods to verify their properties. This is challenging because real-world may be subject disturbances that affect performance, but are unknown a priori. work develops safety-verification strategy wherein data collected online and incorporated into reachability analysis approach check real-time the system avoids dangerous regions of state space. Specifically, we employ an optimization-based moving horizon estimator (MHE)...

A supervisory control approach using hierarchical reinforcement learning (HRL) is developed to approximate the solution optimal regulation problems for a control-affine, continuous-time nonlinear system with unknown drift dynamics. This result contains two objectives. The first objective policy that minimizes infinite horizon cost function of each dynamic programming (ADP) sub-controller. second design switching rule, by comparing approximated value functions ADP sub-controllers, ensure...

This paper applies a reinforcement learning-based approximately optimal controller to motorized functional electrical stimulation-induced cycling system track desired cadence. Sufficient torque achieve the objective is achieved by switching between quadriceps muscle and electric motor. Uniformly ultimately bounded (UUB) convergence of actual cadence neighborhood approximate control policy are proven for both motor via Lyapunov-based stability analysis provided developed dwell-time conditions...