This paper proposes a novel event-triggered adaptive dynamic programming (ADP) control method for nonlinear continuous-time system with unknown internal states. Comparing the traditional ADP design fixed sample period, samples state and updates controller only when it is necessary. Therefore, computation cost transmission load are reduced. Usually, based on entire which either infeasible or very difficult to obtain in practice applications. integrates neural-network-based observer recover...

Recent studies on sequential attack schemes revealed new smart grid vulnerability that can be exploited by attacks the network topology. Traditional power systems contingency analysis needs to expanded handle complex risk of cyber-physical attacks. To analyze transmission under topology attacks, this paper proposes a Q-learning-based approach identify critical sequences with consideration physical system behaviors. A realistic flow cascading outage model is used simulate behavior, where...

In this paper, an event-triggered near optimal control structure is developed for nonlinear continuous-time systems with constraints. Due to the saturating actuators, a nonquadratic cost function introduced and Hamilton-Jacobi-Bellman (HJB) equation constrained formulated. order solve HJB equation, actor-critic framework presented. The critic network used approximate action estimate law. addition, in proposed method, signal transmitted aperiodic manner reduce computational transmission cost....

This paper presents the design of a novel adaptive event-triggered control method based on heuristic dynamic programming (HDP) technique for nonlinear discrete-time systems with unknown system dynamics. In proposed method, law is only updated when condition violated. Compared periodic updates in traditional (ADP) control, can reduce computation and transmission cost. An actor-critic framework used to learn optimal value function. Furthermore, model network designed estimate state vector. The...

This paper investigates the consensus problem for linear multi-agent systems with heterogeneous disturbances generated by Brown motion. Its main contribution is that a control scheme designed to achieve dynamic in directed topology interfered stochastic noise. In traditional ways, coupling weights depending on communication structure are static. A new distributed controller based Riccati inequalities, while updating associated gain matrix state errors between adjacent agents. By introducing...

In this paper, we develop and analyze an optimal control method for a class of discrete-time nonlinear Markov jump systems (MJSs) with unknown system dynamics. Specifically, identifier is established the to approximate states, approach MJSs developed solve Hamilton-Jacobi-Bellman equation based on adaptive dynamic programming technique. We also detailed stability analysis approach, including convergence performance index function existence corresponding admissible control. Neural network...

In this paper, we present a new model-free globalized dual heuristic dynamic programming (GDHP) approach for the discrete-time nonlinear zero-sum game problems. First, online learning algorithm is proposed based on GDHP method to solve Hamilton-Jacobi-Isaacs equation associated with optimal regulation control problem. By setting backward one step of definition performance index, requirement system dynamics, or an identifier relaxed in method. Then, three neural networks are established...

Model-based dual heuristic dynamic programming (MB-DHP) is a popular approach in approximating optimal solutions control problems. Yet, it usually requires offline training for the model network, and thus resulting extra computational cost. In this brief, we propose model-free DHP (MF-DHP) design based on finite-difference technique. particular, adopt multilayer perceptron with one hidden layer both action critic networks design, use delayed objective functions to train online over time. We...

By employing neural network approximation architecture, the nonlinear discounted optimal regulation is handled under event-driven adaptive critic framework. The main idea lies in adopting an improved learning algorithm, so that control law can be derived via training a network. stability guarantee and simulation illustration are also included. It highlighted initial stabilizing policy not required during implementation process with combined rule. Moreover, closed-loop system formulated as...

This paper presents a novel robust regulation method for class of continuous-time nonlinear systems subject to unmatched perturbations. To begin with, the problem is transformed into an optimal by constructing value function auxiliary system. Then, simultaneous policy iteration (SPI) algorithm developed solve within framework adaptive dynamic programming. implement SPI algorithm, actor and critic networks are employed approximate control function, respectively, Monte Carlo integration...

Goal representation heuristic dynamic programming (GrHDP) control design has been developed in recent years. The performance of this demonstrated several case studies, and also showed applicable to industrial-scale complex problems. In paper, we develop the theoretical analysis for GrHDP under certain conditions. It shown that internal reinforcement signal is a bounded index can converge its optimal value monotonically. existence admissible proved. Although method investigated many areas...

In this paper, we study the constrained optimization problem of a class uncertain nonlinear interconnected systems. First, prove that solution can be obtained through solving an array optimal control problems auxiliary subsystems. Then, under framework approximate dynamic programming, present simultaneous policy iteration (SPI) algorithm to solve Hamilton-Jacobi-Bellman equations corresponding By building equivalence relationship, demonstrate convergence SPI algorithm. Meanwhile, implement...

This paper provides an adaptive event-triggered method using dynamic programming (ADP) for the nonlinear continuous-time system. Comparing to traditional with fixed sampling period, samples state only when event is triggered and therefore computational cost reduced. We demonstrate theoretical analysis on stability of method, integrate it ADP approach. The system dynamics are assumed unknown. corresponding algorithm given neural network techniques applied implement this method. simulation...

This paper develops a new online learning consensus control scheme for multiagent discrete-time systems by goal representation heuristic dynamic programming (GrHDP) techniques. The agents in the whole system are interacted with each other through communication graph structure. Therefore, agent can only receive information from itself and its neighbors. Our is to design GrHDP method achieve which makes all track desired dynamics simultaneously performance indices reach Nash equilibrium. local...

Thermodynamic formulas for investigating systems with density- and/or temperature-dependent particle masses are generally derived from the fundamental derivation equality of thermodynamics. Various problems in previous treatments discussed and modified. Properties strange quark matter bulk strangelets at both zero finite temperature then calculated based on new thermodynamic a mass scaling, which indicates that low-mass near \ensuremath{\beta} equilibrium multiquark states an antistrange...

Goal representation globalized dual heuristic dynamic programming (Gr-GDHP) method is proposed in this paper. A goal neural network integrated into the traditional GDHP providing an internal reinforcement signal and its derivatives to help control learning process. From architecture, it shown that obtained can be able adjust themselves online over time rather than a fixed or predefined function literature. Furthermore, directly contribute objective of critic network, whose process thus...

In this paper, a novel nonlinear learning controller called fuzzy-based goal representation adaptive dynamic programming (Fuzzy-GrADP) is proposed. the proposed GrADP method, network introduced to generate an internal reinforcement signal critic help provide general mapping between input and output actions. Moreover, in architecture, action improved by using fuzzy hyperbolic model, which combines merits of model neural model. Based on back-propagation technique, parameters membership...

Networked control systems (NCSs) provide many benefits, such as higher accuracy and better robustness with the successively increasing computational complexity communication burden. This results in traditional adaptive dynamic programming method having difficulty meeting real-time requirements of industrial systems. In this paper, a novel event-triggered globalized dual heuristic is proposed to reduce required samples while guaranteeing stability system. method, NCSs can communicate update...

The adaptive dynamic programming controller usually needs a long training period because the data usage efficiency is relatively low by discarding samples once used. Prioritized experience replay (ER) promotes important experiences and more efficient in learning control process. This paper proposes integrating an capability of prioritized ER design into heuristic (HDP). First, one time-step backward state-action pair used to tuple and, thus, avoids model network. Second, systematic approach...

Crowd counting in smart surveillance systems plays a crucial role Internet of Things (IoT) and cities, can affect various aspects, such as public safety, crowd management, urban planning. Using data to centrally train model raises significant privacy concerns. Traditional methods try alleviate the concern by reducing focus on individuals, but still needs be thoroughly resolved. In this work, we develop horizontal federated learning (HFL) framework models which preserve simultaneously. This...

In this paper, we propose an optimal control method based on the solution of Hamilton-Jacobi-Bellman (HJB) equation for continuous-time nonlinear system with bounded unknown perturbation. The robust is converted into corresponding appropriate performance index and equivalence transformation proved, i.e., problem can globally asymptotically stabilize system. Adaptive dynamic programming (ADP) approach presented to iteratively approximate obtain policy. A neural network adaptive weights...

An attacker can very possibly make significant damage for the power grid with a proper sequence of timing and attacks. Existing approaches neglect system generation loss also identification critical attack sequences. In this paper, we investigate reinforcement learning approach to identify minimum number attacks/actions reach blackout threshold. The will only have limited topological information systems. Proper state vectors, action vectors reward are designed in smart security environment....