This paper presents Verisig, a hybrid system approach to verifying safety properties of closed-loop systems using neural networks as controllers. We focus on sigmoid-based and exploit the fact that sigmoid is solution quadratic differential equation, which allows us transform network into an equivalent system. By composing network's with plant's, we problem verification can be solved state-of-the-art reachability tools. show decidable for one hidden layer general if Schanuel's conjecture...

This paper addresses the problem of detection and identification sensor attacks in presence transient faults. We consider a system with multiple sensors measuring same physical variable, where some might be under attack provide malicious values. setup, which each provides controller an interval possible values for true value. While approaches exist detecting attacks, they are conservative that treat faults way, thus neglecting fact may faulty measurements at times due to temporary...

This article focuses on the design of safe and attack-resilient Cyber-Physical Systems (CPS) equipped with multiple sensors measuring same physical variable. A malicious attacker may be able to disrupt system performance through compromising a subset these sensors. Consequently, we develop precise resilient sensor fusion algorithm that combines data received from all by taking into account their specified precisions. In particular, note in presence shared bus, which messages are broadcast...

This paper describes a verification case study on an autonomous racing car with neural network (NN) controller. Although several approaches have been recently proposed, they only evaluated low-dimensional systems or constrained environments. To explore the limits of existing approaches, we present challenging benchmark in which NN takes raw LiDAR measurements as input and outputs steering for car. We train dozen NNs using reinforcement learning (RL) show that state art can handle around 40...

This article addresses the problem of verifying safety autonomous systems with neural network (NN) controllers. We focus on NNs sigmoid/tanh activations and use fact that is solution to a quadratic differential equation. allows us convert NN into an equivalent hybrid system cast as verification problem, which can be solved by existing tools. Furthermore, we improve scalability proposed method approximating sigmoid Taylor series worst-case error bounds. Finally, provide evaluation over four...

This paper presents Verisig, a hybrid system approach to verifying safety properties of closed-loop systems using neural networks as controllers. Although techniques exist for input/output the network itself, these methods cannot be used verify (since they work with piecewise-linear constraints that do not capture non-linear plant dynamics). To overcome this challenge, we focus on sigmoid-based and exploit fact sigmoid is solution quadratic differential equation, which allows us transform...

A logger in the cloud capable of keeping a secure, time-synchronized and tamper-evident log medical device patient information allows efficient forensic analysis cases adverse events or attacks on interoperable devices. secure as such must meet requirements confidentiality integrity message logs provide tamper-detection tamper-evidence. In this paper, we propose design for cloud-based using Intel Software Guard Extensions (SGX) Trusted Platform Module (TPM). The proposed receives from dongle...

Recent advances in deep learning have enabled data-driven controller design for autonomous systems. However, verifying safety of such controllers, which are often hard-to-analyze neural networks, remains a challenge. Inspired by compositional strategies program verification, we propose framework and verification network controllers. Our approach is to decompose the task (e.g., car navigation) into sequence subtasks segments track), each corresponding different mode system go straight or...

This paper aims to improve the design of modern Medical Cyber Physical Systems through addition supplemental noninvasive monitors. Specifically, we focus on monitoring arterial blood oxygen content (CaO2), one most closely observed vital signs in operating rooms, currently measured by a proxy -- peripheral hemoglobin saturation (SpO2). While SpO2 is good estimate O2 finger where it measured, delayed measure its arteries. In addition, does not incorporate system dynamics and poor predictor...

The tight interaction between information technology and the physical world inherent in cyber-physical systems (CPS) can challenge traditional approaches for monitoring safety security. Data collected robust CPS is often sparse may lack rich training data describing critical events/attacks. Moreover, operate diverse environments that have significant inter/intra-system variability. Furthermore, monitors are not to sparsity variability result inconsistent performance be trusted Towards...

This work considers the problem of performing resilient sensor fusion using past measurements. In particular, we consider a system with n sensors measuring same physical variable where some might be attacked or faulty. We setup in which each provides controller set possible values for true value. Here, more precise provide smaller sets. Since lot modern multidimensional measurements (e.g. position three dimensions), sets considered this are polyhedra.

This work considers the problem of attack-resilient sensor fusion in an autonomous system where multiple sensors measure same physical variable. A malicious attacker may corrupt a subset these and send wrong measurements to controller on their behalf, potentially compromising safety system. We formalize goals constraints such who also wants avoid detection by argue that attacker's capabilities depend amount information she has about correct sensors' measurements. In presence shared bus...

This report presents the results of a friendly competition for formal verification continuous and hybrid systems with artificial intelligence (AI) components. Specifically, machine learning (ML) components in cyber-physical (CPS), such as feedforward neural networks used feedback controllers closed-loop are considered, which is class classically known intelligent control systems, or more modern specific terms, network (NNCS). We broadly refer to this category AI NNCS (AINNCS). The took place...

This paper considers the problem of incorporating context in medical cyber-physical systems (MCPS) applications for purpose improving performance MCPS detectors. In particular, many additional data could be used to conclude that actual measurements might noisy or wrong (e.g., machine settings indicate is improperly attached patient); we call such context. The first contribution this work formal definition context, namely information whose presence associated with a change measurement model...

As devices in the Internet of Things (IoT) increase number and integrate with everyday lives, large amounts personal information will be generated. With multiple discovered vulnerabilities current IoT networks, a malicious attacker might able to get access misuse this data. Thus, logger that stores securely would make it possible perform forensic analysis case such attacks target valuable In paper, we propose LogSafe, scalable, fault-tolerant leverages use Intel Software Guard Extensions...

In this tutorial, we present a design methodology for medical parameter-invariant monitors. We begin by providing motivational review of currently employed alarm techniques, followed the introduction approach. Finally, case study example to demonstrate critical shunt detection in infants during surgical procedures.

This paper presents the context-aware filter, an estimation technique that incorporates context measurements, in addition to regular continuous measurements. Context measurements provide binary information about system's which is not directly encoded state; examples include a robot detecting nearby building using image processing or medical device alarming vital sign has exceeded predefined threshold. These can only be received from certain states and therefore modeled as function of current...

Autonomous systems operating in uncertain environments under the effects of disturbances and noises can reach unsafe states even while using finetuned controllers precise sensors actuators. To provide safety guarantees on such during motion planning operations, reachability analysis (RA) has been demonstrated to be a powerful tool. RA, however, suffers from computational complexity, especially when dealing with intricate characterized by high-order dynamics, making it hard deploy for runtime...

This article is concerned with the security of modern Cyber-Physical Systems in presence transient sensor faults. We consider a system multiple sensors measuring same physical variable, where each provides an interval all possible values true state. note that some might output faulty readings and others may be controlled by malicious attacker. Differing from previous works, this article, we aim to distinguish between faults attacks develop attack detection algorithm for latter only. To do...

We propose a simplified high-level programming language based on blocks and links dragged workspace which generates the skeleton code for robotic applications involving different types of robots. In order to develop such that still can guarantee flexibility in term implementation, our approach takes advantage robot operating system (ROS). ROS is open source meta-operating provides message passing structure between processes (or nodes) across network (inter-process communication). framework,...

This paper considers the problem of sensor attack detection for multiple operating mode systems, building upon an existing method that uses a transient fault model with fixed parameters. For system, would have to use most conservative parameters preserve soundness in detection, thus not being effective some modes. To address this problem, we propose adaptive appropriate parameter values accordance change system. The benefit our proposed system is demonstrated using real measurement data...

This report presents the results of a friendly competition for formal verification continuous and hybrid systems with artificial intelligence (AI) components. Specifically, machine learning (ML) components in cyber-physical (CPS), such as feedforward neural networks used feedback controllers closed-loop are considered, which is class classically known intelligent control systems, or more modern specific terms, network (NNCS). For future iterations, we broadly refer to this category AI NNCS...

This paper describes a verification case study on an autonomous racing car with neural network (NN) controller. Although several approaches have been proposed over the last year, they only evaluated low-dimensional systems or constrained environments. To explore limits of existing approaches, we present challenging benchmark in which NN takes raw LiDAR measurements as input and outputs steering for car. We train dozen NNs using two reinforcement learning algorithms show that state art can...