Autonomous driving has achieved significant milestones in research and development over the last two decades. There is increasing interest field as deployment of autonomous vehicles (AVs) promises safer more ecologically friendly transportation systems. With rapid progress computationally powerful artificial intelligence (AI) techniques, AVs can sense their environment with high precision, make safe real-time decisions, operate reliably without human intervention. However, intelligent...

Autonomous driving has achieved significant milestones in research and development over the last decade. There is increasing interest field as deployment of self-operating vehicles promises safer more ecologically friendly transportation systems. With rise computationally powerful artificial intelligence (AI) techniques, autonomous can sense their environment with high precision, make safe real-time decisions, operate reliably without human intervention. However, intelligent decision-making...

The rapid growth of research in explainable artificial intelligence (XAI) follows on two substantial developments. First, the enormous application success modern machine learning methods, especially deep and reinforcement learning, have created high expectations for industrial, commercial, social value. Second, emerging growing concern creating ethical trusted AI systems, including compliance with regulatory principles to ensure transparency trust. These threads a kind “perfect storm”...

Significant progress has been made recently in developing few-shot object segmentation methods. Learning is shown to be successful settings, using pixel-level, scribbles and bounding box supervision. This paper takes another approach, i.e., only requiring image-level label for segmentation. We propose a novel multi-modal interaction module that utilizes co-attention mechanism both visual word embedding. Our model labels achieves 4.8% improvement over previously proposed It also outperforms...

In distributional reinforcement learning (RL), the estimated distribution of value function models both parametric and intrinsic uncertainties. We propose a novel efficient exploration method for deep RL that has two components. The first is decaying schedule to suppress uncertainty. second an bonus calculated from upper quantiles learned distribution. Atari 2600 games, our outperforms QR-DQN in 12 out 14 hard games (achieving 483 \% average gain across 49 cumulative rewards over with big...

Discounted reinforcement learning is fundamentally incompatible with function approximation for control in continuing tasks. It not an optimization problem its usual formulation, so when using there no optimal policy. We substantiate these claims, then go on to address some misconceptions about discounting and connection the average reward formulation. encourage researchers adopt rigorous approaches, such as maximizing reward,

We present the first provably convergent two-timescale off-policy actor-critic algorithm (COF-PAC) with function approximation. Key to COF-PAC is introduction of a new critic, emphasis which trained via Gradient Emphasis Learning (GEM), novel combination key ideas Temporal Difference and Emphatic Learning. With help critic canonical value we show convergence for COF-PAC, where critics are linear actor can be nonlinear.

We consider the problem of mapless collision-avoidance navigation where humans are present using 2D laser scans. Our proposed method uses ego-safety to measure collision from robot's perspective and social-safety impact actions on surrounding pedestrians. Specifically, part predicts intrusion action into interaction area with humans. train policy reinforcement learning a simple simulator directly evaluate learned in Gazebo real robot tests. Experiments show smoothly transferred different...

The popular Proximal Policy Optimization (PPO) algorithm approximates the solution in a clipped policy space. Does there exist better policies outside of this space? By using novel surrogate objective that employs sigmoid function (which provides an interesting way exploration), we found answer is "YES", and are fact located very far from We show PPO insufficient "off-policyness", according to off-policy metric called DEON. Our explores much larger space than PPO, it maximizes Conservative...

In this paper, we propose the Quantile Option Architecture (QUOTA) for exploration based on recent advances in distributional reinforcement learning (RL). QUOTA, decision making is quantiles of a value distribution, not only mean. QUOTA provides new dimension via use both optimism and pessimism distribution. We demonstrate performance advantage challenging video games physical robot simulators.

In this paper, we propose an actor ensemble algorithm, named ACE, for continuous control with a deterministic policy in reinforcement learning. use (i.e., multiple actors) to search the global maxima of critic. Besides perspective, also formulate ACE option framework by extending option-critic architecture intra-option policies, revealing relationship between and options. Furthermore, perform look-ahead tree those actors learned value prediction model, resulting refined estimation. We...

In this paper we introduce the concept of pseudo-MDPs to develop abstractions. Pseudo-MDPs relax requirement that transition kernel has be a probability kernel. We show new framework captures many existing also factored linear action models; special case. Again, relation models and works are discussed. use general theory for bounding suboptimality policies derived from pseudo-MDPs. Specializing framework, recover results. give leastsquares approach constrained optimization learning model as...

In deep neural network, the cross-entropy loss function is commonly used for classification. Minimizing equivalent to maximizing likelihood under assumptions of uniform feature and class distributions. It belongs generative training criteria which does not directly discriminate correct from competing classes. We propose a discriminative with negative log ratio between significantly outperforms on CIFAR-10 image classification task.

In this paper we consider the problem of finding a good policy given some batch data.We propose new approach, LAM-API, that first builds so-called linear action model (LAM) from data and then uses learned collected in approximate iteration (API) to find policy.A natural choice for evaluation step algorithm is use least-squares temporal difference (LSTD) learning algorithm.Empirical results on three benchmark problems show particular instance LAM-API performs competitively as compared with...

This paper extends many of the recent popular policy evaluation algorithms to a generalized framework that includes least-squares temporal difference (LSTD) learning, (LSPE) and variant incremental LSTD (iLSTD). The basis this extension is preconditioning technique solves stochastic model equation. also studies three significant issues new framework: it presents rule step-size can be computed online, provides an iterative way apply preconditioning, reduces complexity related near (TD) learning.

Learning an effective representation for high-dimensional data is a challenging problem in reinforcement learning (RL). Deep (DRL) such as Q networks (DQN) achieves remarkable success computer games by deeply encoded from convolution networks. In this paper, we propose simple yet very method with DRL algorithms. Our key insight that features learned algorithms are highly correlated, which interferes learning. By adding regularized loss penalizes correlation latent (with only slight...

Dyna is an architecture for model based reinforcement learning (RL), where simulated experience from a used to update policies or value functions. A key component of search control, the mechanism generate state and action which agent queries model, remains largely unexplored. In this work, we propose such states by using trajectory obtained Hill Climbing (HC) current estimate function. This has effect propagating high regions preemptively updating estimates that likely visit next. We derive...

Recent developments in Neural Architecture Search (NAS) resort to training the supernet of a predefined search space with weight sharing speed up architecture evaluation. These include random schemes, as well various schemes based on optimization or reinforcement learning, particular policy gradient, that aim optimize parametric distribution and shared model weights simultaneously. In this paper, we focus efficiently exploring important region neural learning. We propose Deep Deterministic...