Model-based reinforcement learning approaches carry the promise of being data efficient. However, due to challenges in dynamics models that sufficiently match real-world dynamics, they struggle achieve same asymptotic performance as model-free methods. We propose Model-Based Meta-Policy-Optimization (MB-MPO), an approach foregoes strong reliance on accurate learned models. Using ensemble dynamic models, MB-MPO meta-learns a policy can quickly adapt any model with one gradient step. This...

Given a set of empirical observations, conditional density estimation aims to capture the statistical relationship between variable $\mathbf{x}$ and dependent $\mathbf{y}$ by modeling their probability $p(\mathbf{y}|\mathbf{x})$. The paper develops best practices for finance applications with neural networks, grounded on mathematical insights evaluations. In particular, we introduce noise regularization data normalization scheme, alleviating problems over-fitting, initialization...

Credit assignment in Meta-reinforcement learning (Meta-RL) is still poorly understood. Existing methods either neglect credit to pre-adaptation behavior or implement it naively. This leads poor sample-efficiency during meta-training as well ineffective task identification strategies. paper provides a theoretical analysis of gradient-based Meta-RL. Building on the gained insights we develop novel meta-learning algorithm that overcomes both issue and previous difficulties estimating...

We present a novel deep neural network architecture for representing robot experiences in an episodic-like memory that facilitates encoding, recalling, and predicting action experiences. Our proposed unsupervised episodic model as follows: First, encodes observed actions latent vector space and, based on this second, infers most similar episodes previously experienced, third, reconstructs original episodes, finally, predicts future frames end-to-end fashion. Results show conceptually are...

We introduce PACOH-RL, a novel model-based Meta-Reinforcement Learning (Meta-RL) algorithm designed to efficiently adapt control policies changing dynamics. PACOH-RL meta-learns priors for the dynamics model, allowing swift adaptation new with minimal interaction data. Existing Meta-RL methods require abundant meta-learning data, limiting their applicability in settings such as robotics, where data is costly obtain. To address this, incorporates regularization and epistemic uncertainty...

We present SIM-FSVGD for learning robot dynamics from data. As opposed to traditional methods, leverages low-fidelity physical priors, e.g., in the form of simulators, regularize training neural network models. While accurate already low data regime, scales and excels also when more is available. empirically show that with implicit priors results mean model estimation as well precise uncertainty quantification. demonstrate effectiveness bridging sim-to-real gap on a high-performance RC...

Bayesian structure learning allows inferring network from data while reasoning about the epistemic uncertainty -- a key element towards enabling active causal discovery and designing interventions in real world systems. In this work, we propose general, fully differentiable framework for (DiBS) that operates continuous space of latent probabilistic graph representation. Contrary to existing DiBS is agnostic form local conditional distributions joint posterior inference both distribution...

Learning the causal structure that underlies data is a crucial step towards robust real-world decision making. The majority of existing work in inference focuses on determining single directed acyclic graph (DAG) or Markov equivalence class thereof. However, aspect to acting intelligently upon knowledge about which has been inferred from finite demands reasoning its uncertainty. For instance, planning interventions find out more mechanisms govern our requires quantifying epistemic...

Modelling statistical relationships beyond the conditional mean is crucial in many settings. Conditional density estimation (CDE) aims to learn full probability from data. Though highly expressive, neural network based CDE models can suffer severe over-fitting when trained with maximum likelihood objective. Due inherent structure of such models, classical regularization approaches parameter space are rendered ineffective. To address this issue, we develop a model-agnostic noise method for...

When data are scarce meta-learning can improve a learner's accuracy by harnessing previous experience from related learning tasks. However, existing methods have unreliable uncertainty estimates which often overconfident. Addressing these shortcomings, we introduce novel framework, called F-PACOH, that treats meta-learned priors as stochastic processes and performs meta-level regularization directly in the function space. This allows us to steer probabilistic predictions of meta-learner...

Meta-Learning aims to speed up the learning process on new tasks by acquiring useful inductive biases from datasets of related tasks. While, in practice, number available is often small, most existing approaches assume an abundance tasks; making them unrealistic and prone overfitting. A central question meta-learning literature how regularize ensure generalization unseen In this work, we provide a theoretical analysis using PAC-Bayesian theory present bound for meta-learning, which was first...

Existing generalization measures that aim to capture a model's simplicity based on parameter counts or norms fail explain in overparameterized deep neural networks. In this paper, we introduce new, theoretically motivated measure of network's which call prunability: the smallest \emph{fraction} parameters can be kept while pruning without adversely affecting its training loss. We show is highly predictive performance across large set convolutional networks trained CIFAR-10, does not grow...

Inferring causal structure poses a combinatorial search problem that typically involves evaluating structures with score or independence test. The resulting is costly, and designing suitable scores tests capture prior knowledge difficult. In this work, we propose to amortize learning. Rather than searching over structures, train variational inference model directly predict the from observational interventional data. This allows our acquire domain-specific inductive biases for discovery...

In robotics, optimizing controller parameters under safety constraints is an important challenge. Safe Bayesian optimization (BO) quantifies uncertainty in the objective and to safely guide exploration such settings. Hand-designing a suitable probabilistic model can be challenging, however. presence of unknown constraints, it crucial choose reliable hyper-parameters avoid violations. Here, we propose data-driven approach this problem by meta-learning priors for safe BO from offline data. We...

Meta-learning can successfully acquire useful inductive biases from data. Yet, its generalization properties to unseen learning tasks are poorly understood. Particularly if the number of meta-training is small, this raises concerns about overfitting. We provide a theoretical analysis using PAC-Bayesian framework and derive novel bounds for meta-learning. Using these bounds, we develop class PAC-optimal meta-learning algorithms with performance guarantees principled meta-level regularization....

Existing generalization bounds fail to explain crucial factors that drive the of modern neural networks. Since such often hold uniformly over all parameters, they suffer from over-parametrization and account for strong inductive bias initialization stochastic gradient descent. As an alternative, we propose a novel optimal transport interpretation problem. This allows us derive instance-dependent depend on local Lipschitz regularity learned prediction function in data space. Therefore, our...

Inferring causal structures from experimentation is a central task in many domains. For example, biology, recent advances allow us to obtain single-cell expression data under multiple interventions such as drugs or gene knockouts. However, the targets of are often uncertain unknown and number observations limited. As result, standard discovery methods can no longer be reliably used. To fill this gap, we propose Bayesian framework (BaCaDI) for discovering reasoning about structure that...