Demand response (DR) for residential and small commercial buildings is estimated to account as much 65% of the total energy savings potential DR, previous work shows that a fully automated management system (EMS) necessary prerequisite DR in these areas. In this paper, we propose novel EMS formulation problems sectors. Specifically, formulate EMS's rescheduling problem reinforcement learning (RL) problem, argue RL can be approximately solved by decomposing it over device clusters. Compared...

We propose randomized least-squares value iteration (RLSVI) -- a new reinforcement learning algorithm designed to explore and generalize efficiently via linearly parameterized functions. explain why versions of that use Boltzmann or epsilon-greedy exploration can be highly inefficient, we present computational results demonstrate dramatic efficiency gains enjoyed by RLSVI. Further, establish an upper bound on the expected regret RLSVI demonstrates near-optimality in tabula rasa context. More...

A search engine usually outputs a list of $K$ web pages. The user examines this list, from the first page to last, and chooses attractive page. This model behavior is known as cascade model. In paper, we propose cascading bandits, learning variant where objective identify most items. We formulate our problem stochastic combinatorial partial monitoring problem. two algorithms for solving it, CascadeUCB1 CascadeKL-UCB. also prove gap-dependent upper bounds on regret these derive lower bound in...

A stochastic combinatorial semi-bandit is an online learning problem where at each step a agent chooses subset of ground items subject to constraints, and then observes weights these receives their sum as payoff. In this paper, we close the computationally sample efficient in semi-bandits. particular, analyze UCB-like algorithm for solving problem, which known be efficient; prove $O(K L (1 / Δ) \log n)$ $O(\sqrt{K n n})$ upper bounds on its $n$-step regret, $L$ number items, $K$ maximum...

We study the use of randomized value functions to guide deep exploration in reinforcement learning. This offers an elegant means for synthesizing statistically and computationally efficient with common practical approaches function present several learning algorithms that leverage demonstrate their efficacy through computational studies. also prove a regret bound establishes statistical efficiency tabular representation.

We address the problem of best policy identification in preference-based reinforcement learning (PbRL), where occurs from noisy binary preferences over trajectory pairs rather than explicit numerical rewards. This approach is useful for post-training optimization generative AI models during multi-turn user interactions, preference feedback more robust handcrafted reward models. In this setting, driven by both an offline dataset -- collected a rater unknown 'competence' and online data with...

A matroid is a notion of independence in combinatorial optimization which closely related to computational efficiency. In particular, it well known that the maximum constrained modular function can be found greedily if and only constraints are associated with matroid. this paper, we bring together ideas bandits matroids, propose new class bandits, bandits. The objective these problems learn how maximize on This stochastic initially unknown. We practical algorithm for solving our problem,...

A stochastic combinatorial semi-bandit is an online learning problem where at each step a agent chooses subset of ground items subject to constraints, and then observes weights these receives their sum as payoff. In this paper, we consider efficient in large-scale semi-bandits with linear generalization, solution, propose two algorithms called Combinatorial Linear Thompson Sampling (CombLinTS) UCB (CombLinUCB). Both are computationally long the offline version can be solved efficiently. We...

Most recommender systems recommend a list of items. The user examines the list, from first item to last, and often chooses attractive does not examine rest. This type behavior can be modeled by cascade model. In this work, we study cascading bandits, an online learning variant model where goal is $K$ most items large set $L$ candidate We propose two algorithms for solving problem, which are based on idea linear generalization. key in our solutions that learn predictor attraction...

Multi-armed bandit (MAB) is a class of online learning problems where agent aims to maximize its expected cumulative reward while repeatedly selecting pull arms with unknown distributions. We consider scenario the distributions may change in piecewise-stationary fashion at time steps. show that by incorporating simple change-detection component classic UCB algorithms detect and adapt changes, our so-called M-UCB algorithm can achieve nearly optimal regret bound on order $O(\sqrt{MKT\log...

Thompson sampling is an algorithm for online decision problems where actions are taken sequentially in a manner that must balance between exploiting what known to maximize immediate performance and investing accumulate new information may improve future performance. The addresses broad range of computationally efficient therefore enjoying wide use. This tutorial covers the its application, illustrating concepts through examples, including Bernoulli bandit problems, shortest path product...

We propose a bandit algorithm that explores by randomizing its history of rewards. Specifically, it pulls the arm with highest mean reward in non-parametric bootstrap sample pseudo design rewards such is optimistic sufficiently high probability. call our Giro, which stands for garbage in, out. analyze Giro Bernoulli and derive $O(K Δ^{-1} \log n)$ bound on $n$-round regret, where $Δ$ difference expected optimal best suboptimal arms, $K$ number arms. The main advantage exploration easily...