Under a Bayesian framework, we formulate the fully sequential sampling and selection decision in statistical ranking as stochastic control problem, derive associated Bellman equation. Using value function approximation, an approximately optimal allocation policy. We show that this policy is not only computationally efficient but also possesses both one-step-ahead asymptotic optimality for independent normal distributions. Moreover, proposed easily generalizable approximate dynamic...

In this paper, we propose a new unbiased stochastic derivative estimator in framework that can handle discontinuous sample performances with structural parameters. This work extends the three most popular estimators: (1) infinitesimal perturbation analysis (IPA), (2) likelihood ratio (LR) method, and (3) weak to setting where they did not previously apply. Examples probability constraints, control charts, financial derivatives demonstrate broad applicability of proposed framework. The...

We formulate the statistical selection problem in a general dynamic framework comprising fully sequential sampling allocation and optimal design selection. Because traditional probability of correct measure is not sufficient to capture both aspects this more framework, we introduce integrated better characterize objective. As result, usual policy choosing with largest sample mean as estimate best no longer necessarily optimal. Rather, choose that maximizes posterior selection, which function...

In this note, we consider the statistical ranking and selection problem of finding best alternative when performances each must be estimated by sampling. We provide a myopic allocation policy that asymptotically achieves sampling ratios given optimal computing budget allocation, an approximate solution large deviations rate for decreasing probability false selection. analyze asymptotic ratio both known variances unknown under Bayesian framework. Numerical results substantiate theoretical results.

Contextual simulation optimization problems have attracted great attention in the healthcare, commercial, and financial fields because of need for personalized decision making. Besides randomness outputs, larger solution space makes learning more challenging. In current work, Li, Lam, Peng use a Gaussian mixture model (GMM) as basic technique to deal with this difficulty. To address computational challenge updating GMM-based Bayesian posterior, they present computationally efficient...

The probabilistic diffusion model (DM), generating content by inferencing through a recursive chain structure, has emerged as powerful framework for visual generation. After pre-training on enormous unlabeled data, the needs to be properly aligned meet requirements downstream applications. How efficiently align foundation DM is crucial task. Contemporary methods are either based Reinforcement Learning (RL) or truncated Backpropagation (BP). However, RL and BP suffer from low sample...

We consider selecting the top-m alternatives from a finite number of via Monte Carlo simulation. Under Bayesian framework, we formulate sampling decision as stochastic dynamic programming problem and develop sequential policy that maximizes value function approximation one-step look ahead. To show asymptotic optimality proposed procedure, asymptotically optimal ratios optimize large deviations rate probability false selection for have been rigorously defined. The is not only proved to be...

Common random numbers and the standard clock method are examples of effective variance reduction techniques that also share information simulation resources when generating realizations different simulated systems whose performances being compared. This sharing computing potentially widely computational requirements for models important considerations in allocating replications among candidate designs with objective maximizing probability selecting best design, we formulate optimal budget...

Simulation is often used to estimate the performance of alternative system designs for selecting best. For a complex system, high-fidelity simulation usually time-consuming and expensive. In this paper, we provide new framework that integrates information from multifidelity models increase efficiency A Gaussian mixture model introduced capture clustering in models. Posterior obtained by analysis incorporates both cluster-wise idiosyncratic each design. We propose budget allocation method...

A Simulation-Based Approach for Calibrating Stochastic Models

We present a gradient-based algorithm for solving class of simulation optimization problems in which the objective function is quantile output random variable. In contrast with existing (quantile derivative) estimation techniques, aim to eliminate estimator bias by gradually increasing sample size, our incorporates novel recursive procedure that only requires single at each step simultaneously obtain and derivative estimators are asymptotically unbiased. show these estimators, when coupled...

In this note, we study a simulation optimization problem of selecting the alternative with best performance from finite set, or so-called ranking and selection problem, in special low-confidence scenario. The most popular sampling allocation procedures do not perform well scenario, because they all ignore certain induced correlations that significantly affect probability correct We propose gradient-based myopic policy takes into account, reflecting tradeoff between correlation two factors...

Artificial neural network (ANN) has been widely used in automation. However, the vulnerability of ANN under certain attacks poses a security threat to critical automation systems. Previous research shown that adding noise ANNs can enhance robustness. Nonetheless, striking balance between robustness and task performance remains challenging, as excessive improves but hampers performance, while low offers minor improvement. In this work, we propose learn distribution optimal injected noise,...

Monte Carlo simulation is a commonly used tool for evaluating the performance of complex stochastic systems. In practice, can be expensive, especially when comparing large number alternatives, thus motivating need to intelligently allocate replications. Given finite set alternatives whose means are estimated via simulation, we consider problem determining subset that have smaller than fixed threshold. A dynamic sampling procedure possesses not only asymptotic optimality, but also desirable...

In Peng et al. (2015b), we show that the probability of correct selection (PCS), a commonly used metric, is not necessarily monotonically increasing with respect to number simulation replications. A simple counterexample where PCS may decrease additional sampling provided motivate problem. The reference identifies induced correlations as source non-monotonicity, and characterizes general scenario under which phenomenon occurs by condition coefficient variations difference in sample means are...

We propose a dynamic sampling allocation and selection paradigm for finding the alternative with optimal quantile in Bayesian framework. Myopic policies (MAPs), analogous to existing methods classic ranking selecting mean, computationally efficient are derived quantile. Under certain conditions, we prove that proposed MAPs procedures consistent, which means best would be eventually correctly selected as sample size goes infinity. Numerical experiments demonstrate schemes can significantly...

We consider the popular tree-based search strategy within framework of reinforcement learning, Monte Carlo Tree Search (MCTS), in context finite-horizon Markov decision process. propose a dynamic sampling tree policy that efficiently allocates limited computational budget to maximize probability correct selection best action at root node tree. Experimental results on Tic-Tac-Toe and Gomoku show proposed is more efficient than other competing methods.

The software and data in this repository are a snapshot of the that were used research reported on paper An Efficient Node Selection Policy for Monte Carlo Tree Search with Neural Networks by Xiaotian Liu, Yijie Peng, Gongbo Zhang, Ruihan Zhou.

Parameter estimation and statistical inference are challenging problems for stochastic volatility (SV) models, especially those driven by pure jump Lévy processes. Maximum likelihood (MLE) is usually preferred when a parametric model correctly specified, but traditional MLE implementation SV models computationally infeasible due to high dimensionality of the integral involved. To overcome this difficulty, we propose gradient-based simulated method under hidden Markov structure which covers...

We consider a simulation optimization problem for context-dependent decision-making, which aims to determine the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$m$</tex-math> </inline-formula> designs all contexts. Under Bayesian framework, we formulate optimal dynamic sampling decision as stochastic programming and develop sequential policy efficiently learn performance of each design under context....