Stochastic Activity Month

September 2012

"Stochastic Operations Management"

SUMMARY

Stochastic Operations Management (SOM). SOM is concerned with decision making under uncertainty. In real life, examples abound of situations in which decisions must be made, costs must be minimized, etc., while only partial information is available. One may think of revenue management, inventory control, staffing of call centers, the organization of warehouses, traffic management, and many other topics. Stochastic operations research uses, develops and provides tools from probability theory and operations research to make good decisions.

ORGANIZERS

PROGRAMME

VISITORS

ABSTRACTS

Eugene Feinberg

Infinite-state discrete-time Markov Decision Processes

We describe recent developments in optimization of Markov Decision Processes with uncountable state spaces, unbounded cost functions and noncompact action sets under two major objective criteria: the total expected discounted costs and average costs per unit time. The talk presents the results on the existence of solutions to optimality equations and inequalities, the existence of stationary optimal policies and convergence of value iteration and vanishing discount factor procedures. We also discuss Fatou's lemma for weakly converging measures and an extension of Berge's theorem of the minimum to noncompact action sets. This study is mainly motivated by applications to inventory control. Other possible applications include control of the workload in queues, economics, and Partially Observable Markov Decision Processes.

PRESENTATION

Infinite-state continuous-time Markov Decision Processes

This is a talk on optimization of continuous-time Markov Decision Processes with infinite state spaces and possibly unbounded transition intensities. We describe how such models can be defined and discuss solutions to Kolmogorov's equations, sufficiency of Markov policies, and reduction to discrete time. The major motivation is control of queues and networks with customer reneging or with unlimited numbers of servers.

PRESENTATION

Mor Harchol-Balter (Carnegie Mellon University, Computer Science Department)

Exact Analysis of M/M/k with Setup Times and Other Open Variants

Setup times (also known as exceptional first service) are an important concept in queueing theory. When a server is not in use, it is common to turn it off, so as to save power or cost; hence a new arrival often needs to wait for a server to turn on (a setup time) before the job can run. While the effect of setup times is well-understood for a single-server queue, in the case of multiple servers almost nothing is known. We start by presenting approximations for the M/M/k/Setup system based on our paper in Performance 2010. We then present our current research on the first exact analysis of the M/M/k/Setup system. In doing so, we introduce a new analysis technique, which we show can be applied to solving many other related open problems, including the M/M/k/Setup/DelayedOff system, where we purposely wait before turning off a server.
(Joint work with: Anshul Gandhi, Sherwin Doroudi, Alan Scheller-Wolf, and Ivo Adan.)
 

SPONSORS