Stochastic Activity Month

September 2012


"Stochastic Operations Management"









Stochastic Operations Management (SOM). SOM is concerned with decision making under uncertainty. In real life, examples abound of situations in which decisions must be made, costs must be minimized, etc., while only partial information is available. One may think of revenue management, inventory control, staffing of call centers, the organization of warehouses, traffic management, and many other topics. Stochastic operations research uses, develops and provides tools from probability theory and operations research to make good decisions.



Ivo Adan TU Eindhoven
Richard Boucherie U Twente
Onno Boxma TU Eindhoven
Geert-Jan van Houtum TU Eindhoven
Michel Mandjes UvA
Bert Zwart CWI/ VU Amsterdam



4 September Kick-off Seminar Mor Harchol-Balter
5-6 September Mini Course Infinite-state discrete-time Markov Decision Processes (1)

Infinite-state continuous-time Markov Decision Processes (2)

Eugene Feinberg
10-12 September Workshop Pricing and Staffing Keynote speakers:
- Frank Kelly
- Marty Reiman
- Assaf Zeevi, ...
18 September
Academia meets industry (day of lectures) Industrial applications of
SOM, with duo lectures from academia and industry
Pairs of speakers:
- Geert-Jan van Houtum +  Aafke Visser-van Boekel (OcÚ-Technologies B.V.)
- Ivo Adan + Vanderlande researcher
- Michel Mandjes + IBIS researcher
- Richard Boucherie + Fred Boer (LUMC)
- Marko Boon + Tim Jonker (Municipality Eindhoven)
27 September Day of lectures Gaussian and Levy processes and their applications in queues, finance and risk. Speakers:
Krzysztof Dębicki
- Offer Kella
- Michel Mandjes
?? Mini courses - Pricing - Assaf Zeevi



Shaul Bar-Lev University of Haifa
Krzysztof Dębicki Univ. Wroclaw
Eugene Feinberg State University of New York at Stony Brook
Mor Harchol-Balter Carnegie-Mellon University, Pittsburgh
Offer Kella Hebrew University of Jerusalem
Frank Kelly University of Cambridge
Vidyadhar Kulkarni University of North-Carolina
David Perry University of Haifa
Adam Wierman Caltech
Yifan Xu University of Binghamton
Assaf Zeevi Columbia University




Eugene Feinberg

Infinite-state discrete-time Markov Decision Processes

We describe recent developments in optimization of Markov Decision Processes with uncountable state spaces, unbounded cost functions and noncompact action sets under two major objective criteria: the total expected discounted costs and average costs per unit time. The talk presents the results on the existence of solutions to optimality equations and inequalities, the existence of stationary optimal policies and convergence of value iteration and vanishing discount factor procedures. We also discuss Fatou's lemma for weakly converging measures and an extension of Berge's theorem of the minimum to noncompact action sets. This study is mainly motivated by applications to inventory control. Other possible applications include control of the workload in queues, economics, and Partially Observable Markov Decision Processes.


Infinite-state continuous-time Markov Decision Processes

This is a talk on optimization of continuous-time Markov Decision Processes with infinite state spaces and possibly unbounded transition intensities. We describe how such models can be defined and discuss solutions to Kolmogorov's equations, sufficiency of Markov policies, and reduction to discrete time. The major motivation is control of queues and networks with customer reneging or with unlimited numbers of servers.


Mor Harchol-Balter (Carnegie Mellon University, Computer Science Department)

Exact Analysis of M/M/k with Setup Times and Other Open Variants

Setup times (also known as exceptional first service) are an important concept in queueing theory. When a server is not in use, it is common to turn it off, so as to save power or cost; hence a new arrival often needs to wait for a server to turn on (a setup time) before the job can run. While the effect of setup times is well-understood for a single-server queue, in the case of multiple servers almost nothing is known. We start by presenting approximations for the M/M/k/Setup system based on our paper in Performance 2010. We then present our current research on the first exact analysis of the M/M/k/Setup system. In doing so, we introduce a new analysis technique, which we show can be applied to solving many other related open problems, including the M/M/k/Setup/DelayedOff system, where we purposely wait before turning off a server.
(Joint work with: Anshul Gandhi, Sherwin Doroudi, Alan Scheller-Wolf, and Ivo Adan.)