**Stochastic
Activity Month**

**September 2012**

**"Stochastic
Operations Management"**

SUMMARY |
SPONSORS |
VISITORS |

**ORGANIZERS**

Ivo Adan | TU Eindhoven |

Richard Boucherie | U Twente |

Onno Boxma | TU Eindhoven |

Geert-Jan van Houtum | TU Eindhoven |

Michel Mandjes | UvA |

Bert Zwart | CWI/ VU Amsterdam |

4 September | Kick-off | Seminar | Mor Harchol-Balter |

5-6 September | Mini Course | Infinite-state discrete-time
Markov Decision Processes (1) Infinite-state continuous-time Markov Decision Processes |
Eugene Feinberg |

10-12 September | Workshop | Pricing and Staffing | Keynote speakers: - Frank Kelly - Marty Reiman - Assaf Zeevi, ... |

18 SeptemberPOSTPONED |
Academia meets industry (day of lectures) | Industrial applications of, with duo lectures from academia and industrySOM |
Pairs of speakers: - Geert-Jan van Houtum + Aafke Visser-van Boekel (Océ-Technologies B.V.) - Ivo Adan + Vanderlande researcher - Michel Mandjes + IBIS researcher - Richard Boucherie + Fred Boer (LUMC) - Marko Boon + Tim Jonker (Municipality Eindhoven) |

27 September | Day of lectures | Gaussian and Levy processes and their applications in queues, finance and risk. | Speakers: - Krzysztof Dębicki - Offer Kella - Michel Mandjes |

?? | Mini courses | - Pricing | - Assaf Zeevi |

Shaul Bar-Lev | University of Haifa |

Krzysztof Dębicki | Univ. Wroclaw |

Eugene Feinberg | State University of New York at Stony Brook |

Mor Harchol-Balter | Carnegie-Mellon University, Pittsburgh |

Offer Kella | Hebrew University of Jerusalem |

Frank Kelly | University of Cambridge |

Vidyadhar Kulkarni | University of North-Carolina |

David Perry | University of Haifa |

Adam Wierman | Caltech |

Yifan Xu | University of Binghamton |

Assaf Zeevi | Columbia University |

**Eugene
Feinberg**

**Infinite-state discrete-time
Markov Decision Processes **

We describe recent developments in optimization of Markov Decision Processes
with uncountable state spaces, unbounded cost functions and noncompact action
sets under two major objective criteria: the total expected discounted costs and
average costs per unit time. The talk presents the results on the existence of
solutions to optimality equations and inequalities, the existence of stationary
optimal policies and convergence of value iteration and vanishing discount
factor procedures. We also discuss Fatou's lemma for weakly converging measures
and an extension of Berge's theorem of the minimum to noncompact action sets.
This study is mainly motivated by applications to inventory control. Other
possible applications include control of the workload in queues, economics, and
Partially Observable Markov Decision Processes.

**
PRESENTATION**

**Infinite-state continuous-time Markov Decision Processes **

This is a talk on optimization of continuous-time Markov Decision Processes with
infinite state spaces and possibly unbounded transition intensities. We describe
how such models can be defined and discuss solutions to Kolmogorov's equations,
sufficiency of Markov policies, and reduction to discrete time. The major
motivation is control of queues and networks with customer reneging or with
unlimited numbers of servers.

**Mor
Harchol-Balter (Carnegie Mellon University, Computer Science Department)**

Exact Analysis of M/M/k with Setup Times and Other Open Variants

Setup times (also known as
exceptional first service) are an important concept in queueing theory. When a
server is not in use, it is common to turn it off, so as to save power or cost;
hence a new arrival often needs to wait for a server to turn on (a setup time)
before the job can run. While the effect of setup times is well-understood for a
single-server queue, in the case of multiple servers almost nothing is known. We
start by presenting approximations for the M/M/k/Setup system based on our paper
in Performance 2010. We then present our current research on the first exact
analysis of the M/M/k/Setup system. In doing so, we introduce a new analysis
technique, which we show can be applied to solving many other related open
problems, including the M/M/k/Setup/DelayedOff system, where we purposely wait
before turning off a server.

(Joint work with: Anshul Gandhi, Sherwin Doroudi, Alan Scheller-Wolf, and Ivo
Adan.)