Probabilistic models are widely used for description and an analysis of various processes in system reliability, risk, queuing, data communication, logistic and storage systems. The book contains various applications of the theory of continuous-time Markov-modulated processes in operation research. All analytical results are illustrated by numerical computations. Used algorithms allow overcoming computation difficulties successfully. For example, a calculation of transient probabilities of states for a continuous-time finite Markov chain uses eigenvalues and eigenvectors of the corresponding matrix (generator). In a more complex case of differential or integral equations, such a simple explicit form of a solution is missing. The explicit form of solution is presented by means of infinity sums of functions. For example, often we have to deal with the so-called renewal equation. Its solution is presented as an infinite sum of the renewal function. In this case, an approximation of functions of interest and iterative computation procedures are used.

Contents:

• Continuous-Time Markov Chains


• Markov-Modulated Chains


• Shortest Paths in Markov Modulated Networks


• Network Flows with Markov-Modulated Costs of Arcs


• Wear Process Modulated by Cycling Continuous-Time Markov Chain


• Alternating Poisson Processes and Their Estimation


• Analysis of Markov-Modulated Poisson Processes


• Monitoring of Continuous-Time Markov Chains


• Reliability System k-out-of-n in an External Random Environment


• Statistical Inferences in the Case of a Random Environment Existence


• Review of Other Applications

Readership: The book is intended for specialists in different areas, who are interested in mathematical modeling of various production processes. It is assumed that the reader is familiar with foundations of the probability theory and the matrices. The book can be recommended as a manual for masters and PhD students.

Key Features:

• The present monograph does not require special knowledge. It contains everything needed for the successful application of Markov-modulated processes to a wide range of problems in various areas


• The presentation is not limited to analytical formulas, but is accompanied by a description and illustration of appropriate computational procedures for their implementation