Discrete-Time Markov Chains

BOOK CHAPTER published 2013 in Markov Processes for Stochastic Modeling

Authors: Oliver C. Ibe

Discrete Dynamic Programming with Sensitive Discount Optimality Criteria

JOURNAL ARTICLE published October 1969 in The Annals of Mathematical Statistics

Authors: Arthur F. Veinott

Discrete-time Markov chains

BOOK CHAPTER published 1997 in Markov Processes for Stochastic Modeling

Authors: Masaaki Kijima

Discrete-Time Markovian Decision Processes with Incomplete State Observation

JOURNAL ARTICLE published February 1970 in The Annals of Mathematical Statistics

Authors: Yoshikazu Sawaragi | Tsuneo Yoshikawa

Asymptotic Optimality and Rates of Convergence of Quantized Stationary Policies in Continuous-Time Markov Decision Processes

JOURNAL ARTICLE published 10 September 2022 in Discrete Dynamics in Nature and Society

Research funded by National Natural Science Foundation of China (11961005,2021021)

Authors: Xiao Wu | Yanqiu Tang

Editors: Luca Pancioni

Sensitive Discount Optimality

OTHER published 15 April 1994 in Markov Decision Processes

Asymptotic Optimality for $C_p, C_L$, Cross-Validation and Generalized Cross-Validation: Discrete Index Set

JOURNAL ARTICLE published 1 September 1987 in The Annals of Statistics

Authors: Ker-Chau Li

Discrete-Time Markov Chains

BOOK CHAPTER published 18 October 2001 in Stochastic Processes and Their Applications

Markov Renewal Programs with Small Interest Rates

JOURNAL ARTICLE published April 1971 in The Annals of Mathematical Statistics

Authors: Eric V. Denardo

Consistency of Akaike's Information Criterion for Infinite Variance Autoregressive Processes

JOURNAL ARTICLE published 1 June 1989 in The Annals of Statistics

Authors: Keith Knight

A new strong optimality criterion for nonstationary Markov decision processes

JOURNAL ARTICLE published 23 November 2000 in Mathematical Methods of Operations Research (ZOR)

Authors: Xianping Guo | Peng Shi | Weiping Zhu

Markov Processes with Discrete States in Continuous Time

BOOK CHAPTER published 4 September 2017 in The Theory of Stochastic Processes

Random Truncation Models and Markov Processes

JOURNAL ARTICLE published 1 June 1990 in The Annals of Statistics

Authors: Niels Keiding | Richard D. Gill

On Stationary Markov Processes

JOURNAL ARTICLE published April 1967 in The Annals of Mathematical Statistics

Authors: Richard Isaac

Local stationarity and time-inhomogeneous Markov chains

JOURNAL ARTICLE published 1 August 2019 in The Annals of Statistics

Authors: Lionel Truquet

Adaptive Policies for Markov Renewal Programs

JOURNAL ARTICLE published 1 March 1973 in The Annals of Statistics

Authors: Bennett L. Fox | John E. Rolph

Discrete-time zero-sum games for Markov chains with risk-sensitive average cost criterion

JOURNAL ARTICLE published April 2023 in Stochastic Processes and their Applications

Research funded by Science and Engineering Research Board (MTR/2021/000307)

Authors: Mrinal K. Ghosh | Subrata Golui | Chandan Pal | Somnath Pradhan

CONTROL OF MARKOV PROCESSES WITH DISCRETE TIME AND SEMI-MARKOV PROCESSES

BOOK CHAPTER published December 2005 in Mathematical Theory of Adaptive Control

Chapter 5. Mixing for markov processes

BOOK CHAPTER published 31 December 1993 in Large Deviations for Discrete-Time Processes with Averaging

Mixtures of Markov Processes

JOURNAL ARTICLE published March 1962 in The Annals of Mathematical Statistics

Authors: David A. Freedman