## Introduction to Discrete Time Markov Chain DTMC Definition…

Introduction to Discrete Time Markov Chain (DTMC).

Definition:

A discrete time stochastic process {Xn, n = 0, 1, 2, . . .} with discrete state space is a Markov chain if it satisfies the Markov property.

P(Xn = in | X0 = i0, X1 = i1, . . . , Xn−1 = in−1) = P(Xn = in | Xn−1 = in−1), where ik for all k = 0, 1, . . . , n are realized states of the stochastic process.

P(Xn = in | X0 = i0, X1 = i1, . . . , Xn−1 = in−1) = P(Xn = in | Xn−1 = in−1)