A Simple Computer Game Using a Markov Chain Starts at the Cave

question 22

Essay

A simple computer game using a Markov chain starts at the Cave or the Castle with equal probability and ends with Death (you lose) or Treasure (you win).The outcome depends entirely on luck.The transition probabilities are: $\begin{array}{c}\text { Next State }\\\begin{array}{lllll}&&\text { Cave }&\text {Castle }&\text {Death }&\text {Treasure }\\&\text { Cave } & .4 & .3 & .2 & .1 \\\text { Current }&\text { Castle } & .5 & .3 & .1 & .1 \\\text { State }&\text { Death } & 0 & 0 & 1 & 0 \\&\text { Treasure } & 0 & 0 & 0 & 1\end{array}\end{array}$


A.What is the average number of times you would expect to visit the Cave and the Castle, depending on which state is the starting state?
B.What is the mean time until absorption for the Cave and the Castle?
C.What is the likelihood of winning the game?

---

Definitions:

Developing Follicle

A developing follicle is a fluid-filled sac in the ovary that contains a maturing egg. It releases the egg during ovulation in the menstrual cycle.

Progesterone

A steroid hormone involved in the menstrual cycle, pregnancy, and embryogenesis of humans and other species.

Estrogens

Female sex hormones produced by the ovary; promote the development and maintenance of female reproductive structures and of secondary sex characteristics.

Prolactin

A hormone produced by the anterior pituitary gland that stimulates milk production after childbirth and affects many other functions in the body, including metabolism, immune system regulation, and water and electrolyte balance.