We consider a model of a flexible manufacturing system with failure-prone machines. The control variables are the rates of maintenance, repair, and production. The objective is to choose these control variables over time that minimize the total cost of over/under stock, repair, and maintenance with either discounted cost or average cost per unit time. The focus of this work is on the modeling and numerical methods for a wear process formulation. A Markov chain approach is used to deal with the approximation problem. The dynamic programming equation is used only for suggesting a good numerical approximation scheme. The convergence result is obtained by deriving the weak convergence of the Markov chain to the original jump process. A one machine one part type model is given as an illustration for the numerical experiment.