Computers and Chemical Engineering 26 (2002) 1399– 1413 www.elsevier.com/locate/compchemeng Markovian inventory policy with application to the paper industry
K. Karen Yin a,*, Hu Liu a,1, Neil E. Johnson b,2 a Department of Wood and Paper Science, Uni6ersity of Minnesota, St. Paul, MN 55108, USA b IT Minnesota Pulp and Paper Di6ision, Potlatch Corporation, Cloquet, MN 55720, USA
Received 20 August 2001; received in revised form 18 April 2002; accepted 18 April 2002

Abstract
This paper concerns problem formulation and solution procedure for inventory planning with Markov decision process models.
Using data collected from a large paper manufacturer, we develop inventory policies for the finished products. To incorporate both variability and regularity of the system into mathematical formulation, we analyze probabilistic distribution of the demand, explore its connection with the corresponding Markov chains, and integrate these into our decision making. In particular, we formulate the Markov decision model by identifying the chain’s state space and the transition probabilities, specify the cost structure and evaluate its individual component; and then use the policy-improvement algorithm to obtain the optimal policy.
Application examples are provided for illustration.
© 2002 Elsevier Science Ltd. All rights reserved.
Keywords: Inventory; Production planning; Markov chain; Markov decision process; Optimal policy; Paper manufacturing

1. Introduction
Inventory management is one of the crucial links of any supply chain. To manufacturers, it entails managing product stocks, in-process inventories of intermediate products as well as inventories of raw material, equipment and tools, spare parts, supplies used in production, and general maintenance supplies. In a broader sense, it comprises all kinds required to run a business including storage, personnel, cash and transportation facilities, etc. This paper focuses on inventory of finished products. A manufacturing company needs an inventory policy for each of its products to govern when and how much it should be replenished. Good inventory management offers the potential not only to cut costs but also to generate new revenues and higher profits. On the contrary, undersupply causes stockout and leads to lost sales; whereas oversupply hinders free cash flow and may cause forced markdowns. As a result

* Corresponding author. Tel.: +1-612-624-1761; fax: + 1-612-6256286
E-mail addresses: kyin@umn.edu (K.K. Yin), liux0295@tc.umn. edu (H. Liu), neil.johnson@potlatchcorp.com (N.E. Johnson).
1
Fax: + 1-612-625-6286.
2
Fax: + 1-218-879-1079.

of improper inventory policies, both will diminish earnings and can have enough impact to make a company non-profitable. Due to the everchanging market conditions, the dynamic and random nature of the demands, the close and complicated relationship between resource/production planning and product inventory management, as well as the process uncertainties, matching supply with demand has always been a great challenge. Being able to offer the right product at the right time for the right price remains frustratingly elusive (Fisher, Raman, & McClelland, 2000) to manufacturers and retailers.
Process scheduling and planning have attracted growing attention in many industries. Numerous papers in the area of design, operation and optimization of batch as well as continuous plants have been published
(see, Applequist, Samikoglu, Pekny, & Reklaitis, 1997;
Bassett, Pekny, & Reklaitis, 1997; Pekny & Miller,
1990; Petkov & Mararas, 1997 and the references therein). The main objective of inventory management is to increase profitability. A frequently used criterion for choosing the optimal policy is to minimize the total costs, which is equivalent to maximizing the net income in many cases. Scientific inventory management requires a sound mathematical model to describe the behavior of the underlying system and, quite often, an

0098-1354/02/$ - see front matter ©