Vendor managed inventory (VMI) takes on a new form in our case study. It occurs between a parent company and its several subsidiaries along the Yangtze River, and the parent company is responsible for the delivery of coal which is the raw material for production. Since coal is firstly supplied from the supplier to the port by train, and is then transferred to the subsidiaries by river using water carriers, we propose to establish a central warehouse at the port which should posses a fleet of vessels, making the ordering and delivery decisions under the VMI mode. Both decisions are based on Markov Decision Process (MDP), to optimize the replenishment strategy with the consideration of holding, shortage and transportation costs. An algorithm based on Modified Policy Iteration (MPI) with an action elimination procedure is applied to the MDP model, so an approximate optimal solution can be found within reasonable time. It is shown that the proposed strategy is able to reduce the overall costs of the system in contrast to the current mode, whilst also guaranteeing higher service level to the subsidiaries.