Abstract: The TD(λ) learning and least squares temporal difference (LSTD) learning algorithms that approximate the bias value function of an average reward Markov decision problem are proposed.Approximations are comprised of linear combinations of fixed feature functions whose weights are incrementally updated during a single endless process of the problem.