Probability Vector Estimation under Constraints by Discounting

Kurzfassung/Abstract

The focus of this paper is on using observations to estimate an unknown probability vector p = (p1,...,pN) supposed to underlie a multinomial process. In some technical applications, e.g., parameter estimation for a hidden Markov chain, numerical stability can be guaranteed only if we assume each estimate for a probability conforming to the constraint of being always above a positive threshold depending on the particular technical application. Aiming at such estimates we present a fast discounting algorithm which comprises ad-hoc methods known as absolute discounting, linear discounting, and square-root discounting as special cases. In order to base discounting on probabilistic principles, we adopt a Bayesian approach, and we show that, presupposing an arbitrary nonvanishing prior, minimizing the maximum-norm of a certain risk vector defined by a one-sided loss function leads to a new consistent estimator.
It turns out to be quite natural to derive from this an (in general inconsistent) estimator meeting the above described constraints. Using asymptotic statistics, we show that a good approximation to this estimator can be reached by means of our fast discounting algorithm in context with an appropriate adjustment of square-root discounting.