Estimation error: Distribution and pointwise limits

In this paper, we examine the distribution and convergence properties of the estimation error W = X - X(Y), where X(Y) is the Bayesian estimator of a random variable X from a noisy observation Y = X + σZ where σ is the parameter indicating the strength of noise Z. Using the conditional expectation framework (that is, X(Y) is the conditional mean), we define the normalized error Eσ = W/σ and explore its properties.Specifically, in the first part of the paper, we characterize the probability density function of W and Eσ. Along the way, we also find conditions for the existence of the inverse functions for the conditional expectations. In the second part, we study pointwise (i.e., almost sure) convergence of Eσ as σ → 0 under various assumptions about the noise and the underlying distributions. Our results extend some of the previous limits of Eσ as σ → 0 studied under the L2 convergence, known as the MMSE dimension, to the pointwise case.