Question

Starting with some data (which may come from an experiment or a simulation), we often use statsitics to answer a few typcical questions:

• How well does the data match some assumed (null) distribution [hypotehsis testing]?
• If it doesn’t match well but we think it is likely to belong to a known family of distributions, can we estiamte the parameters [point estimate]?
• How accurate are the parameter estimates [interval estimates]?
• Can we estimate the entire distribution [function estimation or approximation]?

Most commonly, the computational approaches used to address these questions will involve

• minimization off residuals (e.g. least squeares)
  • Numerical optimization
• maximum likelihood
  • Numerical optimization
  • Expectation maximization (EM)
• Monte Carlo methods
  • Simulation of null distribution (bootstrap, permutation)
  • Estimation of posterior density (Monte Carlo integration, MCMC, EM)

Rarely (i.e. textbook examples), we can find a closed form solution to these problems.