bayesian_statistics.txt (2615B)
1 Bayes Theorem: 2 3 a rule for updating a belief when evidence arrives. Concretely: 1000 people, 4 10 have a disease. The test catches 9 of those 10, and also falsely flags 99 5 of the 990 healthy. You test positive — so you're one of the 9 + 99 = 108 6 positives, of whom 9 are actually sick. That's ~8%. The theorem is just that 7 counting written symbolically: P(A|B) = P(B|A)·P(A) / P(B). The base rate 8 (10/1000) does most of the work, which is why the answer feels wrong at first. 9 10 Bayes Statistics 11 12 the school of statistics that treats probability itself as a degree of belief, 13 and uses the theorem above as its engine. You start with a prior (what you 14 believed before data), multiply by the likelihood (how well each hypothesis 15 explains the data), get a posterior. Contrast with frequentist statistics, 16 which treats parameters as fixed unknowns and probability as long-run 17 frequency, giving you p-values and confidence intervals instead of a 18 distribution over "what the parameter probably is." Practical upshot: 19 Bayesian methods let you inject prior knowledge and get honest uncertainty, 20 at the cost of having to justify the prior and usually needing MCMC or 21 variational methods to compute anything. 22 23 Bayes Filter 24 25 Bayes' theorem applied recursively over time to estimate a hidden state from 26 noisy observations. Two alternating steps: 27 28 • Predict: push the belief forward through a motion/dynamics model (uncertainty grows) 29 30 • Update: multiply by the likelihood of the new measurement (uncertainty shrinks) 31 32 The posterior from one cycle becomes the prior for the next. It's the abstract 33 template; the famous algorithms are special cases — the Kalman filter assumes 34 everything is Gaussian and linear so the belief stays a mean + covariance, the 35 extended/unscented Kalman filter relaxes linearity, and the particle filter 36 drops the Gaussian assumption entirely and represents the belief as a cloud of 37 weighted samples. Standard tooling in robot localization, SLAM, sensor fusion, 38 and object tracking. 39 40 Goal of Bayes Filter: To estimate state x, given observation z and control 41 (action) u, which is to estimate the probability of a agent in state x, 42 given z and u 43 44 p(x|z,u) 45 46 Derivation of Bayes filter 47 48 Kalman Filter 49 50 Linear Kalman Filter 51 52 Linear Kalman Filter is really just a special case of Bayes Filter, where the 53 state transition and measurement function are linear and follow Gaussian 54 distribution. The state transition function is, 55 56 xₜ= Aₜxₜ₋₁+Bₜuₜ+ ϵₜ 57 58 Extended Kalman Filter 59 60 Reference 61 62 Probalistic Robotics, by Sebastian Thrun, Wolfram Burgard, Dieter Fox 63 64 https://bayesianstatistics.com/