Layman Description of the Kalman Filter

Question

I want to know about Kalman Filter but i tried searching different links including Electrical Engineering StackExchange but the information available there was hardly digestible.

All I am able to understand is that Kalman Filter is an estimator / predictor of future state of any system ,based on a model that is formed by previous measurements / readings.

So please kindly explain in simple words, what is Kalman Filter? What are its applications in Signal / Image processing and how is it different from other filters like Low pass, high pass etc?

Answer 1

Simple Description

Imagine you're in a car that is traveling at 70MPH with cruise control. Because the cruise control isn't perfect, your actual speed might vary slightly. This imperfection is called "process noise".

Now lets also imagine the car is being tracked using GPS. Because GPS isn't perfect, there will be some noise in the sensor reading. This imperfection is called "measurement noise".

If we designed a Kalman filter to estimate the actual vehicle position, it would do the following:

1. Use the process information to predict the next vehicle position.
2. Take a vehicle position measurement reading.
3. Calculate the difference between the measurement and the predicted position. This is called the residual.
4. Takes this residual and weighs it against the process noise and measurement noise to optimally update the position estimate.
5. Repeat.

So essentially the Kalman filter looks at the discrepancy between our model prediction and measurement and then generates an estimate that leans towards one or the other depending on the process noise and measurement noise.

If we have high process noise (i.e. our cruise control is really bad) and low measurement noise, the Kalman filter will favor the measurement reading when updating the estimate.

If we have low process noise and high measurement noise (very inaccurate GPS reading), the Kalman filter will favor the predicted position when updating the estimate.

The overall idea is that combining information from our process model and measurement will yield a more accurate estimate than using simply one or the other.

Comments on math

The Kalman filter is essentially just an application of recursive least squares where the estimation error covariance is minimized. I would suggest reading up on "least squares estimation" and "recursive least squares estimation".

Also, Kalman "filter" is somewhat of a misnomer. It should be viewed more as an "estimator" that uses statistics to estimate a value. I've always reserved the term "filter" for something that explicitly attempts to influence the frequency content of a signal. While a Kalman filter does influence the frequency content of a signal, I don't believe it does so in an explicit fashion like a high-pass or low-pass filter.

Answer 2

KF is actually a mixture of a deterministic state propagator and a statistical estimator.

Despite it's name including the term filter, Kalman filter is not a simple frequency selective one. It's indeed a statistical recursive estimator of a state of a (linear) dynamic system. Yet on a broader sense it's called as a filter as it will separate a desired signal (the state) from corrupted measurements; hence filtering.

A linear dynamical system (LDS) is a physical construct whose behaviour is described by (linear) differential equations;

$$ \dot{X} = A X + Bu + W $$

where the vector $X$ is the state of the system, the matrix $A$ is the state propagation information (derived based on the differential equation of the LDS) , $u$ is a vector of deterministic inputs and $W$ is the state noise; depicting the uncertainity in the LDS propagation model $A$.

KF takes advantage of this deterministic behaviour of a LDS before estimating the current state $X$ purely from noisy measurements.

$$ Y = C X + Du + V $$

where $Y$ is the observation about the system and $V$ is the measurement noise on it.

Given a previous estimate of the state, it first deterministically propagates this previous estimate into current estimate through the matrix $A$ (process equation), and then performs a new finer estimate on the difference of the current measurement and propagated estimate using the measurment equation.

Answer 3

Perhaps an analogy might be constructive.

Consider a submarine commander with a fat tanker in the cross hairs of his periscope.

He needs to shoot his torpedoes, not at where the target is now, but at some place where the torpedo will intersect with the target.

A skilled commander will have knowledge about how fast or slow the tanker can go. knowledge about how it turns, ect. This is the model.

Looking through the periscope, the commander tries to determine the position, speed, and heading of the target. These are the states of the model. A measurement of the state of the target is less than ideal through a periscope. The commander must make an educated guess of the state.

The commander “predicts” where the target will be and shoots his torpedo. After the expected run time, he raises the periscope and assess the scene. The torpedo ran ahead of the target and missed. The commander decides he had the speed too fast, and the heading was further away. He mentally updates his estimate of the state of the target.

The process repeats with a new attack.

A Kalman Filter doesn’t shoot torpedoes but does use the difference between what the state of the model predicts, and uses the error between the prediction and the measurement to update the estimate of the state.

Ideally there is a “true” model and it’s state is estimated in these recursive “predict-measure-update” cycles.

Typically, models are first order and can produce useful predictions over small steps.

Answer 4

See: What is the relationship between a Kalman filter and polynomial regression?

In over-simplified form, eyeball a line though a cloud of data samples, look where that line might point one sample into the future; and, when you get a new sample check, how good that estimate might have been; then redo, but optimize for a lot less arithmetic per step.

A Kalman filter is usually just an optimization of one, or a set of simple fixed or weighted estimators, such as linear or low order regression fits, plus optional extrapolation and statistical distribution estimation.

So, basically, a statistical summarization of a bunch of previous data into a vastly smaller amount of state variables.

When getting a new ime sample, instead of re-running an entire regression on an entire previous data set plus the one new sample (potentially zillions of inputs and zillions of arithmetic operations), a Kalman filter simply uses the state information left by a previous time step computation plus the one new sample. Using one data input, plus a few state inputs, results in vastly fewer math operations to provide an updated fit and/or statistical estimation, compared to re-running entire regressions on long data sets. Thus also requiring a lot less computation, data storage (computer memory), and bandwidth.