Machine Learning Unsupervised Backpropagation

Question

I'm having trouble with some of the concepts in machine learning through neural networks. One of them is backpropagation. In the weight updating equation,

delta_w = a*(t - y)*g'(h)*x

t is the "target output", which would be your class label, or something, in the case of supervised learning. But what would the "target output" be for unsupervised learning?

Can someone kindly provide an example of how you'd use BP in unsupervised learning, specifically for clustering of classification?

Thanks in advance.

Answer 1

The most frequent method is to train an autoencoder to produce outputs that are equal to the inputs. As a result, the network will strive to develop a representation that "compresses" the input distribution the best it can.
The derivatives of the error function are computed using backpropagation when training an artificial neural network with regard to the weights in the network. It's called thus because the "errors" are "propagating" backwards through the network. In this scenario, you'll need it because the ultimate error with regard to the target is determined by a function of functions (of functions ... depending on how many layers in your ANN.) The derivatives then allow you to tweak the variables to improve the error function, while the learning rate keeps things in check.
This isn't necessary in unsupervised methods. When using k-Means to minimize the mean squared error (MSE), for example, you may minimize the error directly at each step given the assignments; no gradients are required. The expectation-maximization (EM) approach is far more powerful and accurate than any gradient-descent based method in other clustering models, such as a combination of Gaussians.