How can one improve a Pre-trained neural networks i.e AleNet to fit your domain/ to fit another domain/ transfer learning
The concept of transfer learning came into being when some researchers found that a deep neural network, after training on a particular recognition task (eg. Object Recognition ), can be applied on another domain (eg. Bird Subcategorization) giving state-of-the-art results. This idea has powerful implications, as a model can be pre-trained and then applied on the required problem.
You can use a pre-trained model in two ways:
- As a feature extractor; Train model on a bigger data (possibly on a similar domain), use on on the required problem as on “off-the-shelf” model.
- As a fined-tuned-network; Train model on a bigger data, then freeze the upper-layers (set the learning rate of the earlier layers to minimum) and retrain on the required problem.
PS: To read in depth about transfer learning, here’s a resource