Model Fine Tuning

 

What is fine tuning?

When using transfer learning we are trying to leverage knowledge from training one model on another. While our earlier layers might represent things such as corner parts of images, or curves, our final layer will be used to to identify one of the categories we were trying to identify (In Imagenet there are  actually categories and sub categories e.g. animal and tiger so our final layer would represent these).

In order to use this model for a new purpose, lets say identifying clothing. We need to throw this layer away and replace it with one that represents the categories we need e.g. shirt, coat, tshirt etc. This final layer that we add starts with random weights which then need trained. 

Fine tuning in Fast.ai v2

A model is tuned in fast.ai by calling the fine_tune method on a learner, learn.fine_tune()

Calling this method results in training this final layer of weights. While this is done all the previous weights are left unchanged (this is called freezing). 

After this the model trains all layers fror the amount of epochs passed in (this is training the model unfrozen)