Menus > Layers menu > 
Layers > Delete > Unused Layers
 
 
 

Deletes all layers with no objects on them.