When & why use an oblique rotation?

I want to make sure I'm getting something: A rotation is chosen based on how interpretable it makes the factors. However, it seems like the complications that come with oblique rotations are only worth it if they substantially increase the interpretability. Is this right? Based on this, I would say that the problem in the problem set only warrants an orthogonal rotation as the gains from an oblique rotation seem marginal.

This is basically correct...oblique rotations increase interpretability and can be used in cases in which you believe factors should in fact be correlated. Ultimately one must decide whether interpretability has increased enough to warrant the additional model complexity -- i.e., greater number of parameters and fewer degrees of freedom -- that comes with relaxing the assumption that the phi matrix is identity.