Answer: Trochlear palsy.
The trochlear nerve (CN IV) supplies the superior oblique muscle, whose main actions are intorsion and depression of the eye (best tested in adduction). When this muscle fails, the patient cannot depress and intort the adducted eye, so that movement is lost.
Key facts about CN IV:
It is the smallest cranial nerve but has the longest intracranial course. It is the only cranial nerve to exit from the dorsal surface of the brainstem. It arises in the midbrain and runs forward and laterally to reach the superior oblique.
Why not the others: A third nerve palsy gives ptosis, a 'down and out' eye and a dilated pupil. A sixth nerve palsy causes loss of abduction (lateral rectus). A facial nerve palsy affects facial muscles, not eye movement. Ref: https://www.ncbi.nlm.nih.gov/books/NBK537244/