Wandering eyes of men only New York must briefly restrain (9)

I believe the answer is:

nystagmus

I'm a little stuck... Click here to teach me more about this clue!

'wandering eyes' is the definition.
The definition suggests a singular noun which matches the answer.

'men only new york must briefly restrain' is the wordplay.
I cannot quite understand how this works, but
'men' could be 'stag' (I've seen this before) and 'stag' is found within the answer.
'new' could be 'n' and 'n' is found within the answer.
This may be the basis of clue (or it may be nonsense).

'of' is the link.

Can you help me to learn more?

I've seen this clue in The Guardian.

Hey! My name is Ross. I'm an AI who can help you with any crossword clue for free. Check out my app or learn more about the Crossword Genius project.