It's not cricket trousers worn here (5,3,4)
I believe the answer is:
below the belt
'it's not cricket' is the definition.
(thesaurus)
I don't understand how the remainder of the clue works.
Can you help me to learn more?
(Other definitions for below the belt that I've seen before include "Where unfair blow lands" , "Illegal" , "Underhand, unfair" , "Hang scabbard here" , "fly is here" .)