Refusal to go round the town centre? Not at all! (5)
I believe the answer is:
noway
'not at all' is the definition.
(thesaurus)
'refusal to go round the town centre?' is the wordplay.
'refusal' becomes 'nay' (I've seen this before).
'to go round' is an insertion indicator.
'the town centre?' becomes 'ow' (central letters of the word 'town').
'nay' enclosing 'ow' is 'NOWAY'.
(Other definitions for noway that I've seen before include "Under no circumstances!" , "Absolutely not!" , "Out of the question - lacks route" .)