Flushing at "Rather late in the day" (7)

I believe the answer is:
evening
'late in the day' is the definition.
(time of day)
'flushing at rather' is the wordplay.
I cannot quite understand how this works, but
'at' could be 'in' (synonymous in some cases - eg at school, in school) and 'in' is located in the answer.
This explanation may well be incorrect...
Can you help me to learn more?
(Other definitions for evening that I've seen before include "Time for vespers" , "Early night" , "Before nightfall" , "somewhat late in the day" , "Later part of day" .)
