Incidentally, it's beside the bright yellow Eton caps (2,3,3)
I believe the answer is:
by the bye
'incidentally' is the definition.
(synonyms)
'beside the bright yellow eton caps' is the wordplay.
'beside' becomes 'by'.
'bright yellow eton caps' becomes 'bye' (I can't justify this - if you can you should believe this answer much more).
'by'+'the'+'bye'='BY THE BYE'
'it's' acts as a link.
Can you help me to learn more?
(Other definitions for by the bye that I've seen before include "Incidentally" , "In passing" , "Parenthetically" .)