To some extent, resemble mathematical symbol (6)
I believe the answer is:
emblem
'symbol' is the definition.
(I've seen this before)
'to some extent resemble mathematical' is the wordplay.
'to some extent' indicates a hidden word.
'EMBLEM' is hidden within 'resemble mathematical'.
(Other definitions for emblem that I've seen before include "Symbol; heraldic device" , "Badge, token" , "Crest or insignia" , "Crest or symbol" , "Representative object" .)