Subject that's habitually exasperating from the start to this setter (5)
I believe the answer is:
theme
'subject' is the definition.
('theme' can be a synonym of 'subject')
'that's habitually exasperating from the start to this setter' is the wordplay.
'from the start' indicates taking the first letters.
'to this setter' becomes 'me' (the setter of the clue).
The first letters of 'that habitually exasperating' is 'the'.
'the'+'me'='THEME'
(Other definitions for theme that I've seen before include "Subject of a talk or piece of writing" , "Short melody" , "issue" , "Melody on which variations are constructed" , "Motif, topic" .)