Minister's house fellow swore is empty (5)
I believe the answer is:
manse
'minister's house' is the definition.
(I've seen this before)
'fellow swore is empty' is the wordplay.
'fellow' becomes 'man' (I've seen this before).
'is empty' means to remove the middle letters.
'swore' with its middle taken out is 'se'.
'man'+'se'='MANSE'
(Other definitions for manse that I've seen before include "Scottish vicarage" , "Pile of rector'" , "Church minister's house" , "church property?" , "Religious home" .)