Yes, rector will finally upset part of the church (10)
I believe the answer is:
clerestory
'part of the church' is the definition.
'clerestory' can be an answer for 'part' (I have seen 'Part of church' mean 'clerestory' so perhaps 'part' could also mean 'clerestory'). I'm not certain of the 'of the church' bit.
'yes rector will finally upset' is the wordplay.
'finally' suggests the final letters.
'upset' is an anagram indicator.
The last letter of 'will' is 'l'.
'yes'+'rector'+'l'='yesrectorl'
'yesrectorl' with letters rearranged gives 'CLERESTORY'.
Can you help me to learn more?
(Other definitions for clerestory that I've seen before include "topmost church windows" , "Upper story of large church, admitting light" , "Upper row of windows" , "Part of church" .)