Having to turn ourselves in and get rid of (4)
I believe the answer is:
oust
'get rid of' is the definition.
(I've seen this before)
'having to turn ourselves in' is the wordplay.
'having' becomes 'to' (I am not sure about this - if you are sure you should give a lot more credence to this answer).
'to turn' shows that the letters should be reversed in order.
'ourselves' becomes 'us'.
'in' indicates putting letters inside.
'to' backwards is 'ot'.
'ot' enclosing 'us' is 'OUST'.
'and' acts as a link.
(Other definitions for oust that I've seen before include "Put out, say of office" , "Turn out" , "kick out" , "Send packing" , "Remove and replace" .)