Just get rid of dad but don't let on! (4,3)
I believe the answer is:
keep mum
'don't let on' is the definition.
(both can be used to tell someone to keep a secret)
I don't understand the remainder of the clue.
(Other definitions for keep mum that I've seen before include "Stay silent" , "(don't mention it)" , "Remain silent" , "Don't tell a soul" .)