Annoy fellow inside with firearm (5)
I believe the answer is:
rifle
'firearm' is the definition.
(rifle is a kind of firearm)
'annoy fellow inside' is the wordplay.
'annoy' becomes 'rile' ('rile' can be a synonym of 'annoy'**).
'fellow' becomes 'f' (abbreviation - of a society etc.).
'inside' is an insertion indicator.
'rile' enclosing 'f' is 'RIFLE'.
'with' acts as a link.
(Other definitions for rifle that I've seen before include "One that has stock" , "Sack; firearm" , "disarray" , "Steal and carry off - firearm" , "Root" .)