A small number turned up at church in times past (4)
I believe the answer is:
once
'in times past' is the definition.
(I've seen this before)
'a small number turned up at church' is the wordplay.
'a small number' becomes 'no' (abbreviation; 'small' asks for a short form).
'turned up' says the letters should be written in reverse.
'at' means one lot of letters go next to another.
'church' becomes 'CE' (abbreviation for Church of England).
'no' back-to-front is 'on'.
'on'+'ce'='ONCE'
(Other definitions for once that I've seen before include "At some former time" , "In former times" , "A single time" , "At a previous time" , "In times past" .)