The more there are, the fewer there are (9)
I believe the answer is:
absentees
I'm afraid I can't even guess how this answer relates to the clue.
Can you help me to learn more?
(Other definitions for absentees that I've seen before include "They're not present at the required time" , "One gets fonder of those" , "who may have apologised to chairman" .)