To be brief, to drop the H would be a shame (5)
I believe the answer is:
pithy
'to be brief' is the definition.
(I've seen this before)
I cannot understand how the remainder of the clue works.
Can you help me to learn more?
(Other definitions for pithy that I've seen before include "succinctly expressed" , "Terse, powerful (speech)" , "Succinct" , "Concise and full of meaning" , "Cogent" .)