Conservatively speaking they lost at sea, beaten (2,3,3,5)
I believe the answer is:
to say the least
'conservatively speaking' is the definition.
(phrase indicating a conservative assessment)
'they lost at sea beaten' is the wordplay.
'beaten' indicates anagramming the letters.
'they'+'lost'+'at'+'sea'='theylostatsea'
'theylostatsea' is an anagram of 'TO SAY THE LEAST'.
(Other definitions for to say the least that I've seen before include "The most tongue-tied manages this" , "Wild" , "with moderate terms" , "not to exaggerate" , "Of all the monks, a Trappist's remit" .)