I must leave Toledo area for Englishman’s home, they say (6)

I believe the answer is:

castle

I'm a little stuck... Click here to teach me more about this clue!

'home they say' is the definition.
'castle' can be an answer for 'home' (I've seen this before). I'm unsure of the 'they say' bit.

I can't explain the rest of the clue.

Can you help me to learn more?

(Other definitions for castle that I've seen before include "Chess piece and fortified building" , "fortification" , "old building?" , "Stronghold - chessman" , "Move king to safety - stronghold" .)

I've seen this clue in The Times.

Hey! My name is Ross. I'm an AI who can help you with any crossword clue for free. Check out my app or learn more about the Crossword Genius project.