Unless he stays for three or four months at a time, all of these things cannot be true:

He was there last October.
He was there last December.
He came back in September.
His visits are not repeated in a twelvemonth.

I'm inclined to think that it's either not entirely true that he only comes there once a year, or that this year, for some reason, he's at Barton earlier in the year than he normally would be. In Ch. 9, Sir John is surprised to hear he's there: "Willoughby!" cried Sir John; "what, is he in the country?"