As noted below, (and by JA herself, in Ch. 7) Colonel Brandon seems oddly positioned as a particular friend of Sir John. However, since he is such a friend, I can well imagine that he is accustomed to their habitual talk-without-thinking ways. He may even find them entertaining, when he is in spirits himself. Opening one's mouth and letting any thought in the head fall right out is not that uncommon of a trait - I am sure that the Colonel had heard more than his fair share of impertinent speculation in the past without taking offense. He was stressed and upset when he received the letter, otherwise I can imagine he may have shaken his head in private amusement at Team Chatterbox reaching for a mountain at first hint of a molehill.