Even on my first reading, when this aspect of Anne's behaviour struck me most strongly, I didn't focus on it to the exclusion of all Anne's other qualities! - such as her reasons for declining Charles's offer.

Though there aren't any disinterested casual observers in the story to share my concern - because so few people knew of the broken engagement - I still feel a tinge of disappointment that it's not addressed in Anne's reflections at the end of the story. Only a tinge, mind! It's her prerogative to view things in terms of her own private feelings, and to trust that Captain Wentworth's stated feelings are deep and abiding, without reference to any other outward appearances. :-)