"So, when the child jumps on Anne's back, he is doing what they all do. He sees how they treat her and treats her the same."

How? I'm confused. I think he acts, exactly the opposite of the Musgroves in the Little Walter incident. He ACTS. He does something about it, instead of standing around trying to reason with a two-year old or pointing fingers at other people, blaming them for the child's misbehavior. One does not reason with a toddler, one picks them up and puts them in their proper place. FW with absolutely zero child rearing experience has at least the brains to recognize that Anne needs a little help and he does what he does, instantly without really thinking about it. He is acting ourt of deference for her, not ignoring her, doing what he can to help her.

I think you are right that he observed their neglect and ill-usage of Anne at the party. But possibly before that he overheard the Musgroves lame explanation as to why Anne stayed with their sick child instead of them. I'm sure Wentworth took that as Anne trying to avoid him, but, by the dinner party, a pattern is starting emerge. The Musgroves think of Anne as part nanny, part human victrola. He has to be more than a little disgusted by this, even if he still angry with Anne.