I wonder if this is a case in which we as "modern" women might have a problem understanding the society we are reading about. I believe and I think other might be able to back me up on this, that men really didn't have all that much to do with the children. There were nannies and the mother (to a lesser extent) for the rearing, Dad did the supporting and made cameo appearances to play with them or show them off to visitors. As much as I hate that Charles goes shooting and to the dinner, I think that was "his sphere" and caring for the child was Mary's. Charles would become more active when the boys are older and need to learn to shoot and run the estate.