You seem to be arguing that the Musgroves are incapable of friendship. To me, they don't appreciate Anne's reading or Anne's music playing simply because they are intellectually incapable of recognising the quality of what she does. That doesn't make them incapable of friendship or affection. Even much more intellectually disabled people than the Musgroves are capable of friendship in my opinion.

Oh, and to me civility does suggest affection. At the simplest level it is the affection and recognition of duty towards other people, not just wanting someone because they are useful to them (as you first argued how Anne is wanted by the Musgrovs above).

And there are signs of affection there, that they appreciate her value:
From Chapter 6:
Oh! Miss Anne, I cannot help wishing Mrs. Charles had a little of your method with those children. They are quite different creatures with you!
"I have no scruple of observing to you, how nonsensical some persons are about their place, because all the world knows how easy and indifferent you are about it ... (This, by the way, is when the Miss Musgroves and Anne are walking together without Mary).

Or in Chapter 7, when, rather than Charles Musgrove just regarding her as a servant whose duty it is to nurse his child, on hearing Mary's proposal that Anne stay with little Charles, he tries to argue her out of it, and then proposes further to fetch her after dinner.

From Chapter 9:
the Miss Musgroves' much more cordial invitation to herself to go likewise

In Chapter 10 Louisa says of Anne refusing Charles' proposal: I wish she had accepted him. We should all have liked her a great deal better;

I think that the evidence is pretty good that they do appreciate her, within their limitations. Based on your arguments, I think you should be feeling sorry for Mary rather than for Anne - I can't find any praise of Mary by the Musgroves.