I agree with Jace that Anne is put off by self-important people. I think that even if her conversations with Mrs. Smith are not that stimulating, she doesn't mind because Mrs. Smith seems to value people by discovering what they are as opposed to assigning value based on who they are, as Sir W, E, and Mary do. I admit, we really don't know Mrs. Smith's feelings towards rank and wealth, but her lack of comment on the subject probably speaks volumes!

I also think Anne has had such bad experiences with her own family overvaluing rank and position (both their own and that of others) and making decisions based solely on rank and connections that, upon meeting someone new she may be less patient with people of rank (they had better prove themselves quickly) and more patient with everyday families like the Harvilles and Mrs. Smith. If Sir W were raving about someone to me, that would be enough to make me prejudiced against her until she proved otherwise. Guilty before proven innocent, I know.