No matter how tiresome Elizabeth and her father may be they are his relatives and therefore following the invitation would have been the proper thing to do. Not to mention that there are two other cousins he's (apparently) never met, that he could not be bothered to become acquainted the people whose fate one day might lie in his hands does not bode well. I also find it hard to understand why he never bothered to introduce himself or shown some interest in the estate he was going to inherit. This could have been done without being indelicate. And it would have shown some sense of responsibility - I doubt his future tenants will find him an easy master.

Speaking ill of his relatives (incl. Anne he did not even know) is another sign of ill-breeding and beneath a gentleman.