] He neither cared for his family nor his inheritance - and in both cases it seems because he could not be bothered to accept the responsibility coming along

I keep coming back to the fact that Mr Elliot was treating them no worse than they treated him. They had had nothing to do with him during all his growing up years - "He had not been known to them as a boy; Is this any way for the "head of the family" to treat his relations? Especially the boy he knew would succeed him if he didn't have a son?

"...but soon after Lady Elliot's death, Sir Walter had sought the acquaintance; and though his overtures had not been met with any warmth, he had persevered in seeking it...Mr. Elliot had been forced into the introduction. But, now that Sir Walter knows he won't be having any sons, now that Mr Elliot can be of use to him, especially if he marries Elizabeth, Sir Walter seeks him out - against Mr Elliot's inclination or will - because Mr Elliot has suddenly become important to him.