After all, Admiral Crawford is Mary's guardian, with the obligation to provide accommodation for her, not Henry. Henry does seem to care enough about Mary's prospects to know that a settled, retired, respectable situation is what she wants and requires, and as their sister is already in such a situation, and very willing to offer her such, why deprive Mrs Grant of that happiness? Mrs Grant seems to be as much "in want of some variety at home" as either of her siblings.

Norfolk does not seem closer to town than Northampton (and I suspect it would be wetter, and with an arctic sea breeze), nor are we lead to suspect it has a greater variety or better quality of neighbour - Henry himself seems to have an aversion to living on his estate for that reason. Mary's hesitation seems to be stemming more from what is already being implied of Dr. Grant's character. That does not auger much beyond a good meal with certainty - and if that came with the obligation to attend long services at church every morning and evening, and be on her best behavior to avoid quarreling with a dull and preaching man, or to bear daily witness to his quarreling with her sister, I can see how she would prefer to live with a brother she knows would never stoop to such boorish and selfish behavior.
Henry cares enough to take her back with him at the end of a few days if those fears proved true. The sincerity of his promise to fetch her away again has to be suspected - it is unlikely that he could "fetch her away again at half an hour's notice, whenever she were weary of the place" if he were not there with her when she tired. On the whole, they both seem quite pleasant and frank. The harshest things said against them are what Mary says in jest - he is "the most horrible flirt", "quite spoiled", "detestable" , a heart breaker. She tries to "trick him into marrying" and believes that every body should marry "as soon as they can do it to advantage."

I wonder if this is a lesson learnt from the mother, whose first marriage left her only daughter from it with "no more than five thousand pounds", while her second delivers twenty thousand for the daughter and a small estate for the son.
Maybe Everingham is just the shell of an estate, with all its happy fall of land sold, its timbers felled and its park converted to cotton mills, sheep and wheat, income production. Perhaps the house is occupied by the tenants,or a steward who knows how to humour a callow young master and is accustomed to regard the place as his own and run it as such. It seems that the mother has died or vanished first, then the father died before Henry turned 21, given that they have lived in their fathers family and that Henry was planning his improvements at Westminster, ie. before he was eighteen years old.
It would indicate a certain coldness, if we positively knew he fell to planning improvements as soon as he learnt of his fathers death, but we do not - there is nothing positively wrong about him - he is "black and plain, but still the gentleman", "the most agreeable man the sisters had ever known" although Fanny continues to think him as plain as ever.
It is clear that a dislike of confinement and country life, a passion for society are not good things in this novel - nor ambitious marriage plans (what a contrast to Pride and Prejudice, where Darcy and Bingley can travel at will and still be the most obliging brothers in the world, and where the social skills and beauty of the eldest Bennet sisters bodes only good things, and the danger is in throwing oneself away). The "brilliant acquirements,... manner naturally easy...and general civility and obligingness" of the Bertram sisters have not convinced us they are free of fault, and Henry's possession of the same virtues, and his ambiguous "all is safe with a lady engaged" create a sense of uneasiness, but we don't have any sound reasons for suspecting either of the Crawfords are as positively selfish, as derelict of duty, as devoid of compassion, as mercenary, as all the Bertrams bar Edmund have already shown themselves to be, in their treatment of Fanny.