Look at beginning of chapter 23, the conversation between Lady Bertram, Edmund and Fanny. Lady B keeps saying

"Why did they invite Fanny? I cannot spare her, and I am sure she does not want to go. Fanny, you do not want to go, do you?”

“If you put such a question to her,” cried Edmund, preventing his cousin’s speaking, “Fanny will immediately say No; but I am sure, my dear mother, she would like to go; and I can see no reason why she should not.”

And indeed, Fanny's response is:

“If you cannot do without me, ma’am—” said Fanny, in a self–denying tone.

It is Edmund who suggests that Lady Bertram asks Sir Thomas about it. Lady B says that she will asks her husband "whether she can do without Fanny". Edmund explains that he means his father's "opinion as to the propriety of the invitation’s being accepted or not" and he thinks Sir Thomas "will consider it a right thing by Mrs. Grant, as well as by Fanny, that being the first invitation it should be accepted.”

Sir Thomas sees no problem in Fanny going to dine at the parsonage; he considers Mrs. Grant’s shewing civility to Miss Price, to Lady Bertram’s niece,and he's only surprised that this is the first time that this happens. He also thinks that Fanny was right on asking permission to go. But Lady Bertram continues with her complaints: "Can I do without her? She always makes me tea, etc." until Sir Thomas says that he will ask Mrs. Norris to spend the day with them, and that he will stay with her, too. That's when Lady Bertram accepts that Fanny go.