I agree. I think Elizabeth means Mrs. Clay is safe from Sir Walter’s attentions due to her “personal misfortunes” (5) of tooth and freckle. However I have to side with Anne:

"There is hardly any personal defect," replied Anne, "which an agreeable manner might not gradually reconcile one to." (5)

In my opinion it is a common phenomenon for someone you like to appear more and more attractive the more you like them—this happens to the Miss Bertrams in Mansfield Park:

Her brother was not handsome: no, when they first saw him he was absolutely plain, black and plain; but still he was the gentleman, with a pleasing address. The second meeting proved him not so very plain: he was plain, to be sure, but then he had so much countenance, and his teeth were so good, and he was so well made, that one soon forgot he was plain; and after a third interview, after dining in company with him at the Parsonage, he was no longer allowed to be called so by anybody. He was, in fact, the most agreeable young man the sisters had ever known, and they were equally delighted with him. (MP, 5)

Elizabeth’s response to Anne:

"I think very differently," answered Elizabeth, shortly; "an agreeable manner may set off handsome features, but can never alter plain ones. " (5)