does find Anne wretchedly altered but I think some of his impression is from his own bitterness. Even after 8 years, he is still upset over her rejection and this shows how much feeling he still has for her.

"He had not forgiven Anne Elliot. She had used him ill, deserted and disappointed him; ... It had been weakness and timidity."

"He had a heart for either of the Miss Musgroves, if they could catch it; a heart, in short, for any pleasing young woman who came in his way, excepting Anne Elliot. This was his only secret exception, when he said to his sister, in answer to her suppositions -- "

Why would he specically think of Anne at all unless he still had strong feelings for her? She was as much on his mind as he was on hers.