First Anne approaches and speaks with him. Hope.

Then she is obliged to join her family when their elevated relations - and Mr Elliot - arrive. She might seem to him well entertained by her cousin. "Heard of Miss Anne Elliot. translating from Italian bla bla." Anne looked interested and involved, I suspect. Despair.

Then, when they are about to build up another conversation/connection, that odious man interferes to request Anne's company for Miss Carteret. Perhaps Wentworth feels like a fool, repeatedly abandoned to stand hoping for Anne's good graces?
Which was what happened at Morlands as well, where his umbrella was shunned as Mr Elliot *the priviledged relation and friend* came back for her most eagerly.
Chap 19:
She was very much obliged to him, but declined it all, repeating her conviction, that the rain would come to nothing at present, and adding, "I am only waiting for Mr. Elliot. He will be here in a moment, I am sure."

She had hardly spoken the words when Mr. Elliot walked in. Captain Wentworth recollected him perfectly. There was no difference between him and the man who had stood on the steps at Lyme, admiring Anne as she passed, except in the air and look and manner of the privileged relation and friend. He came in with eagerness, appeared to see and think only of her, apologised for his stay, was grieved to have kept her waiting, and anxious to get her away without further loss of time, and before the rain increased; and in another moment they walked off together, her arm under his, a gentle and embarrassed glance, and a "Good morning to you!" being all that she had time for, as she passed away.