I don't think Darcy was annoyed that dancing ended conversations-in-general - I don't think it consistent with his character, and when Sir William talks to him, Darcy is not over-eager to join in. I still think he wants to talk to Lizzy, not people in general.

]If he wished her conversation, he certainly made no effort to gain it.

True, but Darcy is still in his "listening in on her conversation"-phase. Unfortunately for him, Lizzy discovered that and decided that "if I do not begin by being impertinent myself, I shall soon grow afraid of him." (Ch. 6). She talks to him (about a ball at Meryton which she knows is not Darcy's cup of tea), but Darcy is only allowed one answer, before Charlotte interrupts. Then "gravely glancing at Mr. Darcy" she starts playing. Lizzy's manners are not really inviting here, and an actual conversation between them is aborted because Charlotte wanted Lizzy to play. That is why I think Darcy is not silently indignant because the music and dancing ended conversation in general, but because it ended his opportunity to converse with Lizzy.

]He also is not trying to hear Elizabeth chat with others, because a second later she is moving across the room, obviously disengaged, rather than being witty and vivacious barely out of Darcy's earshot.