I think he was annoyed at dancing ending conversations-in-general, because the dancing had obviously not ended any such a thing with-Elizabeth-in-particular; she was not dancing, and stated she had no intention of dancing when asked. If he wished her conversation, he certainly made no effort to gain it. And dancing at one end of the room would have allowed him a better chance at speaking with her (which the old stick-in-the-mud never does anyway!), yet he does not avail himself of the chance!

He also is not trying to hear Elizabeth chat with others, because a second later she is moving across the room, obviously disengaged, rather than being witty and vivacious barely out of Darcy's earshot.

Also, Sir William starts by trying to engage Darcy in conversation and ends up suggesting they dance, thus:

[Sir William] paused in hopes of an answer; but his companion was not disposed to make any; and Elizabeth at that instant moving towards them, he was struck with the notion of doing a very gallant thing, and called out to her --

So, I have a picture in my head: Darcy is scowling at the dancers, not actually rebuffing Sir William, but certainly not engaged with him, while Sir William stands at his side, all pleasantries and social offerings. Sir William sees Elizabeth, but not because Darcy is looking at her; because she is moving from one side of the room to the other. He gets the idea to be gallant, and tries to get the two to dance.

Now, Elizabeth is discomposed, and refutes the idea that she is trying for a partner. Darcy (who at a more formal dance refused the chance to dance with her) now offers himself as a partner. He is not opposed to dancing if it is with the Elizabeth with whom he is currently intrigued! Yet, the idea did not occur to him until Sir William intervened.