In response to this very question, I like CF's description of the frame of mind he had to adopt in order to play the part properly (The Making of Pride and Prejudice, p. 101).

He knows no one there except the friends he came with. After Bingley reproaches him about not dancing, he does look at Elizabeth (in the novel), but withdraws his eye as soon as he relizes he has caught hers. What he is saying to Bingley is, "You've got the prettiest girl in the room and I'm supposed to be better than you". IMHO he was just telling his friend that, on first impression, Lizzy wasn't as pretty as Jane. It was not meant for Lizzy's ears. Perhaps he should have lowered his voice, but he probably was a little exasperated, or at least agitated and uncomfortable. If Darcy had been the perfect gentlemam, a gentleman without fault, then we wouldn't have our story, would we? Even men of consequence are human.