I'm speculating here, but Darcy begins by dismissing Bingley's humility as boastfulness, proceeds to criticize Bingley's precipitance, and the ensuing discussion, while entirely hypothetical, revolves around Bingley's supposed impetuousness. Perhaps Bingley is simply uncomfortable with people talking about him in this fashion.

If you and Miss Bennet will defer yours till I am out of the room I shall be very thankful; and then you may say whatever you like of me.

But what I really love about the entire exchange is the foreshadowing! You can only fully appreciate the exchange on a second read--one of the reasons JA can be reread over and over.