Thanks to Stephanie for pointing this out! As well as being told in ch.44 that Mr. Gardiner has gone to meet *some* of the gentlemen at Pemberley by noon to go fishing, in ch.45 we have:

(Darcy) had been some time with Mr. Gardiner, who, with *two or three other gentlemen from the house*, was engaged by the river, and had left him only on learning that the ladies of the family intended a visit to Georgiana that morning.