... until they start thinking about Darcy more seriously (ch. 44).

they soon became sensible that the authority of a servant who had known him since he was four years old, and whose own manners indicated respectability, was not to be hastily rejected.

Granted, they have motive now to find good in him (they've become aware of his fervent interest in Lizzy), but they also have personal experience with him, now, that does not contradict Mrs. Reynolds, whose opinion (as far as their experience makes possible) is also supported by the townspeople.