Gianni, that's how I interpret it also -- that Darcy doubts Elizabeth could have become so poised, tasteful and rational by spending all her time with her immediate family at Longbourn. He may suspect that she has spent or spends a significant amount of time elsewhere with people whose positive influence could account for her being so attractive AND so different from her parents and younger sisters and the gossipy, vulgar folks in Meryton.

At the very least, he's paying her a compliment by remarking how different she is from the people among whom he's seen her.

It IS a mystery in this novel -- other posts have already noted it -- how did Jane and Elizabeth become such elegant and rational young women with so little guidance and with no role model at home? Aunt Gardiner seems the closest they have to an classy and sensible "mother-figure," and they don't see her that often...