I agree with you that the line is Mr. Darcy's. From the standpoint of the rules of writing, the author would have had to say that Miss Bingley said the line if it were she who said it. Elizabeth was talking with Mr. Darcy at this point, so Miss Bingley's saying anything would be a change, and that would have to be pointed out.

Also I wonder if Miss Bingley would have said "your own sex" if it was her own sex too.

And after Elizabeth answers the question, Mrs. Hurst and Miss Bingley "both cried out against the injustice of her implied doubt," and that makes it seem to me that Miss Bingley was much more interested in pleasing Darcy by what she said than by really trying to find out what Elizabeth thinks. She probably just watched Darcy and Elizabeth talking at that point.