Leaving aside the later part of the quote for the moment - which is more complex and thoughtful - in regard to this

It is poverty only which makes celibacy contemptible to a generous public! [CH 10]

I tend to think that the part about the 'generous' public is a reproach to the public, in regard to the general treatment received by poor single women. I doubt that JA really thought it generous of the public to have contempt for poor single women, simply because they are poor and single. I'd guess that JA sees Emma's view here as thoughtless and callow.

And I do not think, and certainly hope that JA did not view herself

a single woman, with a very narrow income, [as] a ridiculous, disagreeable, old maid! the proper sport of boys and girls [CH 10]