I am not expert; I am only quoting one. Dr. David M. Shapard says that spunging houses were for the genteel classes only: no servants, no outcasts from society.

If he is wrong, then I apologize for trusting the wrong source. If he is right, then your supposition that a servant would be found there, or a fallen woman, can not be true. (At least, not without more explanation as to how Mrs. Brandon circumvented the system.)