As you say there were alternatives such as the textile industry in the North, also it has occured to me that maybe the demand for servants outstripped the supply.

In the earlier chapters of TGD it says (I think) more Genteel families were leaving the towns to rent or build larger houses in the country. If so, servants could move on to another household if their current place didn't suit them, because the demand for their services had increased.

The fact that the servants didn't seem bothered about obtaining a reference may mean the next mistress was so grateful to get some help, she didn't demand a reference. Elizabeth Shackleton trawled a 16 mile radius for her servants, as well as as far a field as London, but I think this was because she was having such trouble obtaining staff locally (and considering her household circumstances maybe this is not surprising). Maybe if the trawling area was so small, the mistresses didn't need a written reference, they'd know from the grapevine if a servant was untrustworthy.

I think the importance laid on obtaining and having a reference (or "character" as it was then called) is more of a Victorian and Edwardian requirment.