...as you point out, concern can cause the anger -- but the fact that Mr. Bennet goes from anger to indolence with no intermediate stop for concern makes me wonder how much concern there is, if any.

Since we aren't privy to many of Mr. Bennet's thoughts, it's hard to tell specifically why he is so angry. He does tell Lizzy when she tries to talk him out of letting Lydia go to Brighton that "you will not appear to less advantage for having a couple of -- or I may say, three very silly sisters" but he is so full of rationalizations in that scene that it is difficult to tell what he really thinks. Perhaps having Lydia elope and apparently not marry brings home to him that Lizzy's sister's behavior might indeed impact her future happiness, or perhaps he realizes that having a sister who is a "fallen woman" is a bit more serious than having one who is merely silly. He may not care what the neighbors think, but even he has to realize that Jane and Lizzy's pool of potential husbands is among their neighbors, and if the neighbors shun the family, they have considerably less chance of ever marrying.

I think that the root of his rage is the effect that Lydia's behavior will have on Lizzy and Jane's chances of making respectable marriages, as well as perhaps some rage at himself for having been so indolent as to have allowed Lydia to be in that position. By the time he talks about his feelings about his actions, they are more like resigned guilt, but he may have had stronger feelings earlier.