Well, we can write off Mrs. Bennet because she is the original "silly girl" herself, so I don't think she's capable of guiding her daughters properly. She is quite shallow and, as a result, puts the emphasis on appearance and good times -- and not on helping them develop their inner beauty, intelligence and character.

As for Mr. Bennet, I just think he's exhausted being surrounded by all these women and very disappointed after years of waiting for the son to come along. So, he has pretty much just given up. And, as you suggested, he's found solace in his library and his books.