] Sometimes I think I get it - that since he feels/thinks less highly of Lizzy now, its easier for him to be nonchalant about the whole thing. But on subsequent readings I get more confused.

When I read Mr Collins proposal to Lizzy I find him incapable to take Lizzies answers as fact. The second of her refusals she tries to introduce him to the idea that Lady Catherine de Bourgh would not approve of her and he thinks about that but dismisses the thought.
He tries four more times without losing fate, he still thinks he has succeeded when Lizzy leaves the room. It is only when Mrs Bennet start talking about Lizzy as a headstrong girl that the facts sink in.
So I think he finds it a lucky escape that he did not have to bring her in front of Lady Catherine. He even believes that he made the decision of accepting Lizzies no, because he had not asked Mr & Mrs Bennet about changing Lizzies mind. Even though Mrs Bennet said it clearly that she would intervene for him and he did not try to stop her.
I donīt know if that is what you find confusing but I see it as a possibility for another interpretation. :)