You are the psychologist ;-) and I think you're right about the psychological toll his mother and sister must have taken on Edward's (dare I say) self esteem.

and now, looking back from the perspective of Ch 22 (Lucy confides in Elinor), I read this statement of Edward's as referring to his engagement... that because he is shy and awkward, and inadequate in the eyes of his own family, he has felt most at ease with his social inferiors. Around women like his mother and sister, who judge a man by his ambition and poise, he feels like a loser. With a younger, poorer, uneducated girl of no family, he could feel manly.

He didn't know at the time that there were women like the Dashwoods who would be neither haughty and materialistic like women of his own circle, nor "low" like Lucy... that he had a third option.