I think you are correct when you say that Lizzy is attracted to Wickham more by his temper than his attractiveness.

We see some evidence of that in her interaction with Colonel Fitzwilliam. We know he is not as attractive as Wickham: "Colonel Fitzwilliam, who led the way, was about thirty, not handsome, but in person and address most truly the gentleman."

But we also know that Elizabeth enjoys his company "It was plain to them all that Colonel Fitzwilliam came because he had pleasure in their society, a persuasion which of course recommended him still more; and Elizabeth was reminded by her own satisfaction in being with him, as well as by his evident admiration for her, of her former favourite George Wickham..."