I think you've hit the nail on the head. There were plenty of clues (anvils?) that could have pointed each to the truth. But both were busy looking at things from their own narrow perspective (and yes, it is a very human failing!)

Mr. Elton, of course, is very handsome and well-liked, and Miss Woodhouse was always inviting him over, and of course she had her little friend Harriet there so it wouldn't look at all improper, and she was so kind as to always ask for his help in improving Harriet further in social graces...