I'm afraid your initial premise is most likely wrong. Have a look at the link below. The salient points, in no particular order, are:

1. The point of duelling was not to kill but to satisfy honour.

2. Pistols were the generally preferred duelling weapon by about 1780, partly because of the decline in swordsmanship and partly due to the fact that swords were less often carried by military men.

3. Taking careful aim was, at least initially, considered "unsporting", as were rifled barrels (making the weapons more accurate) (see 1. above). The stereotypical form of taking a number of paces, then immediately turning and firing, made taking aim difficult. The development of heavier, more accurate weapons equipped with sights actually led to the decline of duelling, because the element of chance was increasingly eliminated.

4. Duellists sometimes fired to miss. In the famous Burr-Hamilton duel in New York in 1804, the latter fired into the air, as did the Earl of Winchelsea in a duel with the Duke of Wellington in 1829.

5. Statistically, while "a fifth of those involved in duels with swords died, only six and a half percent of those who chose pistols died as a result of their encounter (though a third were injured as a result)" - this despite duelling ranges initially being six yards.

Consequently, it's almost certain that the Willoughby-Brandon duel was with pistols. As duelling pistols by this stage were beginning to be more accurate, I think it would be somewhat unusual for both parties to escape scot free. Therefore, I think the most likely scenarios would have been that one party fired to hit and missed, while the other fired into the air or the ground, that both parties fired to miss, or that both fired to hit but one party's pistol misfired, upon which the seconds (the parties' assistants) decided that honour had been satisfied. Brandon, as the challenger, would have fired first, if they did not fire simultaneously.