Taken at face value this does seem a dreadful thing to say about a baby, doesn't it?

But I am sure I have come across the expression "Born to be hanged". It was used referring to particularly lively, naughty boys, and, in my mind, was uttered with a sense of affectionate exasperation. Maybe JA's comment should be understood in a similar way.

I think it was said of the young hero of Henry Fielding's 1749 novel "Tom Jones".