I was tempted to post a screen shot of Robert Swann delivering the line too, but perhaps that'd be straying too far into adaptations...

"Trust me" conveys the same sort of meaning as "Don't worry" but, being specific to the Colonel's errand, can't be mistaken for a vain attempt to raise hopes for Marianne.