After your outpost is raided, your defenders will recover immediately and continue to defend what's rightfully yours. Don't worry about them, they'll be fine!