When replacing a Badge you previously equipped on a Survivor or Hero with a new one, you will be asked if you would like to remove or scrap the old Badge.
 
Removing the Badge will allow you to keep the Badge in your game for a small Gold fee. Once it's removed the Badge will be available to be equipped again from Badge Inventory.
 
Scrapping a Badge gives you some XP. Once a badge scrapped it can't be recovered.