You can't refuse a delay of game.
if the other team is willing to take a delay of game penalty than you don't give it to them.