During a fever, skip the bath to prevent chilling. Instead, perform lukewarm water sponging and prioritize changing your baby's clothes frequently to help them feel fresh and comfortable while they recover.