Using your plumbing during a power outage is completely safe. Obviously, if you are worried about electrical surges or other rare occurrences then you shouldn’t take the risk for a hot shower. Your safety comes first during a power outage which is why it is important to have flashlights ready for just such an occasion.