Protect Your Tankless Water Heater From Hard-Water Scale A water softener for tankless water heater protection should be installed on the incoming cold-water line, before water reaches the unit. For hard water, a salt-based, demand-regenerated softener is the most reliable choice because it removes the calcium and magnesium that form scale inside the heat exchanger. Add a sediment pre-filter, choose a system sized for your household's water use and peak flow, and keep the tan