Tankless heaters can ideally be somewhat more efficient than storage water heaters. In both kinds of installation (centralized and POU) the absence of a tank saves energy as conventional water heaters have to reheat the water in the tank as it cools off, called standby loss. With a central water heater of any type, water is wasted waiting for water to heat up because of the cold water in the pipes between the faucet and the water heater.

Point of use tankless water heaters are located right where the water is being used, so the water is almost instantly hot, which saves water. They also save even more energy than centrally installed tankless water heaters because no hot water is left in the pipes after the water is shut off. However, point of use tankless water heaters are usually used in combination with a central water heater since they are usually limited to under 6 litres/minute (1.5 U.S. gallons/minute), as the expense of buying a heater for every kitchen, laundry room, bathroom, or sink can outweigh the money saved in water and energy bills. In addition, point of use water heaters until recently were almost always electrical, and electricity is often substantially more expensive than natural gas.

Tankless water heaters can be divided into two categories: "full on/full off" and "modulated". Full on/full off units do not have a variable power output level; the unit is either on or off. Modulated tankless water heaters base the heat output on the flow of water running through the unit. This is usually done through the use of a 'flow sensor', modulating gas valve, inlet water temperature sensor and an outlet water temperature sensor-choke valve and means that the occupants should receive the same output temperature of water at differing velocities, usually within a close range of ±2 °C.

The high efficiency condensing combination boiler[13] provides both space heating and water heating, an increasingly popular choice in UK houses. In fact, combination boilers now account for over half of all the new domestic boilers installed in Britain.

In certain parts of South America as well as Costa Rica and Puerto Rico, a point of use style water heater commonly referred to as the "Electric Shower Head" is used in many residential and some commercial installations. As the name implies, an electric heating element is incorporated into such shower heads to heat the water. However, many of these units are often poorly installed, often with exposed wiring in wet locations.

Under current North American conditions, the most cost effective configuration from an operating viewpoint is usually to use a central tankless water heater for most of the house, and install a point of use tankless water heater at any distant faucets or bathrooms. However, this may vary according to how much electricity, gas and water costs in the area, the layout of the house, and how much hot water is used. Only electric tankless water heaters were available at first and they are still used for almost all point of use heaters, but natural gas and propane heaters are now common. When consumers are considering a whole house gas tankless unit, they are advised to look at how the unit functions when raising the water temperature by about 42 °C (75–77 °F). Thus, if they live in a cold weather climate, they are advised to look at the unit's capacity with 3-10 °C (38–50 °F) inlet water temperatures, and find a size that produces approximately 15 litres/minute (4 gpm) even in winter if they have a typical-sized house and desire what is called a 2-appliance heater. This same unit may produce 25-30 litres/minute (6.3–6.9 gpm) in summer with higher inlet temperatures, but there is greater interest in year round production and usability.