"energy efficiency improvements that, on the broadest considerations, are economically justified at the microlevel, lead to higher levels of energy consumption at the macrolevel."
The structural reason is simple: the market is highly price elastic, so decreasing prices a little lifts total sales considerably. In economics, this is a well solved problem for non-monopoly markets, "Profit Maximisation" occurs when MR = MR (Marginal Revenue equals Marginal Costs). [For monopolies, MR = 2*MC, IIRC.]
In the 70's & 80's at O.T.C., we made record profits each and every year - by exceptional marketing and sales strategy, which included dropping prices every year. [The TV adverts series, like the 'Memories' campaign, won many awards.]
This was driven by the technology: Moore's Law drove the per unit cost of services in both cable and satellite down exponentially.
Profits margins increased because full cost reductions weren't passed on. By the mid-80's it was cheaper from the East Coast to call London or New York than use Telstra to call Perth ($1/min).
This is a lesson Telstra Management never learnt and was obviously lost when O.T.C. was subsumed into 'the Borg' in 1992: passing on part of the Moore's Law savings, Revenue and Profits both increase.
Economic theory is very clear on this point:
Traditional (Premium) Telco Pricing isn't just 'bastardry', it kills your profits which will eventually kill your company.