eco+ Evaluation

This report documents the evaluation of the energy and on-peak demand savings attributable to the eco+ software feature that was rolled out to ecobee thermostats in the San Diego Gas & Electric Co. (SDG&E) service territory. The implementation of the feature was conducted on a randomized encouragement design (RED) basis to facilitate a sound impact evaluation net of selection effects. The feature was offered to eligible residential customers in August 2019. The primary objective of this report is to document the findings of an ex post (after the fact) study that estimates energy and on-peak demand savings attributable to the eco+ software feature between September 2019 and August 2020.

The study was structured as a randomized encouragement design (RED) experiment. With a RED, different randomly selected samples of customers are offered an experimental treatment (in this case, to install the eco+ software feature) and another random group of customers is not offered anything (e.g., the control group). Some who are offered the treatment take it and some do not. Because each sample is a statistical clone of the other due to the random selection, comparing the behavior of the encouraged group with that of the control group allows for an unbiased assessment of the impact of the treatment. This analysis requires a two-step process in order to isolate the impact of the encouragement (e.g., the offer of a treatment) from the treatment itself.

Daily energy and on-peak demand impacts were estimated for two different climate regions in SDG&E’s service territory (Cool and Moderate). Impacts were not estimated in the Hot climate region due to low sample sizes and acceptance rates. Daily energy and on-peak demand impacts were also estimated for net metered (NEM) and non-net metered (non-NEM) customers.

Key findings pertaining to the on-peak load impacts analysis include:
• Winter on-peak load impacts for non-NEM customers were 0.06 kW and statistically significant at the 90% confidence interval
• Winter on-peak load impacts for NEM customers were not statistically significant
• Summer on-peak load impacts for non-NEM customers were 0.05 kW and statistically significant at the 90% confidence level
• Summer on-peak load impacts for NEM customers were 0.16 kW and statistically significant at the 90% confidence level