Dimensions of Energy Behavior: A ToolKit for Assessing Behavior-Based Energy Interventions

Southern California Edison will be presented with more and more ideas for program offerings that are styled as "behavioral" offerings; however, whether these offerings do have an effect on energy savings behaviors is unknown. This standardized scale will allow SCE to assess the relative effectiveness of offerings with behavioral components by tracking changes in the scale. This will help understand which behavioral offerings are more effective overall, as well as to understand which behavioral offerings might be better for a particular customer segment. The lack of a standardized scale has made it difficult to make such comparisons in the past. Use of the scale will help utilities make objective assessments of the claims by behavioral-offering developers. Ultimately, any behavioral program will need to describe its effect on customer behavior. To this end, a standardized tool is necessary to objectively assess effects on behavior. A standardized scale of behavioral effects gives the utilities some metric against which to assess the behavior change claims of third-party developers and implementers. Currently there is no objective standard by which to make these decisions.

Behavioral interventions in demand side management (DSM) programs (i.e., those targeting savings through consumer behavior regarding energy use) are based on the idea that people can be encouraged to use less energy if the underlying determinants of their behavior are influenced in some way. Research on such programs suggests potential savings, but results vary and much is still unknown about the mechanisms by which behavior-based interventions result in energy savings. This is due in part to the way these programs are typically evaluated. Most energy efficiency evaluations use the amount of energy use (measured in kilowatt hours (kWh)) as the dependent variable for measuring effectiveness. Although this is an ideal measure of whether energy efficiency interventions work, additional information could add significantly to our understanding about how and for whom they work. While recent efforts have been made to include additional variables (e.g., attitudes, behaviors, user experience), widespread agreement on variable measurement is lacking and current instruments have not been psychometrically validated to test and improve internal validity. Greater consistency and the use of validated instruments would improve our overall ability to account for variation in treatment effects and identify the underlying causes of intervention failures. As such, this report presents psychometric testing of a set of scales that can be used to collect self-report data as a part of evaluation of behavioral interventions. It builds from preliminary instruments drafted as part of the International Energy Agency Demand Side Management Program Task 24 on Behavior Change (Karlin & Ford, 2013a) by refining and psychometrically validating the following scaled-for-use-infield studies within California. 1. Norms (e.g., efficacy, social norms) 2. Practices (e.g., onetime, habitual) 3. Material culture (e.g., appliance stock) 4. Context (e.g., demographics, housing), 5. User experience (e.g., ease of use, engagement) A full survey instrument with these scales can be completed via computer, paper or phone in 10-15 minutes. Broad use of such instruments can improve and aggregate our overall knowledge across studies and contribute to a more robust understanding of behavioral interventions as a resource. Such an understanding is necessary to provide consistent and reliable EM&V calculations across behavioral interventions in DSM programs that can be compared with supply-side sources of energy and valued in practice as much as they are valued in theory.