Background video by Guryanov Andrey / shutterstock

Background video by Incredible Arctic / shutterstock

Evidence that NILM can help save energy...

1) People want disaggregated energy data

2) Behaviour affects energy consumption

modifying behaviour → reduce energy consumption

3) People are bad at estimatingthe energy consumption of their appliances

→ Fix the ‘information deficit’ then users can operate as rational ‘resource managers’

(I’m now sceptical of this idea)

4) Multiple studies report that disaggregated feedback reduces energy consumption

5) Smart meters

Systematic reviews

• Common in medicine, social sciences etc.
• Distinct from ‘narrative’ reviews
• Aim to collect all papers matching a defined search criteria
• Quantitative summary of each paper and biases
• Quantitative synthesis of all results

Background image from UCSF

Literature search

• ‘disaggregated AND [energy|electricity] AND feedback’
• ‘N[I|A|IA]LM AND feedback’

The studies

12 groups of studies identified

Research questions

Q1. Can disaggregated electricity feedback enable ‘energy enthusiasts’ to save energy?

• Very likely...
• Weighted-mean energy reduction = 4.5%
• A lot of uncertainty...

Biases

The Hawthorne Effect

• Hawthorne effect is illustrated by Schwartz et al. 2013:
  • Randomised controlled trial
  • 6,350 participants split into 2 groups: control & treatment
  • Treatment received weekly postcard saying: ‘You have been selected to be part of a one-month study of how much electricity you use in your home... No action is needed on your part. We will send you a weekly reminder postcard about the study...’
  • Treatment group reduced energy consumption by 2.7%!
• Failure to control for Hawthorne very likely to be strong positive bias
• 8 studies did not control for Hawthorne

Other biases

• 6 studies used attention-grabbing displays
• Home-visits
• 10 studies were short (4 months or less)
• Cherry-picking statistical analyses and comparison periods?
• 8 studies used sub-metered data, hence avoiding mistrust from participants
• Publication bias?

Q2. How much energy would the whole population save?

• All 12 studies suffer from ‘opt-in’ bias
• Subjects self-selected hence are probably more interested in energy than the average person
• Very likely to be a strong positive bias

Q3. Aggregate versus disaggregated feedback

• 4 of the 12 studies directly compared disaggregated against aggregate feedback
  • 3 studies found aggregate to be more effective
  • 1 study found aggregate to be equally effective
  • 2 field trials & 2 lab experiments

The 2 field trials...

Sokoloski 2015

• Randomised controlled trial
• 70 households recruited in California
• 3 weeks
• Rebecca Sokoloski, Disaggregated Electricity Consumption: Using Appliance-Specific Feedback to Promote Energy Conservation, M.A. thesis in Psychology, California State University San Marcos, 2015

Sokoloski’s results

Sokoloski’s results

Energy reductions:

• IHD: 8.1% (statistically significant)
• Disaggregation: 0.5%
• Control: -2.5%

Sokoloski’s results

Findings from surveys:

• Follow-up survey revealed that the disag group were not significantly more likely to be willing to replace large, inefficient appliances compared to controls or IHD group.
• Neither controls nor the disag group significantly increased their perception of control (initial survey versus follow-up).
• IHD group did increase their perception of control.

Sokoloski’s results

Findings from surveys:

• Users viewed their devices:
  • 0.86 times per day for disag users
  • 8.16 times per day for IHD users

PG&E 2014 trial

• 1,685 PG&E customers
• 3 months
• Half got IHD & half got Bidgely
• Users choose intervention
• Did not tease apart consumption of IHD vs Bidgely
• Churchwell et al., HAN Phase 3 Impact and Process Evaluation Report, technical report by Nexant, 2014

PG&E 2014 trial results

• IHD users significantly more likely to report taking actions to reduce electricity usage and to use their device to deduce power demand of individual appliances(!)
• Several users did not trust the disag data.
• IHD more successful in communicating power demand now

Bidgely have redesigned their website since these studies

Conclusions

• NILM has many uses! This talk just considered one use!
• Available evidence suggests that aggregate feedback is more effective than disag feedback
• But these results confounded by effect of IHD versus website
• Disag feedback might drive savings of 0.7% - 4.5% in general population
• Disag feedback might drive larger savings in ‘energy enthusiast’ populations
• Fine-grained disag may not be necessary
• But! Lots of gaps in our knowledge. Cannot robustly falsify any hypotheses yet.

Suggestions for future studies

• Compare aggregate versus disagg (both on an IHD)
• Compare 2 groups:
• Aggregate on an IHD
• Aggregate (on an IHD) + disagg (on a website)
• Compare fine-grained disag versus coarse-grained disag
• If you have data then please consider releasing it; or writing a paper; or collaborating with someone who will write a paper with you!

Users might become more interested in disag feedback if:

• Energy prices increase
• Concern about climate change deepens
• Disag accuracy increases or if designers communicate uncertain estimates
• Lots of ideas in the literature about how to improve disag feedback. e.g. disag by behaviour; or display feedback near appliances; or provide better recommendations etc.

This presentation is based on my paper: "Does disaggregated electricity feedback reduce domestic electricity consumption? A systematic review of the literature", In 3rd International NILM Workshop, Vancouver, Canada, 14-15 May 2016.