Background video by Guryanov Andrey / shutterstock

What is disaggregation?

Aggregate Energy Bill

Itemised Energy Bill

The many names of ‘energy disaggregation’

Disaggregation research

George Hart

George Hart, Prototype Nonintrusive Appliance Load Monitor, MIT Energy Lab / EPRI Technical Report, 1985

George Hart, Nonintrusive Appliance Load Monitoring, Proceedings of the IEEE, 1992

Why bother with disaggregation?

Background video by Incredible Arctic / shutterstock

Background image from phys.org/Gregory Heath/CSIRO

Evidence suggesting that disaggregated bills might help save energy...

(Ideas I believed when I started my PhD)

1) People want disaggregated energy data

2) Behaviour affects energy consumption

modify behaviour → modify energy consumption

1977: the Twin Rivers program

Socolow, The twin rivers program on energy conservation in housing: Highlights and conclusions,Energy and Buildings, 1977

2:1 range in energy consumption between identical houses

Socolow, The twin rivers program on energy conservation in housing: Highlights and conclusions,Energy and Buildings, 1977

3) People are bad at estimatingthe energy consumption of their appliances

→ Fix the ‘information deficit’ so users can operate as ‘rational resource managers’

(I’m now sceptical of this idea)

4) Multiple studies report that disaggregated feedback reduces energy consumption

5) Smart meters

Systematic reviews

• Common in medicine, social sciences etc.
• Distinct from ‘narrative’ reviews
• Aim to collect all papers matching a defined search criteria
• Quantitative summary of each paper and biases
• Quantitative synthesis of all results

Background image from UCSF

Literature search

• ‘disaggregated AND [energy|electricity] AND feedback’
• ‘N[I|A|IA]LM AND feedback’

The studies

12 groups of studies identified

Research questions

Q1. Can disaggregated electricity feedback enable ‘energy enthusiasts’ to save energy?

• Very likely...
• All 12 experiments were opt-in
• Weighted-mean energy reduction = 4.5%
• Full meta-analysis probably not possible
• A lot of uncertainty...

Biases

The Hawthorne Effect

• Hawthorne effect is illustrated by Schwartz et al. 2013:
  • Randomised controlled trial
  • 6,350 participants split into 2 groups: control & treatment
  • Treatment received weekly postcard saying: ‘You have been selected to be part of a one-month study of how much electricity you use in your home... No action is needed on your part. We will send you a weekly reminder postcard about the study...’
  • Treatment group reduced energy consumption by 2.7%!
• Failure to control for Hawthorne very likely to be strong positive bias
• 8 studies did not control for Hawthorne

Other biases

• 6 studies used attention-grabbing displays
• Home-visits
• 10 studies were short (≤ 4 months)
• Cherry-picking statistical analyses or comparison periods?
• 8 studies used sub-metered data, hence avoiding mistrust from participants
• Publication bias?

Q2. How much energy would the whole population save?

• All 12 studies suffer from ‘opt-in’ bias
• Subjects self-selected hence are probably more interested in energy than the average person
• Very likely to be a strong positive bias

How much energy would the whole population save?

• No “perfect” correction for opt-in bias
• Study in Sweden (Vassileva et al. 2012):
  • 2,000 households given access to website analysing their aggregate energy demand
  • Only 32% accessed the website. They saved 15%.
  • Those who did not access website did not reduce energy.
  • Average saving = 32% x 15% = 5%

Q2. How much energy would the whole population save?

• Average opt-in rate = 16%
• Average saving across population = 16% x 4.5% = 0.7%

Q3. Is ‘fine-grained’ feedback necessary?

Q3. Is ‘fine-grained’ feedback necessary?

Home Energy Analytics (HEA) studies

• Average reduction of 6.1%
• But no control group; and home-visits for some
• Coarse-grained feedback may be sufficient
• No studies directly compared fine-grained feedback against coarse-grained.

Q4. Aggregate versus disaggregated feedback

• 4 of the 12 studies directly compared disaggregated against aggregate feedback
  • 3 studies found aggregate to be more effective
  • 1 study found aggregate to be equally effective
  • 2 field trials & 2 lab experiments

The 2 field trials...

Sokoloski 2015

• Randomised controlled trial
• 70 households recruited in California
• 3 weeks
• Rebecca Sokoloski, Disaggregated Electricity Consumption: Using Appliance-Specific Feedback to Promote Energy Conservation, M.A. thesis in Psychology, California State University San Marcos, 2015

Sokoloski’s results

Sokoloski’s results

Sokoloski’s results

Energy reductions:

• IHD: 8.1% (statistically significant)
• Disaggregation: 0.5%
• Control: -2.5%

Sokoloski’s results

Findings from surveys:

• Follow-up survey revealed that the disag group were not significantly more likely to be willing to replace large, inefficient appliances compared to controls or IHD group.
• Neither controls nor the disag group significantly increased their perception of control (initial survey versus follow-up).
• IHD group did increase their perception of control.

Sokoloski’s results

Findings from surveys:

• Users viewed their devices:
  • 0.86 times per day for disag users
  • 8.16 times per day for IHD users
• Returning devices:
  • 2 of 7 (29%) wanted to return disag device
  • 2 of 30 (7%) wanted to return IHD

PG&E 2014 trial

• 1,685 PG&E customers
• additional no-contact controls
• 3 months
• Half got IHD & half got Bidgely
• Users choose intervention
• Did not tease apart consumption of IHD vs Bidgely
• Churchwell et al., HAN Phase 3 Impact and Process Evaluation Report, technical report by Nexant, 2014

PG&E 2014 trial results

• IHD users significantly more likely to report taking actions to reduce electricity usage and to use their device to deduce power demand of individual appliances(!)
• IHD more successful in communicating power demand now

PG&E 2014 trial results

Most common complaint from Bidgely users was about the disag feature:

• Several users didn’t trust the disag data
• Some were unsure whether they should assist the algorithm by turning loads on or off
• Some thought categories were too few or too broad
• Some didn’t like that they couldn’t add new disag categories

PG&E 2014 trial results

Frequency of viewing devices

PG&E 2014 trial results

Percentage of customers saying they saved energy

PG&E 2014 trial results

Reported actions taken in response to feedback

Bidgely have redesigned their website since these studies

Conclusions

• NILM has many uses! This talk just considered one use!
• Available evidence suggests that aggregate feedback is more effective than disag feedback
• But these results confounded by effect of IHD versus website
• Disag feedback might drive savings of 0.7% - 4.5% in general population
• Disag feedback might drive larger savings in ‘energy enthusiast’ populations
• Fine-grained feedback may not be necessary
• But! Lots of gaps in our knowledge. Cannot robustly falsify any hypotheses yet.

Suggestions for future studies

• Compare aggregate versus disagg (both on an IHD)
• Compare 2 groups:
• Aggregate on an IHD
• Aggregate (on an IHD) + disagg (on a website)
• Compare fine-grained feedback versus coarse-grained feedback

Users might become more interested in disag feedback if:

• Energy prices increase
• Concern about climate change deepens
• Disag accuracy increases or if designers communicate uncertain estimates
• Lots of ideas in the literature about how to improve disag feedback. e.g. disag by behaviour; or display feedback near appliances; or provide better recommendations etc.

This presentation is based on my paper: "Does disaggregated electricity feedback reduce domestic electricity consumption? A systematic review of the literature", In 3rd International NILM Workshop, Vancouver, Canada, 14-15 May 2016.

Appendix

Why reduce energy consumption?

2015 Paris agreement on Climate Change

Background image from phys.org/Gregory Heath/CSIRO

Future Antarctic contributions to global mean sea-level (GMSL)

Past and future changes in CO2 and mean temperature

Clark et al., Consequences of twenty-first-century policy for multi-millennial climate and sea-level change,Nature Climate Change, 2016