Thursday, March 30, 2017

Data is good, 'mkay?

Early this month, the story broke of a study in the Netherlands that had found large inaccuracies in the measurement of "smart" electricity meters. I started writing this post in response to that story, but it's far too long, so I've broken it into parts. Part One is why 98% of the bad things people say about smart meters are wrong. Part Two will be about the Dutch story specifically. Part Three, if I get that far, will be about what is wrong with smart meters.

Even so, it's a wall of text. Sorry about that.

I like "smart" meters.

Like most houses in New Zealand, my home is already fitted with one - has been since 2013. But more pertinently: as the billing and reconciliation manager of a small electricity retailer, I handle about a million smart meter reads every month. And I am, perforce, actively involved in working to get the things deployed "everywhere". (The scare quotes are there because we all know they'll never be everywhere. Hey, there are people out there who are still running Windows 98. But for statistical values of "everywhere", it is possible.)

And so I am more than averagely aware, both of the uses of these things, and of the popular resistance to them. This resistance is well organised. It looks like a genuine grassroots movement, but at the same time I would guess it's quietly egged on by interested parties, such as electricity retailers, meter manufacturers and meter readers, as well as the usual troublemakers (journalists and self-styled activists).

Sample website here. (I am indebted to, for providing a central repository of the more plausible arguments. It beats heck out of such amateurish efforts as, which earnestly argues that all wireless technology is evil because RADIATION!!!) Marginally more rational arguments are:

  1. remote disconnection
  2. data privacy/security
  3. incompatibility between meter types and readers
  4. lack of benefit to customers
  5. inherent unreliability in the meters themselves.

The last of these is pretty technical, and it's really the theme of part two of this series. For now, let me consider the less geeky objections. To deal with the weakest point first:


If your smart meter is located under your pillow, then it's just possible that you might absorb as much radiation from the smart meter, during the night, as you would during a two-minute conversation on a mobile phone. If it's located a few feet away from your bed, then the dosage will be less than 2% of that. If it's, like mine, on the far side of the house - then in terms of radiation strength, you would have a hard time even detecting it above the background radiation of terrestrial broadcasts, phone signals, alarms, car keys, baby monitors, garage door openers and all the other hum of 21st-century technology going on all around you. And don't even think about getting a wifi router, smart TV, or IOT-enabled device of any sort.

Of course there's no such thing as "totally harmless radiation", in the same way as there's no such thing as "totally harmless sunlight" - but most of us have long ago decided that the undoubted benefits outweigh the hypothetical risk. If you want to dissent from that opinion, then... I'm sorry, but I see no reason why the rest of us should have to pay for your hyper-caution. (It's not a huge cost, yet. At present, you're probably costing us something in the ballpark of $5-7 per month. But that figure will rise sharply as the number of legacy meters declines, because the economies of scale in handling them will fade away, so be ready for a substantial surcharge in a few years' time.)

Remote disconnection

Really, there are three separate fears under this heading. Remote disconnection may be done by your power company, by the police/government/similar, or by criminals (terrorists or other hackers). So let's take them in turn.

The procedure for your power company to disconnect you is closely regulated. We have to make certain efforts to warn you before we do it. There are laws prescribing the methods, wording, and timing of those communications. And - this is the key point - those laws are exactly the same, regardless of what type of meter you have. The big difference is that the whole process is a lot quicker (and easier to reverse), and cheaper, with a smart meter. This alone is a big win for customers. And if we do it wrong, either way, you can sue us.

At this point, I once had the rejoinder "for now, maybe, but what happens when the capacity is getting slim and the grid is desperate to shed load?" The answer to that is a thing called a ripple relay, which disconnects some loads in your house that are both heavy, and unlikely to be time-critical. (Generally, that means "an immersion heater".) The worst that happens is you have to go without hot water for a few hours, while the rest of your house continues to work as normal. Of course having no hot water is bad - but only until you compare it to the alternative, which is uncontrolled brownouts affecting your entire supply.

If you're worried about the police/government taking the time to disconnect your power supply - get your priorities straight. I've never heard of them doing that, and can only imagine that if a SWAT team is about to bash in your door, the power supply to your house is not going to affect things much either way. (Okay, so you may have some half-baked fantasy about standing them off with a homemade Gauss rifle or something. I recommend buying a battery. And life insurance.)

As for evil hackers/terrorists doing it: well, terrorists already have much simpler, low-tech ways of doing much greater damage (Google "northeast blackout 2003" to see what I mean). If it does happen to you - well, congratulations on being the world's first ever victim of an entirely new type of crime. Now call your power company and tell them what's happened, and your lights will be back on in a jiffy.

Data privacy/security

Probably the most overblown of all concerns. The issue here is the granularity of data.

When I look at a set of smart meter reads, I see usage per half-hour for the previous month. (Note, the previous month. We don't get the data in anything like "real time". Not even the meter owner knows how much you're burning right now. There are companies that get the data only one day late, but that's a significant amount of extra work for them.) I may be able to see, for instance, that your usage was around 0.2-0.3kW overnight, then started to ramp up at 5 a.m., peaking at 2kW at 7 a.m. and then dropping off to a steady daytime rate...

And that's all very interesting. It gives me an idea of what time your household gets up, showers and has breakfast. But - that's about all it tells me. It doesn't tell me what you have for breakfast, or whether you watch TV while you eat it, or whether you shower in the morning. There's no way of distinguishing between "a TV turned on for the whole half hour" and "a toaster turned on for a few minutes". I might guess that the 5 a.m. ramp-up is a timed dishwasher - but that's only a wild guess, it might just as easily be an immersion heater, or even an early riser getting in some quality screen time before the rest of the household gets up. There are dozens of possible explanations for any given pattern, and without more data there's no way of picking one.

And that is how I know that companies like ONZO are full of shit. In order to do what they claim to do, they would need more - much more - than your smart meter reads. I would guess that when they demo their technology, they do it on households that have a lot of interconnected and/or spying stuff in the house (an IOT-enabled toaster, or an XBox, or a smart TV, or a cable box, or Alexa, for instance - any of these would provide some serious quality spy data to supplement the meter reads). If you have any of those things in your house, then you're giving away far more data than your meter can gather.


This is a real problem in the UK. It is not, however, a problem in New Zealand, even though our technology is not all that exciting. Because it's not a technological problem at all - it's a structural or organisational problem.

There are two big companies that provide metering in Auckland. (Actually there are about eight, but only two major players controlling well over 90% of the market - Metrix and AMS.) These companies fit and maintain the smart meters, and collect read data from them. They pass that data to us retailers in (reasonably) well defined file types. If you want to be a retailer in New Zealand, you learn those file types and love them - and that's all you have to do. Dealing with the fine detail of extracting raw data from the meter and translating it into this standard format - that's the meter owner's problem (and we, the retailer, pay them for the service). If you switch retailer, the same meter continues recording the same data and it's collected by the same company - the only thing that changes is who they send it to for billing.

From what I gather, in the UK, power companies are supposed to read their own meters. That's... well, let's just say that it looks very much like a racket got up by the big, established retailers to raise barriers to entry for competition. It needs to be fixed with a clawhammer.

Lack of benefit to customers

This is probably the hardest case to argue, because it means engaging with a counterfactual: how different would the consumer's experience be, if there were no smart meters? And that, of course, drags us into a whole briar patch of assumptions.

There are some solid numbers we can point to. First, and most obviously, the cost - of all things related to meter reads and changes - drops dramatically with a smart meter. You want to get a read because you're moving in or out? That drops from $15 to zero. You've been disconnected and want to be reconnected? Price drops from $150 (hey, disconnection/reconnection is two site visits) to $40. You've paid your overdue bill and want to be reconnected immediately? The average time to accomplish that task drops from 2-3 hours to about 15 minutes. All of these are real, tangible gains for the customer.

Then there's timeliness and accuracy of billing. You may not think it's a great benefit to have an accurate bill. But when your immersion heater is playing up, or your kid has taken to leaving the heater on all night, or someone has switched your spa pool on and left it - often the first you'll know about it is when you see your power bill. If some of your bills are estimated, you may not know anything about it until two, three months after the problem has started, at which point you're already several hundred bucks out of pocket. With a smart meter, you'll at least get a chance to notice it much sooner (because every bill is based on a real read). If you ask your retailer for more data (as is your right), then you can even figure out exactly when it started, and hence who's likely to blame.

And finally, there's the competitive effect. I don't know the UK industry in much detail; but here in New Zealand, I know of at least three retailers (out of a total of about 18) who would never even have entered the market, if they hadn't seen Opportunities in the data afforded by smart meters. (One company makes a big advertising feature of "an hour of free power" every day. Can't do that without a smart meter.) So without them, there'd be at least that much less competition.


It's not hard to come up with superficially plausible objections to any new technology. And we live in an age when tens of thousands of wannabe activists are just itching to find a cause, preferably a conspiracy, to latch onto and tear down, to make their names and claim their place in celebrity culture. We reward people in our society for being loud, rather than coherent. Heck, look at Donald Trump.

But we shouldn't judge arguments on the shrillness of their proponents. An awful lot of what gets argued, online, is just plain bollocks, no matter how fervently the true believers cling to it. And the whole "smart meter conspiracy" story is firmly in that category.

Next instalment: the dodginess of the meters themselves. How bad are they, really? (Spoiler: far from perfect, but not nearly as bad as the press would have you believe.)