It's not every day ...

 ... you get to listen to a Nobel prize winning economist, so big shout out to the University of Auckland for its Dean's Distinguished Virtual Public Lecture last night by Nobel Laureate Jean Tirole, Professor at the Toulouse School of Economics, and further hat tip to the university's extending the availability of the lecture to the Law and Economics Association of New Zealand (LEANZ, you are a member, aren't you?).

Tirole was talking about "Digital Dystopia", or as the invite put it, "How transparent should our life be to others? Modern societies are struggling with this issue as connected objects, social networks, ratings, artificial intelligence, facial recognition, cheap computer power and various other innovations make it increasingly easy to collect, store and analyse personal data. While this holds the promise of a more civilised society ... citizens and human rights activists fret over the prospect of mass surveillance by powerful players engaging in the collection of bulk data in shrouded secrecy. A dystopian scenario will be used to emphasise the excesses that may result from an unfettered usage of data integration in a digital era".

Truth be told, it wasn't the easiest presentation to follow: not because of Tirole, whose style is affable and conversational, but more because Zoom webinars are not the best medium for presenting equation-rich material, or at least not for those of us below Tirolean levels of mathematical deftness. Most of the invited panel of commentators appeared to have read it beforehand, and that was the sensible thing to do - here's a link.

Even if the details of the maths beat me, I got the message, and it's plausible. Tirole said that, at first, people had assumed that the likes of the internet and other modern social tech would be a good thing - empowering the previously voiceless and all that - and momentarily reminding me of the optimism around the Summer of Love before it petered out into drug overdoses in squalid squats. But he reckons that this upbeat assumption, like the flower children's, is worth revisiting, and that there are real risks of technologies like facial recognition becoming oppressive - and effective - methods of totalitarian control. The poster child in his presentation was China's proposed 'social credit' rating system, which looks to bundle all of a person's activity into a composite measure of whether they are a good citizen in their everyday life and whether they are going with the Communist Party flow, with potentially unpleasant personal consequences if they aren't (for example though restrictions on their access to credit, employment, education or travel).

In the discussion afterwards, there were quite a few questions (including mine) about whether the social credit rating would be as effective as feared. My thinking had been that people would see through the government's rating as a political device - I was reminded of the crack in the Soviet Union that "we pretend to work and they pretend to pay us" - and would actually judge you by (for example) your buyer or seller scores on whatever is the Chinese equivalent of eBay. 

Tirole, as we should have expected, had thought of that, and his answer was that an authority minded to go down the social credit rating route would deal to the private scoring systems to stop them being used in exactly that way. On p4 of the paper he says, "The state must eliminate competition from independent, privately-provided social ratings. Because economic agents are interested in the social reliability of their partners, but not in whether these partners’ tastes fit with the government’s views, private platforms would expunge any information about political views from their ratings. This competition would lead to de facto unbundling, with no-one paying attention to the government’s social score". 

He also said that people couldn't just ignore the ratings when they carried real penalties, and he pointed to an insidious feature of the rating system, 'guilt by association'. You might well want to allow the poorly rated dissident to buy a business class air ticket, but your own rating will suffer if you do. It reminded me that we've been here before: how many people kept buying from Jewish shops in 1934, when the SS were taking notes? Tirole also raised some interesting historical parallels, quoting Aldous Huxley's letter to George Orwell in 1949, where Huxley felt that oppressive governments would find it easier to go for lower cost routes than running gulags. Orwell was right in the shorter-term, but Huxley might be closer today: "the recent developments fit well with his overall vision" (p6).

What should be done? "A key challenge for our digital society will be to come up with principle-based policy frameworks that discipline governments and private platforms in their integration and disclosure of data about individuals" (pp35-6). But as he also says with very considerable understatement, "The exact contours of such disciplined principles are still to be identified", particularly (I'd add) because we also want to keep sight of the very large benefits the new platforms have brought.  Tirole argued for the desirability of keeping divisive issues out of the databases and aiming to "monitor platforms' foray into political coverage unless platform regulation is performed by one or several entirely independent agencies".