TVO Original Margin Of Error – LONG SUMMARY

Download Synopsis

It’s September 11, 2019. The campaign has started and the Canadian federal election is just 6 short weeks away. According to the mainstream news, the national polls suggest the Liberals and the Conservatives are in a tight race. New leaders for the NDP and Bloc could surprise, as well as a resurgent Green Party. Neither of the main parties is projected to win a majority, but right now the Liberals have a small advantage in the ridings count: 147 to 139. 170 ridings is the threshold for a majority win. But scandals within the incumbent Liberal government and pressing environmental issues are shaking up conventional expectations.

But can we believe the polls? After all, in the recent past, traditional pollsters misread several provincial elections, not to mention the upset wins by Donald Trump and the Brexit “Leave” side in 2016.

TVO Original Margin of Error takes a deeper look at a new artificial intelligence method that could shake up the landscape of traditional polling. The documentary asks, if traditional polling methods have made a few embarrassing misreads of recent elections, and are also struggling to solicit responses from would-be voters, could the explosion of online data offer a new way forward? Can social media and a new methodology powered with AI come together to predict public opinion more accurately?

Enter Polly.

Polly is the name of an artificial intelligence algorithm (AI) that analyzes public opinion with an eye towards predicting elections and other human behaviour. According to Erin Kelly, CEO of Advanced Symbolics (ASI), the AI behind Polly correctly predicted Trump’s 2016 presidential win, the successful Brexit “Leave” campaign, and was almost perfect in its tally of the riding count of the most recent Ontario provincial election in 2018, among others.

Part of Kelly’s explanation for the accuracy of the AI method is that, unlike traditional pollsters, Polly doesn’t ask questions of would-be voters directly.

Traditional polling has some fundamental limitations: it is restricted to using landlines, the representative sample is still very small and, people often qualify their responses to direct questions due to embarrassment or in an effort to please the questioner. The perfect example of this can be found in the significant number of suburban white women in the US who polled in 2016 as supporting Hillary Clinton, but actually voted for Trump.

Unlike traditional pollsters, who didn’t pick up on the Trump and Brexit wins until it was too late, Kelly contends that Polly is more accurate because she samples a much larger pool of opinions in a more indirect yet more accurate manner. She maintains that people’s actions and their peer group relationships and interactions are often more illustrative of influences on their choices than how they respond to a questionnaire over their telephones.

Kenton White, ASI’s co-founder and technical lead explains how he leverages social media activity to build and maintain a scientifically representative sample of upwards of 270,000 (rather than the typical 1500 to 2000 respondents in a traditional poll.)  White explains how Polly differentiates social media users to establish truly randomized samples of unconnected users by “crawling” through social media like Facebook and Twitter, to devise a geographically representative and diverse set of people that Polly can track repeatedly over time, back in time, and in real time.

White, who holds a Ph.D. in physics, draws on his background to explain how data can be used in aggregate to predict behaviour patterns in the future. Measurements are key in physics. But so is finding the data source that provides the best signal for predictions. Among public social media platforms, White uses Twitter data because it provides the best indicator of election outcomes. He explains how Polly develops correlations between Twitter user “engagement” on a topic over time and voter support, which enables Polly to predict the most probable outcomes in each federal riding across Canada.

Jumping into the fray of the campaign with four weeks to go, Margin of Error digs deeper into Polly’s track record and how she collects and analyzes data in advance of the election. Ultimately Margin of Error assesses the claims of Polly’s “accuracy” by exploring how well and reliably this AI method predicts the outcome of the October 21st 2019 federal election.

But in the prediction game, there will always be critics. Margin of Error also checks in with traditional pollsters (Ipsos CEO Darrell Bricker and Conservative Party strategist Hamish Marshall), statisticians (Professor Jeffrey Rosenthal), journalists (Chantal Hébert and Supriya Dwivedi), and social scientists (Professors Gillian Hadfield, Philip Mai, Fenwick McKelvey, and futurist Jesse Hirsh) who challenge the claims regarding Polly’s method, accuracy and the value of its social-media data source. These critics also warn that both AI and unregulated social media not only pose a threat to privacy, but ultimately to democracy itself.

When the election dusts settles, Margin of Error reviews the quality of Polly’s predictions and those of traditional pollsters to come to conclusions about the accuracy, value and future of predicting elections, and understanding public opinion using artificial intelligence.


Margin Of Error – SHORT SUMMARY

In the face of a century of traditional polling to predict public opinion, there is a shakeup afoot in the prediction game. With the 2019 Canadian federal election as its backdrop, Margin of Error examines how a small Ottawa startup uses artificial intelligence and public social-media data to forecast voter behaviour. But the promise of new technology also comes with questions about its accuracy, the threat to citizens’ privacy and our democracy itself.