Modi’s Currency Ban Survey App is Unfairly Built to Succeed

What do you do when there is no option to disagree with a survey question set by your Prime Minister?

File picture: Narendra Modi greets his supporters after addressing a public meeting in Vadodara on 16 May 2014 (Photo: Reuters)

A fortnight after Prime Minister Narendra Modi made a sudden announcement about the demonetisation of Rs 500 and Rs 1000 notes, he unreservedly tweeted out to fellow Indians asking for their feedback.

He was referring to a survey on the Narendra Modi App.

Surveying 101

The survey consists of 10 questions to assess the mood of the people on various aspects of the note ban, with the tenth question asking for feedback and suggestions through a text input field.

Question 1of the survey. (Photo: The Quint)

Question 2 of the survey. (Photo: The Quint)

Question 3 of the survey. (Photo: The Quint) 

Question 4 of the survey. (Photo: The Quint) 

Question 5 of the survey. (Photo: The Quint)

Question 6 of the survey. (Photo: The Quint)

Question 7 of the survey. (Photo: The Quint)

Question 8 of the survey. (Photo: The Quint) 

Question 9 of the survey. (Photo: The Quint)

However, the essential purpose of an opinion poll was entirely lost with leading, biased questions, and sometimes even no option to say one disagreed.

Poorly designed surveys usually lead to biased or ill-informed decisions. In the case of the demonetisation survey on the NM App, I do not believe that Mr Modi or anyone trained in marketing research has designed this survey. Rather, it looks like the work of an intern unfamiliar with survey design.

Prithwiraj Mukherjee, Assistant Professor of Marketing, IIM Bangalore

Question one of the survey. (Photo: The Quint)

This question passes the expert’s test as a “yes” or “no” question.

Question two of the survey. (Photo: The Quint)

This asks two questions simultaneously, and is a leading question as well, because it makes no attempt to find out if the respondent thinks that black money or corruption are evil.  

Prithwiraj Mukherjee

Question three of the survey. (Photo: The Quint) 

What moves? There are multiple moves the government may have proposed or executed independent of one another. 

Prithwiraj Mukherjee

An interactive tool asks you to rate the moves for an answer between +1: Bad Experience; +2: Could Improve; +3: It's Okay; +4: Nice; +5: Brilliant. What is the difference between “it’s okay” and “could improve”? Why is “bad” a plus one?

Question four of the survey is vague in the same ways. Voters have to select from “very poor”, “poor”, “don’t know”, “good” and “very good”. (Photo: The Quint)

A slightly better way would have been to list down the prominent “moves” and “efforts” separately, providing information about each in brief. The ideal way to ask both these questions would have been to give the voters a text field to input their feedback and opinion.

Question five of the survey. (Photo: The Quint)

This, too, would serve its purpose best if it were an open-ended text answer instead of a rating chart from “bad” to “excellent.”

People collecting money from a mobile ATM at Pragati Maidan in New Delhi, 14 November, 2016. (Photo: PTI)

Question six of the survey. (Photo: The Quint)

An instance of a “triple-barrelled” question, the voter in this case is forced to answer three questions with one opinion.

The options are “immediate impact”, “medium-long term impact”, “minimal impact” or “don’t know”. In this case, even the framing of the answers is unfair, with no option to say that the move will not have any impact.

Question seven of the survey. The options following this statement are “completely agree”, “partially agree” and “can’t say”. (Photo: The Quint)

Multiple questions and the use of the assertive “will” to induce opinion where there may be none aside, there is no option to disagree. For a question that has been so widely debated in the media and political spheres over the last few weeks, no space to dissent in response to this assertion is a big red flag.

Question eight of the survey. (Photo: The Quint) 

First of all “our efforts to curb...” is severely leading. It is begging for a “not at all response.” Also, it is not established if an inconvenience was felt, and the extent of that inconvenience. Also note the framing of the options: “Not at all” suggests vehement support of the initiative while “yes” is neutral in opposition. Also, there may be people who were somewhat inconvenienced, but did not feel it was worth it, as well as other options. [It is an] extremely poor question.

Prithwiraj Mukherjee

Question nine of the survey (Photo: The Quint) 

Perhaps the most outlandish query of the survey, this back-handed question seems to be aimed at his political opponents and those who have spoken up against the implementation of the move. A “yes” or “no” question, it serves little purpose on gaining mass opinion of the currency ban and seeks more like a rallying call.

But the biggest flaw of this feedback session is its unfair sample size or reach, considering how only 17% of India’s population has access to smartphones as of February 2016.

It is then no wonder that the results of this “historic” survey after just Day One look like this:

Also Read: Is PM Modi’s NaMo App BJP’s ‘Surgical Strike’ On User Privacy?