A fortnight after Prime Minister Narendra Modi made a sudden announcement about the demonetisation of Rs 500 and Rs 1000 notes, he unreservedly tweeted out to fellow Indians asking for their feedback.
He was referring to a survey on the Narendra Modi App.
Surveying 101
The survey consists of 10 questions to assess the mood of the people on various aspects of the note ban, with the tenth question asking for feedback and suggestions through a text input field.
.png?auto=format%2Ccompress&fmt=webp&width=720)
.png?auto=format%2Ccompress&fmt=webp&width=720)
.png?auto=format%2Ccompress&fmt=webp&width=720)
.png?auto=format%2Ccompress&fmt=webp&width=720)
.png?auto=format%2Ccompress&fmt=webp&width=720)
.png?auto=format%2Ccompress&fmt=webp&width=720)
.png?auto=format%2Ccompress&fmt=webp&width=720)
.png?auto=format%2Ccompress&fmt=webp&width=720)
However, the essential purpose of an opinion poll was entirely lost with leading, biased questions, and sometimes even no option to say one disagreed.
Poorly designed surveys usually lead to biased or ill-informed decisions. In the case of the demonetisation survey on the NM App, I do not believe that Mr Modi or anyone trained in marketing research has designed this survey. Rather, it looks like the work of an intern unfamiliar with survey design.Prithwiraj Mukherjee, Assistant Professor of Marketing, IIM Bangalore
This question passes the expert’s test as a “yes” or “no” question.
This asks two questions simultaneously, and is a leading question as well, because it makes no attempt to find out if the respondent thinks that black money or corruption are evil.Prithwiraj Mukherjee
What moves? There are multiple moves the government may have proposed or executed independent of one another.Prithwiraj Mukherjee
An interactive tool asks you to rate the moves for an answer between +1: Bad Experience; +2: Could Improve; +3: It's Okay; +4: Nice; +5: Brilliant. What is the difference between “it’s okay” and “could improve”? Why is “bad” a plus one?
A slightly better way would have been to list down the prominent “moves” and “efforts” separately, providing information about each in brief. The ideal way to ask both these questions would have been to give the voters a text field to input their feedback and opinion.
This, too, would serve its purpose best if it were an open-ended text answer instead of a rating chart from “bad” to “excellent.”
An instance of a “triple-barrelled” question, the voter in this case is forced to answer three questions with one opinion.
The options are “immediate impact”, “medium-long term impact”, “minimal impact” or “don’t know”. In this case, even the framing of the answers is unfair, with no option to say that the move will not have any impact.
Multiple questions and the use of the assertive “will” to induce opinion where there may be none aside, there is no option to disagree. For a question that has been so widely debated in the media and political spheres over the last few weeks, no space to dissent in response to this assertion is a big red flag.
First of all “our efforts to curb...” is severely leading. It is begging for a “not at all response.” Also, it is not established if an inconvenience was felt, and the extent of that inconvenience. Also note the framing of the options: “Not at all” suggests vehement support of the initiative while “yes” is neutral in opposition. Also, there may be people who were somewhat inconvenienced, but did not feel it was worth it, as well as other options. [It is an] extremely poor question.Prithwiraj Mukherjee
Perhaps the most outlandish query of the survey, this back-handed question seems to be aimed at his political opponents and those who have spoken up against the implementation of the move. A “yes” or “no” question, it serves little purpose on gaining mass opinion of the currency ban and seeks more like a rallying call.
But the biggest flaw of this feedback session is its unfair sample size or reach, considering how only 17% of India’s population has access to smartphones as of February 2016.
It is then no wonder that the results of this “historic” survey after just Day One look like this:
(At The Quint, we question everything. Play an active role in shaping our journalism by becoming a member today.)