Yashwant Deshmukh on the challenges of polling in India and the forthcoming Indian Polling Council

Yashwant Deshmukh is the Founder Editor of CVoter, and a Communications Professional with the working experience of Journalist, Pollster, Evaluation Expert, International Observer and TV News Anchor rolled into one. He founded Team CVoter in 1993, when he was still studying in IIMC. After receiving the UNI award for best research dissertation and for topping the 1993 batch across all streams, his company CVoter was hired by the premier news agency UNI to take care of on-line real time election analysis. Team CVoter continued to grow, and is now one of the largest media and stakeholder research agencies in Asia with expertise in Public Opinion Research & Election Studies. Today more than 120 team members work for CVoter across their 24×7 offices in Washington DC, Dubai and New Delhi.

CVoter is also one of my research affiliates for this project. I have been regularly visiting their Delhi office (which is technically in Noida) to learn about how they do their research. I visited Patna to observe their exit polls during the third phase of the Bihar elections.

Now based in Dubai, Yashwant was in Delhi last week for the Bihar election results. Last Friday, I met with him at Janpath Hotel near Connaught Place to talk about his experience doing public opinion research in India, the challenges he has encountered, and a forthcoming association for Indian pollsters called the Indian Polling Council. The transcript of the conversation below has been lightly edited for length and clarity.

On polling in India

Sam Solomon: Please tell me about your background conducting public opinion research in India. How did you get into this field?

Yashwant Deshmukh: I am a major in journalism, actually. I did my post-graduation in journalism from IIMC. I am a journalist. That is my primary identity and my training.

But yes, I have always been interested in elections, ever since I was a kid. Had a personal history of roughing up with [the] Emergency. I wrote a piece on Huffington Post about my experiences during the Emergency. You might like to pull it out and read it once so that you can get the context of how as a kid I observed the Emergency and I got into the ‘77 elections which was a watershed election in Indian electoral history. Elections were always like something that were good for the people, good for the country, something which und[id] the wrong and [did] the right thing. So that’s the kind of personal background that I was raised on.

And then when I became a journalist, I was — I did my research dissertation on elections only, in IIMC — and then I wanted to join some polling organization as a political analyst. So I didn’t get a job there. I got a job in the campus placement in the newspapers and the magazines. But I didn’t get a job with the polling companies. So when I didn’t get a job with the polling companies, I had to start on my own. (laughs)

SS: What polling companies were around at that time?

YD: At that point in time, there were no multinationals around. There were no multinationals around.

The biggest company at that point in time was ORG and Marg — they later got merged as ORG-Marg, later years. But yes, they were the people who did the fieldwork for the initial polls of Prannoy Roy, which were the path-breaking polls in India, in ‘80 and ‘84 elections, to be precise. ‘84 was the time when psephology got a wider acceptance of political polling, in a way, when Prannoy said that Congress was going to end up with more than 400 seats. He got it right. So they were the people who did the fieldwork. Prannoy and Ashok Lahiri, they analyzed those elections. That was their job. They designed the poll and they analyzed the poll. Prior to that, in India the polling was only limited to what Eric de Costa was doing in IIPO [Indian Institute of Public Opinion], and there was no history to political polling as such.

I think that when I was graduating in the early 90s, that was a very critical and interesting time in Indian political life otherwise. It was the peak of what we call the Ayodhya controversy or movement. The BJP was emerging. The Congress was having problems. In [the] ‘91 elections, unfortunately, Rajiv Gandhi got assassinated. It was a very transitional time when PV Narasimha Rao became the Prime Minister. It was the time when [the] economy was opened up. So altogether [the] early 90s were politically speaking very charged, after [the] ‘89 elections, you know. V.P. Singh, Mandal Commission, Ayodhya, opening up the economy. I think there was too much to consume. It was a thorough transition and transformation that was going on.

And it was a time when I was graduating as well and being a journalist, it was interesting for me to see and map the public opinion on different things. So that’s where I started. I floated CVoter when I was studying at IMC in 1993. I got lucky because I topped the ‘93 batch at IMC, and I was awarded a UNI award for my research presentation on elections. There are two primary news agencies in India, PTI and UNI. And UNI the next year actually offered me to analyze the elections for them as the live counting was done. So that was our first major break. Probably my research dissertation and getting an award on that luckily helped me to get into our first assignment.

And after that, because of UNI, the fIeld was going to more than seven hundred newspapers across the world. Slowly and slowly, we started working for other media houses. The Week was the first one to give us a commission on an opinion poll.

SS: When you say “we,” you’re talking about CVoter, right?

YD: Yeah. So that’s where it started. And one after the other, it kept on happening and it was happening at a very frantic pace because we had ‘96 parliamentary elections, then ‘98 parliamentary elections, then ‘99. So within the gap of three years we had three parliamentary elections in India.

And it was also the same time when satellite TV opened up in India. The first 24/7 news channel opened up in India; that was Zee News. Prior to that, the news operations were limited only to another satellite network called Jain News. Incidentally, Professor Yogendra Yadav also did his first election on TV with Jain News. That was the only private satellite network which was doing news in India at that point in time. We are talking about 1996. Then, Zee was the first 24/7 news network. Zee hired me to do the polls. After that, Aaj Tak came in, then Star, and so on.

Basically, being a journalist, being a broadcast analyst, being a TV guy and a media guy, it kind of helped me to be on the screen because I could not just analyze but also interpret the data. It kind of helped me stabilize CVoter in terms of the operations and other things.

This had been going on, then after that we went beyond political research to socioeconomic research. We entered into disaster mitigation research. Conflict resolution research. That’s where the majority of our work right now is going on. Into the international work as well.

I was always grateful to Robert Worcestor, the founder of MORI, because he allowed me way back in the ‘90s to observe the MORI exit polls in the UK. So I learned that part from there and I applied that as much as possible in India. If I have to pick my godfather– I guess Bob has been to many other pollsters across the globe. I’m happy that I learned from him.

It was also the time when the multinationals started coming into India. AC-Nielsen came in. TNS came in. Ipsos came in. Gallup came in. They were following their clients in India, because the economy was opening up. And it was also a transitional time when the bigger Indian companies were being taken over by the major groups. So all of a sudden we realized that now we are the only Indian company working as such in the private sector with that kind of footprint and operation, volume and scale.

We entered the US five years back. We did the last presidential [election] in the US. Rated very well in our very first operation over there. We did South African elections, bigtime. We did the backward integration for the UK elections as well but we did not take the CVoter brand over there. We were working for some other group over there. Hopefully in the next election, we will enter the UK market as well.

SS: During all this time that you’ve working in polling, how has the polling industry in India changed?

YD: Difficult question. So much has changed and still it feels like nothing has changed.

What has changed is the technology of it. Technology has changed a lot.

We have moved the majority of our operation from face-to-face to CATI [computer-assisted telephone interviewing] now because of the simple fact that India is right now the biggest mobile market in the world. Even though 85% mobile penetration is there now. Actually, 85% is of the total population. If you talk about the adult population, it is at saturation, more than 100%. Statistically speaking. We know that about 20% of Indians don’t really have mobile phone. But yes. It’s huge. And because it’s calling party pays, we started CATI operations seven years back. And now it’s getting standardized. I believe that’s the future. That’s why we invested in that.

Online research in India is still far out because only 15% of Indians technically speaking have the Internet facility, but only 5% of them are actually [the Internet] for social media or other purposes like that. It is just that the volumes are too big. It’s not representative as such. It will still take a lot of time to come. CATI is getting there. So technologically speaking, there ha[s] been a lot of improvement.

From the quality [perspective], unfortunately I actually see a downswing because this place was earlier being observed by a handful of key players. They were open to skepticism and criticism. But all of a sudden with the new electronic media and everything, we see many players who are opaque and the methodologies are not being discussed [in the] way they should have been discussed.

The polling industry as such is getting to a stage where less interaction and less education of the media and the media consumers at large is proving to be detrimental for the health of the research industry as such. The concept of polling has arrived. Many people work very hard for that: Eric de Costa in the initial years, Prannoy [Roy], Dhorab [Sopariwala]. They all worked really hard. Professor Yogendra Yadav. He did his part very well. But it’s trivialized now in the media circuit. The media and journalists at large don’t understand. They don’t understand what to expect from the polls, what are the limitations of the polls. And that is why their expectations from the polls is wrong. And that is where it gets trivialized. And when it gets trivialized at the editorial level, it gets automatically trivialized at the readership level. So that is a big challenge that needs to be [addressed].

But technologically speaking, a lot of improvement has happened. And from that perspective, it is getting better, I would say. It is getting better with time. It is getting better with each and every assignment.

But the expectation from the research industry is wrong. The limitations, the plus points, they need to be more discussed and understood more properly.

SS: You said that it feels like everything has changed and yet not much has change. What do you feel hasn’t changed about the polling industry since you got into it?

YD: What hasn’t changed is the pathetic understanding of the polls by the media. What hasn’t changed is the absolute lack of data awareness among the journalists in this country. What hasn’t changed is the knack of sensationalism in the media scene while they are reporting the polls.

What hasn’t changed is the lack of continuity of the data gathering and analytics, the serious component of the research of the changing trends on on political sectors and indicators. What hasn’t changed is [the] absolute absence of philanthropic funds which should go into the research industry. In the West, right from the start and even today, almost 90% of the socioeconomic research is being funded by philanthropic funds. In India, it is a big zero. So the only funding that the polling actually gets in India is either government or the media. And they have their own limitations on doing trackers and serious socioeconomic research. So that hasn’t changed. There was no funding from [philanthropic funds] back then, there is no funding in it today.

So what hasn’t changed is the lack of serious polling on socioeconomic issues. What hasn’t changed is the lack of continuous polling on serious socioeconomic issues. These are the things which have not changed.

SS: When you talk about the lack of quality coverage and awareness of how to analyze data at the media level, could you talk about the specifically with regards to Bihar? We were talking earlier about the headline today for the exit polls, how it just shows the midpoint, it doesn’t show the ranges [of seat projections]. How do you think the media coverage for Bihar has been as compared with other elections?

YD: Basically, it underline[s] all the things which I just mentioned. If the understanding of the polls and the understanding of the research would have been better, it would have been easier for the media to say that these elections are close to call. If something is close to call, why [does] it ha[ve] to be a matter of ego than one needs to take a call? When you take a call that it is close to call, that itself is a call that it is close to call.

The big problem in these Bihar elections has been as usual the media’s inability to accept that this is close to call. And them forcing the pollsters to take a call. Because that’s what we are getting paid for. All the pollsters are getting paid to get a call. But I think the change is that now we are also forcing in the ranges which are overlapping. We are forcing them to say personally, even if it has to be on a personal level like me standing in front of the camera, saying, “Listen. This is close.” Statistically speaking, it’s difficult to call. My ego is not getting bruised or hurt by saying that I am unable to call this. No big deal.

In fact, more than Bihar, I would like you to read the one-year report card of the Modi government programme. It was a brilliant programme on India TV and Times Now where it was a Mood of the Nation poll. Probably the first time where I requested [of] them, “Please don’t force me to give you the seat share and vote share.” Because that’s not really the thing you should be analyzing in a weekly report card.

You are aware that we do the weekly tracker, which is the only such vehicle in India. Nobody does that. It was nice to see week-on-week Modi’s popularity going up or going down and analyzing that. It was fascinating to see that right after than 9 lakh suit controversy Modi’s popularity went down. As they say, pollsters are the chroniclers of history. It was important to see the last fifty-two weeks on different scales of popularity, the satisfaction, the issues, which way the country is going. If I would have given the vote and seat share projection, then nobody would have talked about those issues. “Ah! If the election are held today, Modi would have faced problems or Modi would be flipping more [seats]…” That’s trivializing.

So the big change is that I requested them and somehow I got lucky. Both my editors, Mr. Sharma and Arnab, they actually saw the merit in that thing. And they did a brilliant program which was [a] first in Indian television, that you are having a political program on a one-year report card without showing the vote and seat projection. You never have any state of the nation, mood of the nation [program] without vote and seat [projections] in this country. So that is the change which I am trying to push. I am lucky that I got a couple of good friends in media who are trying to see the merit in it. I hope that it continues.

But somewhere we have to put the foot down, that this is what the limitation is. 90% of the research material that goes into serious socioeconomic research and results go unreported in the media. The only 10% of the political aspect of it gets 100% out-of-proportion coverage. That needs to be changed.

And that is what we need to learn from the West. There are polls year round happening day in and day out on thousands of issues which are mapping the health of the society, the socioeconomic patterns, the issues, the trends, which way the society is heading. Not which way the politics is heading. We need to change that.

To change that, a lot of such surveys [need] to be done on a regular and continuous basis. To do that, a lot of funding is required. Not just from the media, but also from the corporate sector. To map those sentiments, to map the socioeconomic trends of the society, that needs to creep in. Somewhere it has to creep in.

On the challenges of polling in India

SS: What are some of the challenges that you’ve encountered during your career in terms of accurately measuring public opinion in India?

YD: That has to do a lot with two components. One is the training of the researchers. Two is the response rate in different sections of the societies.

Training of the researchers, that’s something which is directly related with the first part of my answer. In the absence of regular polls, even if you train a few researchers here and there, you are likely to lose them. Because if I am going back to Bihar after five years, I am unlikely to get back my trained researchers which I trained five years back. And I am unlikely to get the researcher which I just trained now five years from now unless I have a continuous assignment in Bihar and keep on hiring those people and keep on honing their expertise of interviewing and constant improvement. The researchers is a big issue and that primarily because of the lack of regular research in the field.

The second part would be the response rate of the different categories. In India, you know it’s a very heterogeneous society. Largely speaking, all of us have been getting lesser response [rates] from the minorities, from the Dalit communities, from the females. That’s a historic thing.

That is something which cannot change overnight. We tried to come across these things by say, for example, for minority respondents, trying to field same-faith researchers into those localities to conduct the interviews. That helps [in] getting the response rate better. Fielding more female researchers to talk to more females. That helps. But then, fielding the female researchers increases the cost of the operations. It’s a tricky thing to understand that–

SS: How does it increase the cost?

YD: It increases because of the local sensibilities. Say, for example, if I have to conduct research 100 miles from here, or 200 miles from here, or 500 miles from here into some interior village, and I have to send a team of four researchers over there, ideally I should be sending two females and two males who can move together. If the selected respondent is a female, let the female interviewer take the interview. If the selected respondent is a male, let the male interviewer [take the interview]. Because in the rural areas, it is absolutely improbable for an unknown male to conduct an interview of the female of the household. It’s not part of the culture. It’s not done. Anybody who says that it is done does not understand India or South Asia, to be precise. It’s difficult. It gets easier to get the female response if the female interviewers are interviewing them.

But then if you have to send two researchers in the field, the cost goes up. If you are staying there overnight, if there are two males, you can still pick up one room for them to share. But if there is one male and one female, you have to pick up two rooms. If there is no boarding facility in that area, then you have to ensure that they come back to the district headquarters or their main place before it gets too dark. So your fielding operation times get lower because you field for the security of your female colleagues. And for obvious reasons.

All in all, this is a practical problem for all the polling companies. Barring the metros, barring the areas which are urban, barring the areas where they can hire female researchers who can conduct the interviews and come back on their own safe and sound, it is unlikely to send female researchers to the unknown interior areas without knowing the safety and security [is] in place. That’s the cultural context of it.

And when the clients are not willing to pay extra for covering up those additional costs, then the chances are the [researchers] are, to work within the given budget, more likely to send a male-only team into the field. And those male-only teams are likely to return with lesser female sample because of the non-response.

So it’s a vicious circle. One thing adds to the other. But you can always weight [after the fact]. But in the field operations, these are the things which one has to deal with.

SS: What are some of the challenges you’ve faced in terms of effectively communicating poll results to a client, to a media outlet, or to the public?

YD: To the media outlets, as I said, they always need a number, especially the seat share in an election poll or a political poll. Not everybody understands very kindly that the science of surveys stops at the vote share calculation and conversion of votes into seats is not really part of surveys.

Everybody has got their own algorithm or mathematical formula to do that conversion. But that’s not foolproof. That’s not proven. That’s a work in progress always. Even in the oldest of the First World democracies. What we have seen in the UK elections. Even they could not convert it properly. They could not sense that the Tories would be getting a complete majority in the UK. And they have been doing the polls for — I don’t know — seventy or eighty years now.

The media doesn’t understand. We have been saying this. Everybody. Yogendra has been saying this, I have been saying this, everybody who has been appearing on TV has been saying this for so long. But still, the fascination of media with the seat share is amazing. They just refuse to understand. And the most classical [example] is that some of the media guys are so ignorant that when you talk about the margin of error, they put the margin of error on the seat share calculation. So that is the level of understanding the data. How many media companies have a research editor? Barring one or two, I don’t know if the newspapers or TV stations have research editors in them who understand data. They don’t have [them]. That’s a problem. It’s very difficult.

Yes, classically speaking, the normal viewers, the people actually understand it better. It’s very funny. When I interact within the social media with the people, when I tweet, they understand it better than most of the journalists around. So it’s difficult to make them understand.

Somewhere probably we have to draw the line, that okay, you see we can at worst try to come up with the vote share calculation. This is the seat share calculation. But we can’t hone it. It’s not scientifically proven. It’s only an assertion. It’s only a calculation. It’s only our best idea. But it is not the thing. It’s not the thing.

SS: CVoter is an international firm. You mentioned earlier that you’ve done polling in the United States, South Africa, and the United Kingdom. In a previous conversation that you and I had, you said that nowhere is more difficult to survey than here in India. Why is that?

YD: The sheer heterogeneity of this country. If you look into our operation that you have seen already, look into the kind of languages that we cover, the kind of demographics that we cover, you will understand that polling in India week after week is kind of [like] running a Eurobarometer week after week. It’s as simple as that. If you can understand the difficulties in running the Eurobarometer, you can understand the difficulty of running the Indian operations as well.

And then the sheer length and width of this country. The geography. The topography. How difficult it is to reach the interiors. That is why we took a conscious decision to going to CATI on the mobile, because that is something that shows us that we are reaching the remotest of the places in random probability sample. And we are speaking twelve different languages. And we are doing [the] audio recording of each and every interview. And doing the quality auditing of each and every interview. That kind of quality is possible only in CATI. That is why we took a conscious decision that [the] future is that. We cannot leave the field operations to happen in that way, doing only ten or twenty percent of back-checks in the face-to-face surveys. Everybody understands the handicap of doing that. Everybody understands that the where and why it can go wrong.

For example, in the West, the idea of human labor to be calculated in the per-hour approach. It’s a very objective thing. That you work for eight hours and you will be paid on a per-hour [basis], or how many samples you cover. So the per-sample based approach to the researchers, it is so West[ern].

In India, it doesn’t work like that. Or this part of the world, or even in Africa, it doesn’t work for a simple reason. I send you to a certain village. Thou shall go there and thou shall interview people and come back, and per-interview I am going to pay you this much. Now, you in the best possible way, picking up the autorickshaw, going to the bus station, taking the bus to that village, reaching that village, figuring out the random probability sample, and getting to that person, and the person refuses. And then you have to also worry about, “OK, at four o’clock is the last bus. I have to come back.” So what is happening is that you have spent an entire day doing the best possible work and still you might end up [with] two or three interviews. So if it is not economically viable for the researcher, what do you expect the researchers to do? To give you a complete random probability sample? No. Hell no. They are likely to do a cluster sample and come back and report to you as a random probability sample. Because they are getting paid on a per-sample basis.

This is why about twelve years back, we changed — in our organization, in our system — the per-sample system is wrong. Because the researcher is going there, doing the job. He or she may not get a single interview the entire day. And that’s not his or her fault. So the payment has to be on the complete day basis. That’s where we changed it in our organization. So today in CVoter, it’s been almost for the last twelve odd years, we tell the researchers, “You are being sent to this place. Go and do it. At the end of the day, if you do one interview or zero interviews or ten, you are going to get this much of payment which you deserve [for] doing the hard work. Plus, please make sure, even if you are doing one interview, that one interview should be scientifically, properly done and reported.”

But then this methodology or this way of functioning increases the cost of the operation. And when the quality goes up, the cost goes up. And if the clients are unwilling to pay for that cost, that is where the quality degradation happens. Now, understanding this thing is something which is important. The client’s understanding of this is more important for them to know that this is difficult, this is how it has to be done. So all these things, they need to be taken care of. New, better methodologies and technologies are to be adopted to improvise this further. And the clients need to understand this as well: what are the limitations of it, how it is to be done, and how the best could be achieved from this.

SS: What are some of the tradeoffs that you have to consider when you’re deciding whether to field a survey in person, over the phone, or over the Internet?

YD: Well, Internet is a no-go. First of all, it’s not really representative. For a simple reason that only 5% of the people are actually active on the Internet. The 15% that is floating around is for the people who have the data pack or the data plan, but not everybody who has the data plan is active on the Internet. So first of all, it’s a skewed number. It’s not representative.

The second thing is that even in the West the online opt-in is scientifically not a representative sample. That’s a fact. The so-called river sample is also again not a random probability sample. I mean, it’s a random probability sample among those who are visiting that part and willing to comment on it. So even a river sample is not exactly a simple random probability sample. In India, you cannot even think of doing that simply because number one, it’s not representative, and number two, the incentives with opt-in panels are further going to degenerate in that direction. So as of the current moment, the online thing is not done.

We do a random online thing, when we are doing our CATI every week, we also keep on asking in the final question, “Do you use Internet?” and “Would you like to be part of our panel?” So that is something which is [a] much more randomly recruited panel. But even in that random[ly] recruited panel when we have tried to do the surveys, one reminder, two reminder, three reminders, people are not very keen to click and answer. And probably the third round when we call them from the CATI center, the more likely answer is, “Why are you pushing me to click and answer? Why don’t you simply ask me? I will answer you right now.” So it becomes more like — instead of online — it becomes WAPI, web-assisted personal interview. That means you are talking to them and you are punching the data on their behalf. The only good thing is that some clients understand that and they agree to it whenever they commission it, because we have the audio recordings as the proof of the interview. But still a long way to go on the Internet.

CATI is the future right now because Indians may not read and write but they can certainly talk. This is not my saying. This was something which was said by Dhirubhai Ambani when he was launching his mobile services in India. [In] India, a big number of people are not very educated. A big number are illiterates, to be precise. But they will certainly talk on the phone. Everybody has a mobile these days. So that helps. So CATI is the future, as far as I’m concerned.

SS: Have you found that certain populations — because of cultural factors, because of socioeconomic factors — are more or less likely to be selected and participate in a survey? You discussed this a little earlier.

YD: Yes. The more educated, the more well-to-do, urban, affluent, upper caste, males are more likely to respond. The less educated, females, minorities, lower castes–

SS: Religious minorities?

YD: When I say minorities, I am talking about Muslims especially.

SS: What about Sikhs or–

YD: Sikhs, I never had [problems]. Because Sikhs are, from the class distribution perspective, Sikhs are one of the wealthiest communities in India. Jains are one of the wealthiest communities in India.

So more educated, well off, urban. The response rate of Sikhs is fabulous. [The response rate of] Jains is fabulous. The response rate of Christians is perfectly fine. The response rate of Christians even in the tribals is perfectly fine because in all likelihood they get better education because of the Christian missionaries running the schools. They get better education, so their response rate is better in that way.

But the females at large — regardless of caste, creed, community, or whatever it is, religion — females at large the response rate is lower. The Muslims, the response rate is lower. Unless you are sending the same-faith research into their community to ask the questions.

In certain areas, the Dalit response rate is lower. When I say certain areas, it is directly proportional to how politically empowered the Dalits are in those areas. Before Mayawati happened, the Dalit response rate in UP was very low. But not anymore, because now Dalits in UP are now wearing their identities on their shoulders. So they are not really shy to say. Before Lalu happened in Bihar, the Dalit response rate was low. After Lalu happened in Bihar — one may call his rule as a misrule, or whatever — I have no problem in saying that Lalu Yadav was a game changer in Bihar for that downtrodden, that oppressed Dalit and other communities in Bihar to come up in their social aspirations and their sense of being powerful. So I don’t get a poor response from the Dalits in Bihar. It doesn’t happen anymore.

Yes, in Rajasthan, I still get poor response [rates from Dalits]. In Punjab, I get poor response rates from the Dalits, including the Dalit Sikhs. In Madhya Pradesh, I get poor response from Dalits. So the states where the Dalit political identity is still weak, the political identity being weak is directly proportional to their response rate.

I don’t know how to phrase it correctly. I may sound very unscientific, but I am talking only from my personal experience. The stronger the political identity, the better the response rate of the community is supposed to be.

SS: Do you see this with Muslims as well across states?

YD: Yes. It is there. But with the Muslims, it’s not that they don’t wish to talk. It’s not that. Whenever we send the same-faith researchers, the response rate dramatically increases. They don’t say much, their response rate is down, when there is a sense of polarization. Then they go quiet. But in a normal situation, they speak well. They speak as good as Hindus actually when there is no fear of polarization around.

But yes, it is next to difficult to interview the Muslim females. It is almost impossible because of the cultural context. We must respect and understand that. And it’s not just related to India. I have worked in Indonesia. I have worked in other South Asian countries. I am right now working in the Middle East, bigtime. And I know it’s impossible for a Muslim female to talk to a male stranger. It’s the cultural context of the situation. The same is applicable. And then what happens is that in the puritan way, as many of the methodologists call it, even if she agrees to be interviewed, the chances are that she has to be interviewed in the presence of male members of the family. And when you are asking about the questions that are uncomfortable, can you imagine a Muslim female being interviewed on the uses of contraceptives by a male stranger in the presence of her husband? Can you imagine that? And can you imagine that the answer flowed freely without interruptions or without any things from her husband?

And that applies equally well on the Hindu females. Can you imagine a Hindu stranger talking to a Hindu female in a household about the contraceptives she is using in the presence of her husband or father? I’m sorry, if somebody says yes, he is lying. I’m sorry to be so harsh about using the word “lying” because it’s impossible in this cultural context. There are cultural sensibilities which are to be taken care of, which are to be honored.

And when you honor those cultural sensibilities, that doesn’t mean you are compromising the quality of the data. It only means you are trying to figure a way out on how to get the best possible data. So the answer is that if you have something sensitive like uses of contraceptives, it is always better to send the same-faith female researcher who can interview alone and get the data.

And the same thing happens with the hard-to-reach populations. We did a fabulous survey when we were doing the census for sex workers in one of the districts in Bihar. It was wonderful research. How do you expect a sex worker, a victim of human trafficking, to reveal their socioeconomic problems to a complete stranger? What are the chances like? We know how difficult it is to interview the hard-to-reach populations in stressed conditions. That’s why we did a project over there where we actually trained a big number of children of sex workers who are minimum high school graduates. And we trained them as a researcher to go and conduct the interviews in their localities. Now they knew the localities. They were talking to their moms, their sisters, their neighbors. And they were more likely to get the data correctly. So we named that project as Project VASE, Victims As Social Evaluators. And it worked fabulously.

Even when we were doing our post-tsunami research in Aceh. Because in Aceh, there were conflict victims as well as tsunami victims. It was almost impossible for the normal researchers to go into the interiors, to visit those areas, and talk to the female multi-victims, as we say, those who were the tsunami victims as well as the conflict victims. There we trained a special team of ex-guerrila fighters of GAM [Free Aceh Movement] who had laid down their arms and surrendered and were looking for better jobs. We trained an entire team of them to conduct research on sexual harassment and other cases and other difficulties. Fabulous set of data. Because first of all, they were themselves the victims. They understood the sensitivity of the subject. And they were going to interview other victims. And they were coming up with data which was kind of unheard of. When you get an interview from somewhere in the interior of Aceh, where in the end of the form, the researcher has written that this lady was raped three times by the security forces. Now that’s an insight which no one can ever provide you. That’s an improvisation.

So Project VASE has been something which is very close to my heart. I have been working on that. I used that in Bihar for the sex workers survey. I used that methodology in Aceh. In Sri Lanka, in the northeast of Sri Lanka. It’s amazing. It’s amazing. That is something which I am going to focus more and more on, on how to get better responses from hard-to-reach populations, in special conditions. These are special populations.

But in routine conditions, yes, my solution in India is very simple and straightforward. Sending more female researchers gives you a better female response. Sending more same-faith researchers gives you a better response from Muslims. Sending more same-community researchers gives you a better response from the Dalits.

SS: Going back a bit to your discussion of Muslims, you said that at times when there’s a sense of polarization, the response rate decreases among Muslim respondents. Could you provide some examples of that?

YD: In Gujarat, we have always been getting lesser response post-Godhra [riots in 2002]. In fact, every time there is a Gujarat election we do a special sample in Godhra. That is something which is part of our study also.

Whenever there is a communal riot or something like that, [in the] post-riot scenario, the response rate of minorities actually goes down. Especially in the areas where they are even less spread, but really lesser in number. Probably the fear cycle just takes over. They don’t really wish to talk and be seen as something different. They want to assimilate and be safe. Their response rate goes better only when you are entering a completely ghettoized area where they are kind of feeling safer and the same-faith researchers are going. Then it goes up.

In a normal scenario, they respond as normally as the Hindu or other communities. This happens only when the political surveys are happening and if the atmosphere is charged on communal lines. Then only this thing happens. Otherwise, normally we get the normal response rates.

SS: Did you see that in UP after the Muzaffarnagar riots?

YD: Yes, that happened. That happened in UP after Muzaffarnagar.

SS: The last group I want to check with you is Scheduled Tribes. Do you see a different response rate among Scheduled Tribes?

YD: Yes, yes, yes. It is lesser among the Scheduled Tribes as well. But that lesser response rate is not because of something like they don’t wish to talk. Scheduled Tribes, there is no fear. They wish to talk. They can talk to you easily.

It depends on what exactly are you asking them. So if you are asking about something like– and actually, funnily enough, we do questionnaires like that from clients. “What is your view on Russian President Vladimir Putin? Are you favorable or unfavorable?” Try talking that to a tribal  in Gumla district of Jharkhand. I mean, come on! Come on. I sometimes really wish to ask those who frame those questions sitting in some plush university campus in America or Europe, “Man, go out of your office. Try interviewing ten people you come across outside your office and ask them how many of them know the name of Mr. Putin over there. And you are expecting that a person in a remote Gumla tribal village is going to answer that?” I mean, it is more contextual, more on the subject. So the issue of response among the tribals is not because they don’t answer. You talk about their problems, their day-to-day lives, the issues around them, they are more than happy to answer. It’s just that they are more likely to say in that kind of questionnaire, 99% of the questions will be, “Don’t Know / Can’t Say.” And when you say, “Don’t Know / Can’t Say”, how can we accept this interview where everything is “Don’t Know / Can’t Say?” “Don’t Know / Can’t Say” is information as well. They just don’t know Putin! What is the big deal? Hell. They might even know about Putin but they certainly won’t know anything about the head of the state of Brazil for sure.

How many in this room…? I mean, we are sitting in New Delhi. You go and talk to that table, “What do you think, favorable or unfavorable, of Dilma [Brazilian President Dilma Rousseff]?” The next thing they are going to say, “Ah, Dilmah. They make good tea in Sri Lanka.” (laughs) What are we talking about?

So the issue of non-response also kind of has to be seen in the right perspective. The issue of non-response or lesser response has directly to do with what exactly you are trying to ask.

SS: Do you see big differences in response rates across states?

YD: Yes. There is. There is. The more politically charged-up states actually get a better response rate. The more educated states also give better response rates. The response rate goes down typically in urban, affluent locations where the people just don’t– They are educated. They are upwardly mobile. They are wealthy. They know. But the response rate is going down. They are least interested in talking to you. So that’s a different kind of problem of response. That’s the other end of the socioeconomic spectrum. The affluent class, the upper middle classes, their response rate is going down because they don’t wish to answer. They don’t have the time. They are not interested. They are just too happy counting their money and living their life.

SS: When you said the more politically charged states, which states do you mean?

YD: Bihar, Tamil Nadu, UP, parts of Maharashtra. We are likely to get a good response rate.

People are very happy to talk. Indians are very talkative. They love to talk to you on any issue under the sun. And it’s not that they don’t wish to answer. The problem is that on CATI, for example, for us a ten-minute interview on CATI usually ends up being fifteen or sixteen minutes because people keep on talking. We don’t know how to cut them [off] because it is rude to cut them [off]. We cannot cut them [off]. “Sir, please don’t explain your answer. Just tell it to me yes or not.” No. They don’t tell you as yes or no every time. They give the context and they talk.

Probably they talk to us more because of our media identity. That is something I am agreeing to concede. Our branding is primarily of a media entity. People have seen so much of our name and face on TV and the newspapers that when we say we are calling from CVoter, we are doing a survey for India TV or Times Now, probably coming from the media background opens them up immediately. We also designed the questionnaires in a way which starts with the problems of their life, to open it up. The moment you talk to them about their problems, their lives, they are more likely to open up and talk to you. And then they talk like hell. They keep on talking.

SS: Have you found for election surveys that some states are easier to poll? That you have a better track record with some states than others?

YD: No, nothing of that sort. I believe that from the election polling perspective, the only easier polls are those when the anti-incumbency issue is huge and the feedback from the people are so crisp and clear that there is no confusion whatsoever in your mind that, okay, this government is going to lose. They are the easiest ones. Otherwise, I would not say any particular state is difficult or easier to poll. Every state is similarly as difficult and as easy to poll, but depending on that particular election, whether or not there is a clear cut winner or clear cut loser.

On transparency of pollsters, disclosure of methodology, and the forthcoming Indian Polling Council

SS: A number of the people I’ve spoken to, including researchers — and this is again something you were saying earlier — have critiqued the lack of transparency among pollsters in India when it comes to reporting methodology. What do you think can be done to make pollsters more transparent about how they’re doing their surveys?

YD: We have already taken the first step. You might be aware of this. This we have been trying for a really long time, and now there is a group formally in place called the Indian Polling Council, to start with the colleagues from [names of polling companies have been removed]. They have pitched in together. The idea is basically to make everyone accountable. The idea is to make this entire process more open.

We have been trying. Our office and our working posture has been always open. You have visited yourself. You can talk to any colleague in the industry that those who are confident in their work are always open. When I say open, that means even physically opening up the facilities, to visit the office, see the data, see the process, how it is being done. It is as open as that. So those who are trying to do that in media, especially the ones who have been doing it for a longer time, they really do feel the importance of making it more accessible, more transparent so that the public at large, the media at large gets educated and they can differentiate a good apple from a bad apple. Purely on the merit of the data. Institutionally speaking, it is an important thing to do. So that’s why we have formed that.

The biggest differentiator of those who would be in this organization or this setup and those who would be outside this gambit is the question of accessibility. Anybody who is in this group is committing and opening up for peer review of their data if there is any controversy around. If I am opening up, “Okay, you are happy. Please come and visit my data. See the data. Analyze it. Figure it out. If I got it right, why? If I got it wrong, why?” Whatever it is. If I am opening it up, I am pledging myself to do that, then I am transparent enough for my industry and my peer review. If somebody is not agreeing [to] that and remaining opaque, their problem. But at least people would know who is part of the Indian Polling Council and who is not. Who is opaque and who is transparent.

Somewhere we have to draw the line. And I believe the responsible ones have started realizing that. It took a little bit long. I believe that this should have been done five years back. But better [to] be late than never. We are starting it now, and we hope that this will help us in the long run.

SS: Would the Indian Polling Council determine– members would have to disclose certain things about their methodology, their sampling, their margin of error? I haven’t seen margin of error in almost any of the polls I’ve seen reported here.

YD: Yes. Pretty much. We are going to make a format which will be required to be filled up by all the pollsters every time they release a poll. That standard form would be uploaded on the website. So for any research the disclosures are complete and full. That standard format is applicable to anybody who is willing to offer. And this has been asked by the Press Council of India, that it should be done. This has been advised by the Indian Law Commission, this should be done. And they actually advised that this should be done in consultation, something like the British Polling Council.

I wrote up to Nick Moon, who is the Secretary of the British Polling Council. Nick Moon was running NOP. They immediately agreed to help in every possible way, that we can pick up all those standard protocols, SOPs, from their things and apply. So I wrote to Kathy Frankovich and she happily said that she would be helping in every possible way. I am also a member of WAPOR. So what I am trying to say is that those things which have been done and tested over there, we just have to try and apply everything.

If anybody has a problem with that kind of transparency, it’s for that organization or that person to explain why they are uncomfortable. But if you are to be part of this particular setup, you need to be transparent. You need to be open for a peer review. That’s the bottom line. That if there is a question mark, then you shall be sending the entire dataset for the peer to see through and understand what the problem’s like and help you with the answers. It’s about the intent. Transparency is basically a question of intent. Whether you are opened up or not, whether you have the intent to let the peers review, that’s transparency. Those who are not, it will automatically come out that they are not a part of this group. Because they refuse to follow these procedures. They refuse to follow these standard operating protocols. They refuse to disclose this part of the methodology. They refuse to disclose the minimum amount of information that should be released with every poll. So we are forcing now these guidelines.

This should have been done many years back. I understand that. But, I don’t know why, the seniors at that point of time did not do it. I don’t know why and I am not questioning them. Maybe these were not the important things at that point in time because there were only a handful of players and even then we were in good communication with each other, and offering help and solutions to each other. It was a good, cozy club. We used to talk to each other and give advice to each other. If I had a problem, I could have picked up the phone and [talked to] Mr. Yogendra Yadav and he had been kind enough to suggest [to] me the solution at that point in time.

But now we have moved because all of a sudden we are looking at the industry growing. It’s good that the industry is growing. If it grows, it’s good. But then the growth has to come with the discipline of the quality consciousness and transparency.

SS: How soon will the Indian Polling Council be a more formal organization? Will we see this in place by the time of next May’s elections?

YD: Right before that.

It’s already in place, first of all. We are in regular communication as a group. It’s already in place. We are talking to each other. We are helping each other. It’s just probably I believe that we would like to do it this calendar year, that everything [is] formally in place. Most probably, if all goes well, we would like the general secretariat to be at CSDS for a simple reason that number one, they are the oldest of the bodies who have been doing that, and number two, they are [a] noncommercial venture. They are academic in nature. So they are a nice [place] to be stationed. I hope that Sanjay [Kumar, Director of CSDS] agrees to that. We are likely to have our very first meeting very soon.

SS: Thank you very much.

Advertisements

3 thoughts on “Yashwant Deshmukh on the challenges of polling in India and the forthcoming Indian Polling Council

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s