Analysis of CSDS Tamil Nadu pre-poll in Indian Express

Today’s Indian Express has the Lokniti-CSDS analysis of the pre-poll conducted in Tamil Nadu. The leading article by Rahul Verma and P. Ramajayam examines the pivotal role of women in Chief Minister Jayalalithaa’s victory. Here’s the key takeaway:

The comparative credibility of leadership seems to have acquired a gender dimension in Tamil Nadu. Women voters delivered a decisive verdict in favour of Jayalalithaa and her party.

While the turnout was 73% for both male and female voters, there were 4 lakh more registered women voters than men. Survey data suggest that women voters rallied behind Jayalalithaa in greater numbers than ever before. The AIADMK led the DMK alliance by 10 percentage points among women voters and, thus, Jayalalithaa’s party drew its votes from the bigger share of the pie. Her party has had an advantage among women voters earlier, but the gap has never been this stark and large.

More women than men found Jayalalithaa a better administrator. They perceived her as caring more for them and for the poor than DMK chief M Karunanidhi. Compared to men, women voters were also less likely to think the AIADMK government was corrupt, the data show; on the other hand, they were more likely to think of Karunandhi as very corrupt. Women voters in comparison to men were more likely to give another chance to the AIADMK. This pattern is replicated across caste, class, and other demographic variables.

Links to all articles below:

On the role of women in Jayalalithaa’s victory by Rahul Verma and P. Ramajayam

On why the DMK fell short by Sam Solomon

On the poor showing of the People’s Welfare Front by Pranav Gupta

On the popularity of prohibition in Kerala and Tamil Nadu by Vibha Attri and Jyoti Mishra

Analysis of CSDS Kerala pre-poll in Indian Express

Today’s Indian Express has the Lokniti-CSDS analysis of the pre-poll conducted in Kerala. The headline article by Rahul Verma, Pranav Gupta, and Nitin Mehta examines the reasons for the BJP’s emergence in Kerala. Here’s they key takeaway:

…the BJP is cutting into the support base of both coalitions [LDF and UDF]. The upper-caste Nairs have remained the nucleus of BJP’s social coalition in Kerala and the party in this election received more votes among this section than the UDF. The BJP’s alliance with SNDP (a social organisation of Ezhavas), however, has not resulted in the desired effects. The Ezhavas are a numerically dominant backward-caste community and represent almost half of the state’s Hindu population. The BJP’s support among the community increased by merely five percentage points in comparison to the Lok Sabha election. Even among the Ezhavas, who reported to be associated with the SNDP, the NDA was far behind the two alliances.

The BJP’s vote share among the Dalits has increased significantly in this election. In fact, the BJP and the UDF won equally among the Dalits, who have historically remained aligned with the LDF. The survey data also suggests that the BJP has succeeded in winning a small segment of Christian voters. Does this development indicate a formation of a new social coalition in Kerala? Can Rajagopal replicate BJP’s Goa model where Manohar Parikkar built a coalition around upper-caste Hindus and Christians? Will the BJP succeed in making further inroads in Kerala?

In the past, the BJP’s success in new frontiers like Karnataka, Haryana, Jammu and most recently in Assam is in large part due to the en-masse transfer of a regional party’s support base. The prospect of this option looks bleak in Kerala and thus the BJP needs to prepare for another round of struggle in the state. The party’s vote share is still a few percentage points below the threshold point beyond which it could start making substantial gains in terms of seats.

Links to all articles below:

On the BJP’s emergence in Kerala by Rahul Verma, Pranav Gupta, and Nitin Mehta

On why the LDF win was unlike previous elections by Sandeep Shastri and KM Sajad Ibrahim

On the nuanced perspective of leftist voters by Hilal Ahmed

Kerala and Tamil Nadu Pre-Poll Observation

Today is Election Day in Kerala and Tamil Nadu. Originally, I was planning to write an election guide for each of the state elections going on right now. But time has not allowed for it. I will be heading back to the United States in less than two weeks and am starting to wrap up my research here. Where has the time gone?!

I spent the last week travelling through these two southern states to observe fieldwork for the ongoing CSDS pre-poll election studies. Unlike for Assam and West Bengal, these studies were being fielded before election day because of the quick turnaround on analysis. In both states, but particularly in Kerala, signs of the imminent elections were everywhere. Billboards and posters for the governing United Democratic Front (UDF) competed for space with the challenging Left Democratic Front (LDF). The BJP also had established a strong campaign presence in the capital of Thiruvananthapuram and the surrounding areas. In Tamil Nadu, I saw fewer campaign posters, particularly in the cities of Madurai and Chennai. In the villages surrounding Madurai, however, the two-leaf symbol of the governing Anna Dravida Munnetra Kazhagam (ADMK) and the rising sun symbol of the challenging Dravida Munnetra Kazhagam (DMK) decorated the walls of houses. Flags from the All India Forward Bloc (AIFB), which won the constituency I was visiting in 2011, were also in abundance. By the roadside, streamers with pictures of Jayalaalithaa, Tamil Nadu’s chief minister, or Karunanidhi, her rival from the DMK, hung from different campaign rally sites.

Like my previous post, a concise summary of the field visit is will be presented here rather than an extended narration.

On May 5, I flew down to Thiruvananthapuram by way of Chennai and checked into the guesthouse at the University of Kerala campus in Kazhakuttom. On May 6, I observed the full-day training for Kerala field investigators at the University of Kerala. Sandeep Shastri, National Coordinator of the Lokniti Network and Pro Vice Chancellor at Jain University in Bangalore, led the training. On May 7, I met up with two field investigators working in Nemom, a suburb that is 8 km south of Thiruvananthapuram city center (the assembly constitutency of Nemom was won by the Communist Party of India (Marxist), or CPM, in 2011). We met up late in the afternoon, around 4 PM, and so I only observed one interview since so many listed respondents were not home. On May 8, I met up with two other field investigators working in the village of Pallikkal, which is about 40 km north from Kazhakuttom in the assembly constitutency of Varkala (which was won by the Indian National Congress in 2011). Like the day before, we met late in the day — close to 4 PM — and so I could only observe two interviews. Like in Nemom, many of the selected respondents were not home at the time of our visit.

On May 9, I travelled by overnight train, the Ananthapuri Express, from Thiruvananthapuram to Madurai in Tamil Nadu. Madurai is the third largest city in Tamil Nadu, home to the legendary Meenakshi Amman temple complex. On May 10, I met up with four field investigators working in the area around Usilampatti, a town that is 40 km west of Madurai. I observed the work of FIs interviewing respondents in the countryside villages surrounding Usilampatti: Nallathathunayakkappatti, Sangakavundanppatti, Palaiyur, Karisalpatti, and Kattarripatti. On May 11, I squeezed in a morning visit to the Meenakshi Amman temple of Madurai before joining the FIs conducting in the city of Usilampatti. The assembly constituency of Usilampatti was won by the All India Forward Bloc (AIFB) in 2011. I took an overnight train, the Ananthapuri Express once again, from Madurai and arrived in Chennai on the morning of May 12. In the morning, I ate breakfast at Saravana Bhavan, visited the museum at Fort St. George, and stopped by Higgenbothams Booksellers and Stationers before heading to the airport and flying back to Delhi.

  1. Urban vs. rural areas: As with Assam and West Bengal, respondents in urban areas were more difficult to locate than respondents in rural areas. People on the street and in the neighborhoods of selected respondents were less likely to know the location of other selected respondents in urban areas than in rural areas. This meant that a lot of the time for fieldwork in urban areas (especially Menom) was spent asking people on the street if they know the location of selected respondents and walking around to identify households. In villages, particularly in village centres where older men gather, people were much more likely to identify the names of selected respondents and point us towards their houses.
  2. Low response rate in Kerala: Both researchers from the CSDS office in Delhi and the Lokniti coordinator for Kerala, Professor Sajad Ibrahim Ali, informed me that historically the response rate for Kerala is lower than that of other Indian states. This has been the case in state and national election studies, and as the data were being collected for this election study, Professor Ibrahim informed me that it would also be the case this time. Several team members from Kerala informed me that response rates are higher in north Kerala than in south Kerala, and particularly in Theruvananthapuram and the surrounding areas. No one had a clear answer about why this is. Some field investigators speculated that it had to do with the number of people working who were not home during the day. On my two admittedly short observations of fieldwork, field investigators experienced challenges locating houses and arranging interviews with the selected respondents once they had located the households.
  3. Caste in Tamil Nadu: Ranith, one of the field investigators whom I was observing, informed me that caste was a very sensitive issue in many of the villages around Usilampatti. The Lokniti state coordinator for Tamil Nadu, Professor P Ramajayam said, also said that Usilampatti is notorious for intercaste violence. Violent caste-related incidents in Tamil Nadu are most common between different backwards classes, such as Gounders and Vanniyars, and Scheduled Castes (SC) or dalits. In some of the villages we visited, it was dangerous for field investigators to even raise the issue or ask questions about it (every CSDS election study includes demographic questions about caste). For some interviews, the field investigators would therefore skip any such questions in the questionnaire. Field investigators in Tamil Nadu have to carefully navigate the complexities and sensitivities of caste while collecting data. That all four field investigators were from Usilampatti and the surrounding villages was critical, as they knew which villages it was safe to ask about caste.
  4. Religion: While Tamil Nadu is a state with a large (88%) Hindu majority, Kerala is a state with sizable Muslim and Christian minorities (54% Hindu, 27% Muslim, 18% Christian). The field investigators whom I observed in Kerala expressed skepticism that response rate was different across religion, though they said it was sometimes harder to speak with women in Muslim households (all the field investigators with whom I worked in Kerala were female). Though the neighborhood we were interviewing in Pallikkal was largely Muslim, I did not spend enough time in the field in Kerala to receive an informed impression about how religion shapes fieldwork in Kerala.
  5. Gender: The gendered dynamics of interviews which I observed in Assam and West Bengal were not as much in evidence in Kerala and Tamil Nadu. That is to say, while female respondents who were selected were nearly always interviewed in the presence of their father or husband in Assam and West Bengal, this did not seem to be the case in Kerala and Tamil Nadu. However, any difference may be due to the fact that the interviews I observed with female respondents in Kerala and Tamil Nadu were all conducted by female interviewers; in West Bengal, the interviewers whom I observed were male. In Tamil Nadu, the team of field investigators included two male FIs and two female FIs. This was for the safety of female FIs when traveling through villages in the countryside. It also allowed for male FIs to interview male respondents and female FIs to interview female respondents.
  6. Education level: For a number of respondents in Usilampatti and the surrounding areas, education level was so low that respondents could not understand the content of many of the questions. One respondent, for example, responded to a question about Prime Minister Narendra Modi’s different government initiative by saying that she did not know who Narendra Modi is. Field investigators thus had to make spur-of-the-moment judgments about which questions to ask and which could not be reasonably asked of the respondents. In the case of illiterate respondents who lacked even basic knowledge about political affairs in Tamil Nadu and India, the focus was on collecting correct demographic data and ensuring that the respondent answered the question about voting intention.
  7. Timing: Many of the respondents in Nemom, Pallikkal, Usilampatti, and the surrounding villages of Usilampatti were not home during the daytime because they were at their jobs. In the case of villages around Usilampatti, there was one village for which all or most of the villagers were out in a field working under the MGNREGA (Mohandas Gandhi National Rural Employment Guarantee Act). We went to the field to find the villagers assembled in the shade of the trees. When the FIs began conducting interviews with selected respondents, a village clerk from the panchayat (local administration) showed up and demanded that the interviews cease. The local official told the villagers not to answer any questions being asked by survey researchers. The FIs tried talking to the clerk and showed them their official CSDS letter, but the clerk insisted that we leave. This incident showed how field investigators sometimes must deal with obstacles presented by local government. It also illustrated how unemployed workers receiving benefits through MNREGA (which guarantees 100 days of agricultural work a year) may be less likely to be interviewed for surveys.
  8. Economic migration: Many of the selected respondents who were not available in the villages surrounding Usilampatti had left their village to work in either a larger city of Tamil Nadu or another state like Kerala. Such economic migration meant that many selected respondents in a given village could not be interviewed.

My pictures below. Unfortunately, my phone failed me on the date of May 6 so I lost pictures from the training workshop at the University of Kerala. What was perhaps most visually striking in Tamil Nadu was the placement of beautiful towering temples with elaborate carvings in village centres, even in the tiniest and poorest of villages.

Counting Day is only three days away. CSDS should have some analysis in the papers following the announcement of the election results.

May 7 (Nemom, Kerala)




Billboards for the United Democratic Front and National Democratic Alliance, over graffiti for the Communist Party of India (Marxist)








Picking jasmine



Graffiti for the Indian National Congress / United Democratic FrontIMG_20160507_174525

May 8 (Pallikkal, Kerala)


University of Kerala campus in Kazhakootam













May 9 (Kazhakootam to Madurai)


With Rajeena at the University of Kerala canteen



Aboard the Ananthapuri Express

May 10 (Villages around Usilampatti, Tamil Nadu)


Tamil dailies report on the election


In the Dalit section of a village





Anna Dravida Munnetra Kazhagam (ADMK) and Dravida Munnetra Kazhagam (DMK) posters










The two leaves are the symbol for the ADMK


The sun rising between the mountains symbol of the DMK










Ranith, Thanalakshmi, and Pechi

May 11 (Madurai and Usilampatti, Tamil Nadu)


At the Meenakshi Amman temple










Golden Lotus Tank








Lord Ganesha




A model of the temple complex


Dancing Shiva and Parvati


In the Thousand Pillar Hall








Usilampatti bus station


Interviews in Usilampatti






With Ram’s grandmother


Downtown Madurai

May 12 (Chennai)


Fort St. George museum



Higginbotham’s Booksellers, oldest bookstore in India

Sanjay Kumar on the challenges of survey research and CSDS’ role in Indian society


Sanjay Kumar is the Director at the Centre for the Study of Developing Societies (CSDS). He is also the Co-Director of Lokniti, a Research Programme of the CSDS. His published books and edited volumes include Indian Youth and Electoral Politics: An Emerging Engagement, Changing Electoral Politics in Delhi: From Caste to ClassMeasuring Voting Behaviour in India (with Praveen Rai),  Rise of the Plebeians? The Changing Face of Indian Legislative Assemblies (with Christophe Jaffrelot)and Indian Youth in a Transforming World: Attitudes and Perceptions (with Peter R de Souza and Sandeep Shastri). He has contributed chapters for several edited volumes, written various research reports, published articles in both international and national research journals and writes regularly for popular newspapers.

As an expert in survey research, Kumar has directed various national and state level studies conducted by the CSDS, which include the National Election Studies (NES) 1998, 1999, 2004, 2009 and 2014, various round of bi-annual ‘State of the Nation Surveys’ and various state level studies. While his core area of research is electoral politics, he has also been engaged in survey based studies on Indian Youth, State of Democracy in South Asia, State of Slums of Delhi, State of Indian Farmers and Issues of Electoral Violence.

Sanjay Kumar is also the supervisor for my research project. Last Tuesday, we sat down in his office at CSDS to discuss his background in survey research, the challenges of survey research in India, and how he defined CSDS’ goals and mission. The transcript of the conversation below has been lightly edited for length and clarity.

On his background in survey research

Sam Solomon: Please tell me how you got into the field of survey research.

Sanjay Kumar: I won’t say it was very pre-planned. It was kind of an accident.

I had enrolled myself as a PhD student in the Department of Political Science at Delhi University. But I also wanted to get hands-on experience of how you collect data, how you analyze data. So I came to this place [CSDS] to work along with the team that was headed by Yogendra Yadav and VB Singh. This was 1993, October. They were initiating a project to study assembly elections which was being held in five states: Delhi, Madhya Pradesh, Rajasthan, Chhattisgarh, and some other state. I came to know about the project. So I came and [told] them that I would be happy to work with them on the project. They said yes, this is a small project, we would be happy to associate [with] you for a few months. Maybe two, three, four months.

So I started working on the project. Got more and more interested in the kind of analysis which they were trying to do with the data, the fieldwork, etc. With my initial… not contract but kind of informal understanding that I would be working for three, four months, I also got a chance to the go to the field. I went to Madhya Pradesh to do a study of one constituency. I extended my research to making a comparison of four or five constituencies, looking at the Muslim voting behavior, which is my first article in EPW [Economic and Political Weekly], “Myth of Muslim Vote Bank.”

So I continued. I worked for three, four months. Then the project was getting winded up. But still VB Singh and Yogendra Yadav were looking forward to bigger studies in Bihar and in Lok Sabha elections, 1996. So they checked with me if I would be willing to be part of the project for the next six months or one year’s time. Since my initial experience of working three months, I got very excited about the kind of work which they were trying to do. So I said yes, I would be willing to work for the next six months or so, or one year. Then I think we moved into some kind of formal contract.

And then time passed. After that, we had the assembly elections in Bihar. I was deeply involved in that because I belong to Bihar. So my home in Patna was kind of a central place from where we were running the project. I went around to several constituencies to do the study. I especially studied one constituency, Mokama. But along with that, I was also coordinating the entire study in Bihar.

And by that time, I had almost decided that now, it’s enough. I should actually go back, do my PhD, etc. So that’s the time I gave a hint to Yogendra Yadav, with whom I was closely working, to say, “I think it’s enough. I should go back to my PhD, etc.” And then, I think, this was a conversation I think we were having at the railway station when I came to see him off. He said, “Let’s discuss this when you’re back in CSDS. Why do you want to take a decision immediately?”

And when I came back, we had [a] long chat. He said, “If you look at the history of CSDS, people who have worked here, their careers have gone up. Their careers haven’t gone down. So why do you want to leave? You can continue your PhD. You can continue working with the projects if you want. We have a space for you.” Space, in a sense, space in a place to work with me. I thought several times, and then decided okay. Then he said, “Now we’ll work out another arrangement. In a sense, it would be kind of a lecturer-level arrangement now. You have been working with us the last two years as a research officer.” The initial six months was a research assistant, then it was kind of a research officer. After the Bihar assembly elections, they said, “Now we are heading to the big Lok Sabha elections. Now you have gained experience. We value your association. And you would gain by working with us, and I don’t think your career is going to be on a downward slide. And if you agree to work, then we can work another arrangement. Give you a lecturer’s level position. So you will be a research associate now.”

I decided to continue and then finally after the Lok Sabha elections of 1996, we had a detailed discussion about how I [saw] my career moving. And they said, “Now we want to take this election study forward. We don’t want a break in this. And we would like a person like you in a permanent position. Are you interested in that kind of thing?” The institution advertised for that position. I applied along with several other applicants, and I was selected by the selection committee. Formally, I would say I joined CSDS faculty in April 1998.

[For] my entry into survey research, this is the kind of journey I took. But election as a theme has always interested me. It’s not that it was something not interesting for me. Even when I was in my school days — I remember class 12 — I was so fascinated about elections that I used to do, if I recollect, my kind of psephology. I used to read newspapers and prepare charts on my own. Who is likely to win from which place? Though I miserably failed because this was 1984 elections, and ‘84 election was a wave election for Congress. That was the time, if I remember correctly — and I do remember correctly — that I was kind of anti-Congress. Really did not like Congress. So when I was doing my own kind of psephology, I think I used to be a little biased. When I was making a judgment about who was going to win, I would actually defeat most of the Congress candidates on paper. “Oh, he will get defeated…” I remember defeating on paper [Bollywood superstar] Amitabh Bachchan because he was a Congress candidate contesting against H.N. Bahaguna, who a very tall leader from another party. That’s my entry into survey research.

SS: You joined CSDS informally in 1993, so you’ve been with CSDS for twenty-three years. During that time, which aspects of survey research at CSDS have changed the most and which have remained consistent?

SK: I think the big change is the timelines. When we started working in 1996, ‘96 was not with TV. It was with newspapers. But when we are working with television as a medium, the analysis has to be very quick and very sharp. If one is writing an article in the journal, then you have all the space. You can write 5,000 words. When you’re writing in a newspaper, you have to restrict to 800 words, 1,000 words. But when you are actually providing stuff to the television, it is very little words but it has to be very, very sharp. So I think our rules of analysis, trying to do deep analysis, that’s one thing that has changed. [In] 1996, we would do broad analysis maybe. And it would take a lot of time. That has changed.

The second thing is about the sample. It is also connected with the analysis. Earlier we used to do more aggregate analysis. We would come down to a few states, bigger states where we had bigger samples. But now we know that we not only do analysis for big states, we have been able to do analysis even for small states. Even within the states we actually disaggregate the analysis. We try to go down the level of analysis to figure out what’s happening at the ground level itself. That is a big change.

The other is that, initially, ‘96, ‘98, ‘99, even up until 2004, the survey as a tool we applied only for election studies. Now we have diversified a lot and we apply the survey method to do studies on various other themes. We have done the study on farmers. We have done the study on youth–two studies on youth. We have done studies on Naxals. And various other issues: economic reforms, Muslims, women, eating habits, all kinds of issues we have touched upon. That’s a big change. The initial years, we were only looking at using surveys as a method to study elections, voting behavior. Now we have used that tool to study various other kinds of issues.

On the challenges of survey research in India

SS: What are some of the greatest challenges you’ve encountered during your time at CSDS when it comes to 1) accurately measuring public opinion and 2) effectively communicating the findings from your research to a media outlet or the public?

SK: I think the biggest challenge has been to keep up the reputation CSDS has got. Because as soon as something is put out in the public, that this is a study conducted by CSDS, moreso with regards to elections — vote percentages, which party is likely to win — a large number of people believe it is a truth. And they would say, “If CSDS is saying Party A is winning, definitely Party A would be winning, because they never go wrong.” So that’s the biggest challenge. Maintaining that credibility in the field and trying to do it as accurately as possible.

SS: How do you address that challenge?

SK: Being neutral all the time. We try not to be biased, not to be swayed [by] our likes and dislikes for leaders and political parties. And the big advantage when we are trying to do the analysis, when we are trying to assess which party is likely to win the election, we take advantage of the huge datasets we have in our hands. The historical data.

Suppose we see the Bihar elections. For Bihar, we would have at least eight or ten survey datasets. So we try and look at what has been the pattern when we did surveys in the state. In order to try to be as accurate as possible, we try to be neutral. We try to collect data in a neutral manner, do the analysis in a manner as neutral as possible, plus take advantage of the past data to figure out if at all we made any errors when collecting data in a particular state. That’s how we try to make our findings as accurate as possible.

SS: When it comes to the surveys themselves, have you found that certain types of populations — gender, caste, class — are more likely to be oversampled or undersampled? If so, which populations have those been?

SK: If you look at our sampling technique, our sampling technique is very different compared to the sampling technique of other survey agencies or organizations. We not only sample the constituencies, the polling stations, but we also sample the voters whom we want to interview. All of these are done randomly, systematic random sampling. If we have adopted a good sampling method, a scientific sampling method, and if investigators actually follow that method — when they go to field, they go with a list of people who have been sampled, and they have to interview only those people on the list — if they actually follow that, there is very little chance of oversampling or undersampling any segment of the population. Unless we are looking for a very small sample.

Normally, we take a sample of say, 3,000 or 4,000 in a state. The chances of the sample being overrepresentative [of a demographic segment], over or under, is very [small]. That has been the record of our surveys for the last twenty years. We have put all this data on our website to figure out when did we go wrong with a sample that is representative.

But still, if we look at the chances — still there are chances — normally, we might expect women to be slightly less in our sample because it also has a relationship to the investigators you have. If there are some states where we don’t manage to get an adequate number of women investigators– it becomes extremely difficult to interview Muslim women if you don’t have woman investigators.

SS: Muslim women investigators or just women investigators?

SK: No, just women investigators.

Normally, there would be a lower completion rate in the urban areas because the urban population is more mobile compared to the rural population. Sometimes we also get samples which are underrepresentative in terms of the urban population.

These are the two elements of the sample which I have noticed that sometimes we get wrong, in a sense, underrepresented. All the time they would be underrepresented. Hardly overrepresented. Urban and women.

SS: When it comes to SCs (Scheduled Castes) and Muslims, you have census data that you can consult to get the proper proportion of SCs and Muslims in the states. So it’s not as much of an issue.

SK: I don’t think that we had issues with regards to the Muslims and the dalits, SCs, when it comes to getting a sample.

Unless we are doing an exit poll. We used to do exit polls earlier but now we have stopped. Because if we are doing exit polls, there is always a chance of heavy overrepresentation of young, urban voters. And also this issue of dalits, Muslims, or tribals not being adequately represented in the sample, that’s the case when we do exit polls. That’s largely the case. And that’s why since we are not sure of getting the right representative sample or not, that’s one of the reasons we stopped doing exit polls.

That’s a big problem in exit polls, but not in the pre-poll or post-poll surveys which we conduct. Only because we actually sampled the respondents from the electoral rolls.

SS: What about OBCs (Other Backwards Classes) or intermediate castes?

SK: There are very rough estimates about the proportion of OBCs in different states. It ranges from 40% to 55%, 60%. And it is such a large group that getting them not adequately represented in the sample is rare. But since you don’t have an exact estimate, it’s not like Muslims, it’s not like dalits or tribals, [where] you have data to compare [to the sample]. OBCs are only an estimation. In Bihar, we have roughly about 52%. We don’t know whether they’re actually 45% or they’re 55% or they’re even 60%. So that has not been an issue in the past.

SS: I find the issue of caste to be really interesting, maybe because it’s a foreign phenomenon for me, but also because it’s so complicated. Last week, [the Lokniti survey research team] were trying to figure out how to code the different castes in Tamil Nadu. Everyone in the office is north Indian, and the caste system in Tamil Nadu seems really complicated. We were struggling to figure out which caste gets categorized as upper caste, and which as intermediate, and so on.

How do you do that for every state in India? Do you have the lean on the local knowledge of the Lokniti network?

SK: You will notice that we use a codebook to code several open-ended questions, caste being one of them. That codebook has not been prepared overnight. If you get sometime a chance to look at our codebook [from] ‘96, ‘98, ‘99, [you can see] how it has progressed, how it has improved, how it has become more comprehensive now compared to the past. The 1996 codebook was one sheet, one page, for all the codes: education, occupation, caste, etc. It was just one sheet. Now if you look at our codebook it is a booklet. Six or seven pages have been devoted only to caste.

This has been possible because, as you rightly mention, of inputs from our local coordinators. At one point in time, we actually took it as a project to engage our local coordinators, and requested that they engage sociologists at the local level to figure out where each caste should be placed, and what is the parallel of caste A to castes in other states. Caste coding is really complex, and we have been able to develop [it] after a long, long period of experience.

Still, we won’t say that it’s completely perfect. Because every time we do a survey, we encounter this problem in varying degrees. And we try to develop it as [neatly] as possible.

On the goals and mission of CSDS

SS: Which factors do you weigh when deciding which research projects you are going to go forward with? Naturally, there’s always going to be an audience for election surveys. People like to read about politics and elections. But how do you decide to do a survey on farmers, or youth, or Naxals?

SK: You rightly pointed out that elections keep happening every year, and CSDS is a place that is known for its research on elections. So election studies we keep continuing. On other issues, whether we want to do a survey on Topic A, B, C, or D, the big consideration is: is this a topic [about] which people want to know, people would have curiosity? Is it topical? Do people have concerns about that?

So if you look at the kind of research which we have done on youth, there’s so much focus on youth, not only in India but in South Asia and other parts of the world as well. Their anxieties, aspirations, employment, unemployment. So we have done studies in the past on youth, and also we are in the process of doing another round of the youth study to look at the change over a period of time.

Farmers — if you actually look at the debate going on in the country for the last couple of years, [there is] so much focus on the crisis of farming in India related to land acquisition bill, etc. When somebody approached us with the idea that there is a need to do a proper study of farmers in India, though there are other studies — but mainly done by economists who look at the economic condition of farmers mainly. So we pitched in. We did the study on farmers. Along with looking at the economic conditions, we focused more on anxieties, aspirations, expectations, more on opinion rather than on fact. That is how we picked up that study.

On Naxals, [there is] so much focus on this issue of Naxals. Is it an issue of economic deprivation or is it a law and order problem? That’s the big question people want to address. Some people see it as a law and order problem and they say, “The government should use hard force to just crush them.” The other side says, “It is an economic issue. These are poor people whose land has been taken away by the government. It is an issue of their survival.” So we wanted to do a proper study [to] figure out what is the situation. We are not saying whatever we would say would be the truth. But at least we would try and bring out what is their situation on the ground.

So that’s how we pick up the themes. But we have also been approached to do studies about consumers’ perceptions about the new airports which have been developed: Bangalore, Hyderabad, etc. We were quick to say, “No, we don’t want to get into this.” This is none of our business. If we want to do a study, it has to relate to [the] social sciences. We don’t want to get into airports. We were approached to do a study about something related to electricity. Supply of electricity, are people happy, should prices go up, etc. This was in Madhya Pradesh. And we said, “No, we don’t want to do this.” This is not social science.

SS: Aren’t those issues — airports, electricity — related to issues of economic development as well? I suppose that if an electricity company approaches you and says, “We want you to do this survey, so we know how to price our electricity.” That’s one thing. But how do you draw the line between what is social sciences and what is not?

SK: The prime concern is that when we do the survey, we also want to be clear about how we are going to make use of this data in the future. If it is for the client, just to do the survey and give them the data, is that the end? CSDS is not there to make money off of surveys. So we figure out if this data is going to be useful for us in our future research. Will this data be useful for other social science researchers?

When we thought of the electricity [study], we thought this would be a kind of client servicing. You take money, collect data, and give it to the client. Airports — it would make news in the newspaper for one day, but later on social science researchers won’t be able to make any use of the data. That’s the concern.

CSDS is primarily a social science research institute. This is not a survey agency. We have to be clear about that. We keep reminding ourselves that if a study has something to do with social sciences, society at large, people will make use of the data in the future, then we will get into that.

SS: How do you envision the role of CSDS vis-a-vis Indian society today? What is CSDS’ overall mission for itself, and how does Lokniti’s research fit into that mission?

SK: CSDS’ mission of research is very, very difficult to say. We have been debating for the last several years to figure out what is the core research of CSDS. We figured out that we can’t say, “No, this is the core of CSDS’ research.” We can only say the core of CSDS’ research is to do research in social sciences which would cut across several themes: media being one, urban being another, elections/democracy/politics being an important theme. Also now youth has become an important theme for the last five, ten years because as soon as anybody talks about a good study on youths in India, they would refer to CSDS. Political philosophy. Secularism. So there are six or seven themes which we can say are the themes on which CSDS does research. We won’t be able to say, “This is what CSDS research means.”

Lokniti’s position vis-a-vis CSDS — in CSDS, people not only do research on various themes, they also adopt different methods of doing research. Within that, Lokniti fits into one of the core themes of CSDS, because CSDS was established for doing research on Indian politics. Lokniti now is continuing that tradition of doing research on Indian politics. When CSDS was formed in 1963, the initial studies of politics [were] conducted through survey research: doing surveys in Kerala when the Kerala assembly elections were taking place, then immediately after that we had national elections. CSDS conducted surveys in the 1967 election [and] 1971 elections. Lokniti is trying to the take the CSDS tradition forward of doing studies on Indian politics using surveys as a method. That is why I would say Lokniti is core to CSDS’ functioning.

SS: CSDS is like a tray with different foods and the foods don’t really get mixed together. I’ve been here for more than six months now, and I’ve really gotten to know the Lokniti team well. But I don’t know anyone from Sarai. I don’t even know the other programs that are here at CSDS. Do the other teams interact with each other? Is there ever any talk of working together or does everyone just do their own initiative?

SK: People are always encouraged to do collaborative projects. But we can’t imagine a situation where twenty faculty, twenty-two faculty, will all be working together on one project. This happens only in small institutions, with less than ten faculty, then maybe a large number of faculty will be working together. But in an institution where you have a faculty of, say, twenty-five, you can’t imagine a situation where everybody would be working on the same theme.

But yes, we have forums where people are expected to interact with each other. All the programs make a presentation to the faculty about their work. Every month we have a faculty meeting, but that’s kind of administrative. We don’t discuss academic issues. Every Friday all the faculty members try and sit together to have lunch. That was designed in such a way that people can have exchanges about their work. We have also instituted a system of annual faculty seminars which a very intensive exchange of each other’s work and ideas.

So we have tried to have forums where faculty interact, know each other’s work, and also anybody is welcome to make a presentation whenever someone has conducted a study. Now we have also instituted a system of monthly faculty seminars, which began only last month. Every month one faculty member is going to present a detailed work. We have kept two hours for that. You present your work for an hour, then a one-hour discussion.

So there are platforms available, but I can’t think of a situation where everybody would be interacting on a daily basis with each other’s work. That kind of a situation is only in small institutions, because in small institutions sometimes the director assigns work. Suppose the institution gets three projects, the director assigns these three or four faculty to this project. In CSDS, we don’t have that system. We are very, very autonomous. But still you will find Lokniti working as a team, Sarai working as a team in which three or four faculty members are associated. Four faculty members are working on the Indian Languages program. Then there is a project on violence. Three colleagues are taking that project forward. And then you will find another five to seven faculty members who are not collectively working but they are working on their own projects.

SS: What do you think have been some of CSDS’ contributions towards understanding Indian society?

SK: If you look at the various kinds of studies which have been conducted by CSDS during the last fifty-three years, the kinds of issues on which we have touched upon — Rajeev Bhargava’s work on secularism; Sarai’s work on media and urban space; Hilal’s and Sanjeer’s work on marginality, Muslims, deprivation; Lokniti’s work on elections, youth, farmers — If you look at the range of issues which have been touched upon, the range of methods which have been adopted by various researchers, all this has been in the public domain.

We use four platforms. The media, I would say, print and electronic. Research journals. Books. And these days we also have three workshops which CSDS organizes every year. A workshop in which we try to train young researchers in data analysis; we have been doing that for the last ten years. Another course on researching the contemporary which is in the sixth year. And last year we started this new workshop only for students who belong to the marginal communities: SCs, STs, and Muslims. I think there is a lot of input to the social science community by way of publications and training workshops.

SS: Which topics do you think are in need of additional research here in India? Which topics do you think are underexamined?

SK: There is a need that in social sciences people should try to pick up the themes which have a direct connection to society. Not necessarily with policymaking, but directly with society. There is maybe a little more focus on the historical method of research. I would personally want people to be doing research more on current themes. There are issues [related to] youths. There is so much anxiety about farmers. It is an important issue. Minorities. Reservations is such a hot debate in this country. Secularism. And similar such issues. Issues on which the government is trying to formulate, there is a need to debate those issues. There is a need to formulate rules, or revise the already existing rules and regulations, which relate to the common man.

Also, there is a focus these days on India-China relations. But as you know, CSDS is not a place where we actually do foreign relations. Maybe some research on — not exactly on foreign relations, but something related to relationships with neighbors, which would fit into the big ambit of foreign relations.

SS: Thank you very much.

Yogendra Yadav on Lokniti’s origins, the challenges of polling in India, and the alternative politics of Swaraj Abhiyan


For many Indians, the name Yogendra Yadav is synonymous with polling and psephology. In nearly all my previous interviews, his name has been repeatedly mentioned as an unparalleled authority on polling in India. Well-recognized from his days of forecasting election results on CNN-IBN, he is a widely respected scholar of political science who has written countless articles and edited several volumes on Indian politics.

After eight years of teaching political science at Panjab University in Chandigarh from 1985 to 1993, Yadav joined the Centre for the Study of Developing Societies (CSDS) in Delhi and brought the Centre back into the field of election studies after a two decade hiatus. In 1997, these studies were installed at the Lokniti Programme of Comparative Democracy, where they have continued ever since. The founder-convener of the Lokniti network of professors who fielded and analyzed these studies, Yadav was the director of Lokniti from 1997 to 2003. Since 2004, he has been a Senior Fellow with CSDS (full disclosure: CSDS is my primary research affiliate, and I am working with Lokniti on my research).

In 2012, amidst the India Against Corruption protests shaking Delhi, Yadav left psephology as he became gradually more involved in politics. He co-founded the fledgling Aam Adami Party (AAP) and contested the 2014 Lok Sabha elections on an AAP ticket. Yadav was a senior leader in the National Executive when the party swept the Delhi assembly polls in February, winning sixty-seven of seventy seats in the Delhi Vidhan Sabha.

Shortly after this historic victory, Yadav and fellow AAP co-founder Prashant Bhushan wrote a joint letter to the National Executive questioning Delhi Chief Minister Arvind Kejriwal’s unilateral decision-making approach and expressing concern about compromising the party’s core principles. This started a series of allegations and counter-allegations that ended in March with the expulsion of Yadav, Bhushan, and two other senior AAP leaders, Anand Kumar and Ajit Jha, all for “anti-party activities.”

Yadav and Bhushan continued their involvement in politics through Swaraj Abhiyan, a grassroots movement for an “alternative politics in India” that they started earlier this year. In June, Swaraj Abhiyan launched Jai Kisan Andolan, a movement focused on bringing the plight of Indians’ farmers into the national discourse. Swaraj Abhiyan has since conducted a number of protests and yatras, or marches, to focus on farmers’ issues.

Last Tuesday, I met with Yadav at his apartment in Patparganj to talk about what he terms his “previous life” in polling and his present life in politics. The transcript of the conversation below has been lightly edited for length and clarity.

On public opinion research in India and starting Lokniti

Sam Solomon: You spent several decades doing public opinion research in India. I’d like to know how the methods of research changed during that time.

Yogendra Yadav: I spent two decades doing that. I was lucky to be at CSDS because I think the foundations were already laid there. While people remember the work that Lokniti team did, I did, my colleagues did, unfortunately not many people remember the foundational work done in the 1960s and 1970s. I think we really built on that. We were not doing something radical or innovative. In the 60s, people like Bashiruddin Ahmed, DL Sheth, and VB Singh, along with others, were involved in laying the foundations of political opinion, attitude, behavior research in India. And in many ways we simply built on that. There is an article which traces this entire journey which I had written a lot but is under a generic team name. It’s in the EPW of 2004, I guess? Which is “National Elections Study, An Introduction.” And this piece traces first generation, second generation, third generation of survey research in India. So I guess we made some innovations. We made some improvements. But much of what we did was by way of fine-tuning an already existing model from CSDS.

What we did was to expand it, expand it in terms of the sample, expand it in terms of reach, and expand it in terms of taking it to the public and making it more popular and visible. We made it more frequent. We made it more of an archive, a systematic archive rather than an episodic thing. And we tried to integrate that with an ongoing political commentary. I guess that’s what we did.

Unfortunately, much of that work is known for its forecasting, which is how we got the money to do whatever we wanted to do because we moved away from that foreign-funded research grant model. And naturally people remember the forecasting part of it, which wasn’t bad, which on balance I think was a satisfactory experience. But that’s not why we did it. That’s not really what we wanted to do.

I think the part that we did somewhat better was the post-mortem of electoral verdicts. That was our forte, and I’m glad we managed to do that. And I think that is the part that CSDS has continued after I left. And I feel happy that they are doing it in many ways better than when I was there. I looked at this Bihar thing and the previous election coverage. I felt that this was much better than what we would have done if I was around.

SS: So what were you trying to do then?

YY: This was continuity of the CSDS tradition which is to understand public opinion, attitudes, and behavior.

When I turned to research on Indian politics, which was not my original area, I discovered that the field was shockingly bereft of any serious evidence. I was trained as a student of political philosophy and theory. After ten years of research on that, on intellectual history, on political theory, and such like, I shifted to making sense of Indian politics. And I thought, I came to CSDS in the hopes that much of the evidence lies there, and I just have to go, understand it, and try to out it. And then I discovered how not only did we have a dearth of evidence, but the overall temper, overall style, overall mood, and mode of doing research on Indian politics was something which looked down upon any evidence at all. So much of this was hearsay. Much of this was pure speculation, and I didn’t know how you could research on empirical questions without gathering systematic evidence.

So the entire effort in that decade or two was to create an archive of public opinion and political behavior. And I do think that now we have something of an archive to go back to. I feel bad that we didn’t use that archive as well as we could have used in terms of coming up with theory, in terms of coming up with an understanding which would be different, distinctive. So I think people like me are guilty of underutilizing the data that we created. But frankly the task of creating the data itself was so huge that we were just snowed under.

SS: And the archive is there for scholars who do want to do that.

YY: For future generations, who I hope would do a better job of it than we did.

SS: What were the greatest challenges that you encountered when you were with Lokniti in terms of accurately measuring public opinion?

YY: Let me say what the challenges were not. There is a misimpression that somehow Indians are particularly immune to being measured by survey research. I find that attitude silly. I have never encountered it. If anything, doing surveys in India is easier than doing it in other parts of the world. Especially in the First World, in one respect, that the refusal rate is very low. If you go, especially in rural areas, not only do people give you interviews, they actually welcome you with a cup of tea. Because the survey penetration is so low that actually you can conduct very high quality face-to-face surveys in India and the costs are reasonable, of doing face-to-face surveys. These are not impossible costs. So India is actually still one of the few places in the world where you have the conditions permit a face-to-face survey where the costs allow that, people welcome it, and there is a trained scientific manpower which can actually carry it out. There are very few places in the world where the combination of all these things exists.

So this is really not a problem. Unfortunately, whenever some survey goes wrong or some forecast goes wrong, people fall back to this stupid excuse, that they say, “It’s impossible to do that in India. Indians are very different. Indians are unique.” I don’t think there’s anything of that kind.

What is also not a challenge is sampling. Your piece indicates that there is a problem in India, which is, I mean, we don’t have that kind of stratified data available at the micro-level. But if you go by strict random sampling, you overcome all of that. And because strict random sampling is possible in India, given the census data and given the electoral rolls and so on and so forth, we have been able to circumvent it. And the quality of samples, especially in the polls done by CSDS, has been quite good. I think by international standards these are very high-quality surveys.

The difficulties, the challenges we faced were the following: Number one, the academic culture of Indian social science really looked down upon survey research. So if the theorists were the Brahmins of Indian social science, survey researchers were the coolies. No one thought this was worth respecting. And this occupied such a low position in the hierarchy of Indian social science that the business of gathering evidence, especially on politics, was simply so underdeveloped. Fortunately, in other disciplines like history, for example, you could get up and ask a question on sources, which is to say, “What is your evidence?” But in political science seminars it was rare twenty years ago for anyone to ever ask a question about evidence. What is the basis on which you are making these grand generalizations? No one ever asked this. So that was one problem, a lack of research culture.

[Number two], there was lack of trained personnel in political science. India has a great tradition of statistics. India has a great economic traditions. But in politics, people who were trained in doing surveys and handling quantitative data, it was very, very rare. It still is unusual to find people who can handle data with some degree of sophistication in political science. Otherwise, Indians are spectacular when it comes to data and mathematics. This was the second challenge.

As a result of these two, we did not have a community where there would be a lively exchange of methodological and substantive issues in survey research. There was no methodological innovation and frankly no one actually understood. No one cared. Just as ordinary newspaper readers pick up the newspaper and say, “Oh, you got your forecast wrong. Your survey must be fraud.” Similarly political scientists picked up these things. Most political scientists could not even read bivariate tables in any sensible way, meaningful way, intelligent way. So methodological things, people simply didn’t understand.

While there has been a very intense culture of debate on politics in India, the culture of debate among political scientists was not careful on issues of middle-level generalizations. Most of the Indian debates have been at the very abstract level of generalization. So the lots and lots of generalizations that you need to build a hard theory of politics are simply unavailable. And unfortunately then much of these models came from a readymade repertoire of democratic theory in the West, which came largely from North America. And those theories simply don’t work. It’s a very different context, very different country, where you can learn a thing or two from constitutions. You can learn something in a very grand, general way. But middle-level generalizations about politics are so deeply anchored in the specifics of politics of that locality that to pick North American theories and to apply that to India was so weird.

So on the one hand, we felt we suffered from that complete deficit in our academic culture. On the other hand, we suffered from this excess export from North America. And it was a struggle for us to try and handle that situation there in Lokniti.

And I think we were very, very lucky in the mid-90s when we brought together that group which is called Lokniti today. It was an exceptional group in many ways because the quality of political scientists who agreed to join that, and the simple collegiality which you must have experienced, is rare. Academics don’t work with each other normally. Academics suffer from more envy than ordinary human beings do. But in Lokniti somehow we managed to create that collegiality and this is a group that has now worked for… more than twenty years? Twenty years, it’s just completing. So we were very lucky in terms of collegiality. We were very lucky in terms of intellectual integrity of many of the colleagues in [the] Lokniti group. We were lucky in having administrative and sheer hard work of many of the younger colleagues who we interacted with, who really laid the foundations of this work.

We were not so lucky in having leaders who could connect theory to empirics. And that’s where the limitations of people like me came into [the] fore. Some of us had to do that work. I don’t think we did it sufficiently well. So those were some of the issues and limitations.

The other big issue was lack of committed funding. Every time… I don’t know of any other research archive in the world which had to go through such financial stresses as we did and which had to do all kinds of things just in order to keep afloat. All kinds of things, exit polls, this poll, that poll, because we had no research grant. And it was astonishing that perhaps now Lokniti archives should be one of the biggest archives in the world on public opinion and attitudes and it is a shame that a large experiment of this kind did not have any sustained funding for what it was doing. And we approached everyone in the Indian establishment for that and no one was willing to support that.

What that did was we ended up investing… a great deal of our energy was exhausted in keeping the ship afloat than in steering it. That’s a challenge we had.

SS: Do you think that the existence of Lokniti effected any change in the culture of political science or among academics here?

YY: I dare say it did. Although it’s sort of for future generations to judge. For people like you to read and check and come to an independent conclusion.

I think it did, and in the following ways: Twenty years ago, you could comment on Indian elections, Indian political parties, Indian public opinion without bothering to dirty your hands with any evidence at all. Today it is far more embarrassing to do that than it was then. So we have made it embarrassing for people to do things without evidence. People do it grudgingly. People [do] it while pretending that they’re not doing it. People do it by way of taking potshots at us. But they do it. That’s important.

I must say it was partly facilitated by something which people don’t readily acknowledge, which is that our work gained acceptance in North American and European academica. Because academics working on India were desperately looking for some evidence. They at least knew that evidence matters. And there was none. So very early on in our work, people started appreciating the quality of data that we were generating. They were probably not into our perspective. But they understood the value of data. They started referring to it, and so did media. So Indian media and Western academia started respecting our data. Initially, Indian political scientists and Indian social scientists were indifferent and almost defensive and aggressive. But I think unfortunately given the unequal relationship between Western academia and Indian academia, Indian academics gave in. After, I think, the first five or ten years, it became very hard for them to ignore it because the people they cited cited us. Strangely, by this ironic twist, that is what gave us much greater acceptability. So I guess that has changed.

I also see now, wherever I go in the country, I see young teachers who come to me and say, “I was part of Lokniti workshop. I have done a National Election Study. I have been a researcher or trainer.” In the discipline of political science, there [were] so few opportunities of doing field connect that Lokniti and CSDS work came as a window of opportunity. And lots and lots of youngsters availed of that opportunity. And I guess that may have changed their orientation. There was very little research training available for political scientists. Basically, you read theoretical books and you picked up whatever impression you did from your rickshaw-wallah and from your taxi-wallah, and then you wrote books. But in this there was the first time an opportunity that people actually got seriously trained. So over the last twenty years, we must have trained a few thousand people. And that, I think, is an everlasting contribution which is not acknowledged because we are not seen as a teaching institution. But we actually have been a bigger teaching institution than many universities put together. So that, I think, has changed the culture. Insufficiently, but it has changed.

But, as I said, this is a biased opinion coming from the person who has every interest in overestimating the impact. It is for someone else to measure.

SS: My research is looking specifically at whether certain populations are, because of cultural or socioeconomic factors, more or less likely to be sampled or participate in a survey. I’d like to know over your career, throughout your experience, which populations you found to be … I’ll say, less likely to participate — I realize the question of whether groups are less likely to be sampled or participate is different — less likely to participate in a survey.

YY: Unfortunately, that’s exactly the kind of research I would have liked to see more in India which doesn’t exist. I should not be answering this question with reference to my impressions. Ideally, there should have been fifteen articles on this question. I should have been trying on those articles. But they don’t exist. That’s the problem. The research culture in Indian political science is so limited. So I speak with my impression.

My impression is that first of all it is a function of the method you choose: do you do face-to-face [or] do you do telephonic interview? Telephonic interviews are most exclusionary in India, and unfortunately many of the polls — not the election polls, but most other opinion polls that you see in the newspapers which are not in the time of elections and where there is no fear of being contradicted — most of those are simply telephonic polls. Worse, some of them are Internet polls. Now, that’s just plain unrepresentative. Even though we have telephone penetration which is substantial, even then, I think anything other than face-to-face interview is very unrepresentative. And it’s unrepresentative in simple class terms. It massively overrepresents the upper classes. That’s one.

It’s also a function of method in a different sense. Do you do an exit poll in the context of elections? Or do you do a post-poll survey, which is to say it’s a site? Do you do your survey in a public place? Or do you do it in the setting of their home? The surveys done in public places, especially exit polls, one of the most popular things — the trouble is as you picked up from that quotation… [my quotation of Yadav in The Hindu op-ed] although survey researchers like to believe that they have selected the sample, in reality the sample is selecting himself or herself. They walk to you. If you see the real life context of Indian elections, there is no way you can pick every nth person.

SS: I did see. I saw some exit polls for Bihar.

YY: You did? You went there?

SS: I went with CVoter and I saw some exit polls.

YY: It’s actually impossible. There is no way, if someone says, “Pick every thirteenth person,” how would you do that? There is just no way you can do it. We tried a few exit polls and we realized this is impossible. The bias of the researcher, and the attitude of the respondent, and the willingness of the respondent are far more important than any randomness or any method. So that is bound to overrepresent those who are more articulate, those who are more resourceful. And therefore, in terms of an Indian context, it is straight a function of your class and caste. In places like Bihar, I mean, almost all over the country, because there is a considerable overlap between class and caste, so every one of these things ends up oversampling the caste– the upper castes get oversampled.

First is the question of face-to-face versus telephonic or otherwise, second is exit poll versus this, and third, of course, is random versus any non-random technique. I think given the fact that we do not have sociodemographic information at the precinct level, I normally don’t trust anything other than a strictly random sample. Random sample with a reasonably large size gets you everything. But anything other than random sample almost invariably oversamples again the more articulate, the more well-to-do, men, urban. The slope of privileges is such that sampling tends to flow in the direction of socioeconomically more powerful groups: rich, men, urban, upper caste, media-exposed, educated. And since of these, except the gender, has a significant correlation with each other upon another, there is a very substantial overrepresentation of one section of society.

These three things put together account for the bias in sampling. And therefore over the years we went by the route, which I should say was established by the first generation in CSDS, which is only face-to-face interview, only at home, and strict random selection. Otherwise, you don’t get it.

SS: Something that I’ve seen in your writings, and this is something other researchers have also called for, is greater transparency of pollsters. So the people who are consuming this information can have a better idea of how this research is being done. Who do you think needs to take the lead in making those kinds of changes? Does it have to be the pollsters themselves? Or does it need to be media sponsors, the people who are actually paying for this research?

YY: Ideally, the pollsters themselves. If not, the media. If not, the officials, which is to say the Election Commission. And if not, the government.

Someone has to step in. This cannot go on infinitely because the industry has flourished without any regulatory framework. Now this is not an industry where regulation should come from outside. So yes, ideally, a council of pollsters or something should come into existence.

The only problem is that in India the record of such professional agencies to self-govern has been exceedingly poor. Whether it is the Medical Council, or it is the Media Council, the Editors’ Guild, and so on. Their willingness and capacity to defend the profession against outside onslaught in never matched by their capacity to reform within, and that’s why I remain skeptical that the pollsters themselves would be able to do [this].

The trouble is that the thing called the pollsters has two very different animals: there are places like CSDS, these are non-profit places; and there are these crass commercial operators who would not like to bind themselves down to very serious, rigorous protocols of transparency.

So sometimes I feel maybe the media and those who pay for it should come into it. Their problem is that they are ignorant. They simply don’t know what they are paying for. I rarely come across editors who understand even the basics of what they’re dealing with. So they are like [those] customers in the market who judges things by its package weight. Can the Election Commission do it? Don’t know. They have shown very limited appetite for regulating the media and they usually like very ham-handed methods like bans rather than any sophisticated regulation. And the government… God save us. That’s the last thing I would want to happen. So I don’t know, realistically speaking.

But yes, I continue to believe what you could do is have an agency which has representatives from pollsters and media advertisers which is sanctioned by the Election Commission to do this. And what you need is not a ban. I have repeated it, so much so that I now get tired of saying the same things over and over again. A ban would be a very silly way; it would actually push the industry under the carpet. It would actually complicate the matters even further. What you need are very strict disclosure protocols. I have written [in] several places about what needs to be disclosed, and everyone in the world knows what needs to be disclosed. I don’t have a particularly innovative solution to that.

The only thing I have added to that list is to say in the Indian context you must also disclose the social composition of your sample. You cannot tell me you have a sample of 1,900 and it was picked up from so many cities, so many this, it has this many men and women. Do tell me how does it correlate with or fail to correlate with the known social demographics of the area that we are talking about. That should be a mandatory thing in India. I need to know how many upper castes you have in your Bihar sample, and how many OBCs, and how many SCs, and how many STs. Unless I know that, how do I begin to trust your sample?

The second thing that I have added to that is to say the disclosure should be a three-tiered disclosure structure. One, certain basics should be self-declared, proactive self-declaration. Second, if someone asks you for some cross-tabulations, that should be made available on demand. Probably for a price, but it should be made available in principle. And third, if someone challenges the veracity of the survey itself, someone says, “The whole thing is a fake, you didn’t do it,” there must be a place where the raw data is stored and made available for a group of experts to come and check. It is not impossible to tell fake data from real [data]. If you have any experience of survey research, you should be able to make it out. And the data needs to be stored, and yes, a committee should be able to get access to your raw data, to go into allegations of fake data.

It’s not merely disclosure and transparency. It’s also regulation of quality. If you can set up something like this, then it’s all right.

SS: Have you heard out about the Indian Polling Council that’s being proposed? There was an article in The Hindu about it.

YY: I’ve just heard about it. If it comes about, it will be a good idea. And if everyone agrees to abide by it.

As I said, the record of professional self-regulation in India cutting across professions have been very, very poor: medical profession, legal profession. The only exception to that is auditors. Chartered accountants, for some reason, have managed to hold their bar reasonably high and they have not allowed it to fall. They actually give degrees and they have ensured that the degree does not fall below the quality standard. But otherwise it’s not been a great experience.

On entering politics and Swaraj Abhiyan

SS: I don’t really know, aside from you, of any psephologists or political analysts — I can’t think of any in the American context, I wouldn’t know as well about the Indian context — who decided to enter politics, after spending many years or decades of doing psephology. I’m curious to know what motivated your decision to enter politics.

YY: One exception. There was someone called Biplab Dasgupta. If you recall, one of the first books on Indian elections, voting trends, etc. was written by Biplab Dasgupta and Morris-Jones. I think it’s Trends and Patterns in Indian Politics. Something of that kind, it’s called. Biplab Dasgupta went on to become a member of parliament from the Congress party. Not that there was any relationship between his research and his politics. But he did. So I’m not really the first one.

In my case, it got much more noticed. I do think that [there are] many social scientists who went for active public or political life in India. In my case, it just happened to be more noticed, partly because I was somewhat known on television and partly because of the dramatic context in which I entered. I don’t think I was doing something else exceptional, but the notice that I got was exceptional.

Also, in my life, actually it’s not much of a transition, as colleagues in Lokniti will tell you. All those years when I was in Lokniti, when I was at CSDS, a full-time academic, I wasn’t there in the sense that I was actually quite torn. I used to spend quite a lot of time even then traveling in different parts of the country, meeting social movements. I was with a very small political group even then called Samajwadi Jan Parishad, which was led by my political guru, Kishen Pattanayak. So that was very much a significant slice of my life even then. It’s just that it was below the radar. No one quite noticed it. Not that it was anything secret. Everyone knew about it. But that politics was so spectacularly unsuccessful that no one cared to know more about it. But I was very much involved in a certain form of social and political action all these years of being in Lokniti.

And, as some of the colleagues would tell you, I was always very, very uneasy. I never wanted to be a psephologist in my life. I was, if anything, awkward and embarrassed about it. It wasn’t my life. All this might sound odd because I am known to be a somewhat successful one. But this is what I never wanted to be in my life. And I used to say to my colleagues… initially, they thought this was all pretension; academics have all kinds of pretensions. I used to say, “I am a political animal who has strayed into political science. And I want to get out.” People didn’t believe me because they thought this is posturing. I also used to say that I shall not retire as a professor. This is my nightmare. My nightmare was to retire as a professor at the ripe age of sixty-five or something. I never wanted to do that.

What I had not planned was, of course, the Anna movement and all the developments that happened. But, in fact, my very entry to CSDS in ‘93 was a bit of an accident. Because that is the moment when I had made up my mind to quit academia. I had done political theory. I had done intellectual history. And I was so unhappy with academia that I wanted to quit and go to my village. So in my journey from Panjab University, Chandigarh to my village, I got stuck in Delhi. I got stuck for twenty years. But I saw it as a diversion. So strange though it might sound, psephology was never, never number one priority one in my life. It was always number three or number four. Even in my intellectual pursuits, psephology was number two for me. Number two or number three. In my life pursuit, it was number four or number five. It so happened that it took substantial time and, in terms of public perception, this was the only thing I was known for. But I never associated myself with that.

So when I finally stepped out, I felt relieved. And actually I had said six months before joining politics, I had said, “No more forecasts ever for me in life.” This Uttar Pradesh assembly election in 2012, which we fortunately got right. And that evening on television, I said, “Thank you very much. End of my election forecasting. Never again.”

This business of election forecasting is something which attracted me and it did not attract me. It attracted me because it’s a kind of an itch. There’s a mountain and you want to climb it because it’s there. So election forecasting was like this. This is a technical riddle. Someone needs to solve it. And I knew at least two-thirds of it, of how to solve it. The remaining one-third never came. And I thought, if it’s actually possible to do it, why not do it? So there was a sort of technical riddle-solving part in me, a bit of mathematical curiosity, and a bit of political common sense, which I wanted to use.

The only thing I regret is that ideally, I would have liked to spend five years on this, solve it, and move beyond it. I spent fifteen, and didn’t manage to solve it fully, because there wasn’t any… I wish there was a group of model-builders that I could work with and a lot of background research on survey methodology. I think we were really one or two years away from having solved it properly. But we couldn’t do it fully. We did somewhat better. It wasn’t a record to feel ashamed of. But I think we were on the verge of a technical breakthrough which never happened. That’s the only thing I regret. But I don’t regret moving beyond that at all.

I was a political animal. My principal identity is that of a political animal. And what I’m doing now is what I really always wanted to do with my life. So no regrets.

SS: Was there a moment in 2011 or 2012 where you said, “This is my opportunity to get back to my original calling and step away from psephology?”

YY: Yeah. I mean, in a sense, 2011, the Aam Adami movement and thereafter was… as I said, I was very much in politics theoretically, I mean, notionally, because I was with a political group. I was very much writing. All those years of being in Lokniti, I ran an activist magazine in Hindi. I was the editor of a political magazine, which was read by very few, but it was an ideological magazine. I ran that.

I always wanted to be 24/7. Something in that context told me that yes, this is the moment to do it 24/7. So that’s the difference.

SS: I read a little bit about Swaraj Abhiyan online. I was talking to some of your assistants about it and trying to understand it. Why try to change India’s politics through a grassroots organization like Swaraj Abhiyan as opposed to through a political party?

YY: There’s no opposition between Swaraj Abhiyan and a political party because Swaraj Abhiyan is a political movement. It is not an NGO. It is not simply a social movement. It is a movement which very much aims at creating and being a political alternative, doing alternative politics. It’s just that at this stage of our evolution we are in no position to offer that alternative. We have neither the experience nor the critical mass nor the carefully-developed policy perspective. And that assurance that we would be able to create a democratic organization. There are several prerequisites which we are unable to fulfill.

But in principle I see no reason why we should shy away from doing party politics. Doing party politics is extremely important work. And although I would say that it’s important to do things other than party politics as well, party politics needs to be supplemented.

I keep saying there are five elements of politics: there is electoral politics, about forming governments; there is politics of struggles, movements, agitations; there is politics of constructive work, social, constructive, positive, good work; there is politics of ideas; and there is politics of the inner self. All these five need to be done. And the best politics is something that combines all these five. Unfortunately, we have reduced politics to parties, parties to elections, elections to government. We need to move beyond that. We need to think of politics in a more comprehensive way. And that’s what we would like to do through Swaraj Abhiyan.

So it’s not in opposition to a political party, but, in my ideal universe, Swaraj Abhiyan would do a politics that does not shun but subsumes electoral politics.

SS: When you talk about a “new politics” — this is something that’s all over the website and your assistants were talking about it as well. An “alternative style of politics.” What kind of politics are you talking about?

YY: As I said, a politics which is multidimensional, which subsumes what political parties, what NGOs do, what universities do, what agitations do, and what spiritual gurus do. All of that should be subsumed in politics. The best of politics has always subsumed all these. Mahatma Gandhi’s politics subsumed all these five elements. Which is what we need to do. Politics has to be a much bigger activity of transformation, including transformation of the self.

SS: I don’t know if you’re comfortable speaking about it, but why is Swaraj Abhiyan not working with AAP? I know a little bit about your history with–

YY: With AAP?

SS: Yes. Another way of–

YY: What went wrong? Why are we not with AAP?

SS: Yeah. Exactly. It seems like you–

YY: That’s too well-known a story for me to recount again. Not because I don’t want to talk about it. It’s just that I’ve spoken hundreds of times about it. And the story is so much out in the public domain. All I can say is that there’s nothing which is hidden from [the] public. There’s no hidden story here. Every single thing is in the public domain. When it started, why it started, where it reached. I mean, it’s so much there that it’s a waste of time to talk about it.

SS: I’d like to know how Jai Kisan Andolan fits into the broader vision that Swaraj Abhiyan has of a new politics.

YY: As I said, new politics has to have not just electoral politics in it. It has to have movements and constructive politics. And politics of idea. In all these three respects, the question of the farmer, the land question is the biggest question in India today. Land understood not merely as the land, the piece of land, but land as in the original economic sense of the land, which is to say all natural resources. That is the biggest political question, and farmers are the most neglected social group in our country. Neglected by governments. Neglected and despised by intellectuals. By social scientists and academics as well. There is a deep, deep apathy, if not hostility vis-a-vis farmers in the ruling establishment and among the academics as well.

So in terms of generating ideas, in terms of creating constructive work, and creating political movement, [the] farmers’ question is the biggest question. So we thought we must pick it up. Whether it is electorally viable or not, doesn’t matter. So it is a beginning of a very long-term engagement.

And for me particularly, I come from a village. As I said, twenty years ago, I wanted to go back to my village and be a farmer. That’s what I wanted some twenty-five years ago. So it is now, in terms of my life trajectory and life mission, it is invigorating to connect to that mission and to be able to articulate [it]. The trouble is that farmers cannot articulate farmers’ voice. You have to some middle class connect and some middle class articulation to be able to articulate farmers’ voice. And I happen to have that. And I thought that’s the best use of my time.

SS: [In] your article in The Hindu yesterday, you urged caution about celebrating too much the victory of the mahagathbandhan [in] Bihar. You say, “Much abused anti-Congressism is replaced by a vacuous anti-BJPism.” You describe this in this final line of the editorial as a “call to action.” I’d like to know what kind of action, specifically, you mean.

YY: That’s what Swaraj Abhiyan is all about: creating an alternative politics.

Anti-BJPism is dreadful. It is vacuous. It is poor politics. It is dangerous ideational activity. It is laziness. [The] BJP and Mr. Modi represent a very serious danger to the very idea of India, but that cannot be met by anti-BJPism. We have to understand why Modi came where he did. And we have to overcome that. Going back to the very same forces which are responsible for Modi’s ride in the first place is not a solution to that. You have to create an alternative. Alternative in terms of ideas, alternative in terms of political energy, alternative in terms of organizational protocols and principles. Which is what we want to do in Swaraj Abhiyan. So that’s the long-term solution.

And these short-term ways of trying to combat [the] BJP are suicide. So yeah, you read me right. I think this is suicidal politics. Not just poor, it’s suicidal politics of the Indian seculars. This is an admission of defeat, an admission of their incapacity that they are doing what they’re doing. This is certainly no way to respond to the very significant challenge posed by Mr. Modi. This complacent belief that somehow he will self-destruct and make way for the same old forces to come back is a lazy belief. It’s a dangerous belief. And even if this were to come about, it would do no good to India. That’s exactly what Swaraj Abhiyan is about.

SS: Thank you very much.

Praveen Rai on the challenges of polling in India and why he thinks seat predictions should be banned

Praveen Rai

Praveen Rai is the Academic Secretary at the Centre for the Study of Developing Societies (CSDS) in Delhi. Prior to working as Academic Secretary, he was a Project Manager at Lokniti, Programme for Comparative Democracy, a research programme of CSDS that specializes in election studies (full disclosure: CSDS is my primary research affiliate, and I am working with Lokniti on my research). While at Lokniti, he handled more than fifty election studies and opinion polls between 2005 and 2009.

His research interests include electoral politics, monitoring election opinion/exit polls and political participation of women in India. His seminal work “Electoral Participation of Women in India: Key Determinants and Barriers” was published as a special article in Economic and Political Weekly in 2011. He also co-authored a book, Measuring Voting Behaviour in India, with CSDS director Sanjay Kumar that was published by Sage in 2013. His articles have been published in Indian academic journals, edited books, newspapers and blogs. In much of his writing, he has been critical of the use of opinion polls as tools of political communication.

Last Wednesday, I sat down with Praveen in the CSDS lounge to talk about his experiences managing polls with Lokniti and his criticisms of media reporting on opinion polls. The transcript of the conversation below has been lightly edited for length and clarity.

On his involvement in public opinion research

Sam Solomon: Please tell me about about your background conducting public opinion research in India and how you got into this field.

Praveen Rai: I am basically an LLB. I have done my master’s in history from Delhi University and my law [degree] from Delhi University. I have worked in different private companies, small and big, looking after legal affairs. But somehow that job didn’t interest me, being a very routine thing. So I ha[d] been trying to make a career switch.

I just got a chance to come to Lokniti while I was looking for a job. I was just working here as a research assistant. But while working on that slowly I developed interest in opinion polling.

SS: You didn’t have the interest before though.

PR: No. In fact, I had been interested in politics and looking into election surveys on television and newspapers. That used to interest me. But I had never thought of getting into this area.

So I came here as a research assistant on a very temporary contract looking for a job, making a job change. And slowly I got into this. So I started right from the ground by doing the first time exit poll, which went wrong for me, as a field investigator. And then I did a few more fieldworks, and then slowly I got into questionnaire designing. So with a period of time I got into this field more by default in the sense that that was not my job. But I started learning , right from questionnaire designing to conducting fieldwork to managing large projects, and then data analysis and all that. So slowly I got into this. I started writing about it. I got an opportunity at Lokniti. And over a period of time I developed an interest in this.

So I think I worked almost four years where I was the project manager to Lokniti, and I think I conducted more than fifty or sixty opinion polls. That included election studies and studies in democracy. And different kinds of opinion polls. The Prime Minister’s survey, to find out what are the opinions that people had of Prime Minister Manmohan Singh after his first term. In fact, during his term in three years. So slowly I got into that but then I learned the whole thing and it became an interest area for me.

SS: Over what time period is that? When did you start getting into Lokniti and CSDS?

PR: I have a very long association with Lokniti. As I told you, the first one when I came here in 2002 as a research assistant. So I worked there for six months, and as I told you I did field investigation, I did field supervision, about how interviews are done, how student field investigators do. And after that I again went into the private sector where I was handling corporate communications. But there also I conducted some election studies for some political parties also. That was a learning stage for me.

In 2005, when Lokniti got into an informal contract with CNN-IBN for doing all the election studies and different opinion polls, that was the time I got the invitation from Lokniti. In October 2005. Then I came here. But by that time I was managing projects, I knew how to do field investigation, supervision, and all those things. And from 2005 to December 2009 when I was here, I got a chance to get into questionnaire designing, data analysis, and then I started writing. Now I’m all about opinion polling right from field investigation to putting the reports out.

SS: You have an office at CSDS. Are you still affiliated with Lokniti?

PR: No. In 2009, I got into my current assignment, that is like work as an academic secretary. So my role is a little different. To look about academically and liasoning with funders. The reason why I got here was this is a permanent job. In Lokniti, that was a contractual thing. Every year it used to be renewed. So at the moment I am not actively involved in opinion polls in this sense.

But still I write. If you see over the period of the last four or five years, it has changed in a sense. Now I don’t do much quantitative writings. I think everything is finished. All my writings and book chapters. And then there was a book also which I wrote with Sanjay [Kumar, Director of CSDS]. it’s called Measuring Voting Behavior in India.

Since last year I started tracking opinion polls. The way opinion polls and findings– in fact, now ten or fifteen agencies do [opinion polls]. So five or seven agencies say this is winning, and five or seven agencies are saying the other parties is winning. That confused me a lot. So then I got into writing about opinion polls. I tried to find out, I tried calling them up to find out what is their methodology, where was the fieldwork done, do you have the questionnaires. But the response from the different polling agencies has not been good.

SS: Were some better than others at responding?

PR: No, except for a few agencies like AC-Nielsen and all, which are market leaders in a sense. They are market research agencies who do it in a very, very proper methodological way. In fact, the quality of their surveys are, I would say, as good as Lokniti-CSDS. But apart from that, most of the agencies won’t even spell out who are the people who did the field investigation, what exactly was their sample, was it in residences or on streets. So these are things which are not coming out into the public domain.

Opinions polls are mostly on elections. Apart from elections, you get very few opinion polls. Eight to ten, you’ll get, to see in country which are on other issues. That is mostly done by Lokniti. The agricultural studies, the youth study. Apart from that, most of them are doing [surveys] during elections. Round after round — pre-polls, post-polls, exit polls.

SS: You’ve been involved with Lokniti-CSDS for almost fifteen years, since 2002. I’d like to know how measurement of public opinion has changed over that time, for CSDS and more generally in India.

PR: I told you that since things are not in the public domain, we don’t have a forum where the oldest people who were involved in public opinion polling get together and share those things. Nothing of that sort has developed in the last ten years. Though there have been a lot of people who have been trying to take initiative, I think it is not going. So at the moment I can just speak about CSDS surveys, Lokniti surveys. So I will just speak about the time when I joined and how we changed. And before that we have the history which you can find out from our book where me and Sanjay, we have written a very long chapter on how the surveys in Lokniti have changed.

During my period, in fact, the changes which happened in Lokniti surveys [were] at the stage of getting all the questionnaires here, cleaning it up, data punching, and then doing the analysis. From 2005 onwards, slowly we started decentralizing data entry and data cleaning. We developed it in different states. So now what’s happened is that it’s not centralized in a sense. All the questionnaires from India don’t come here and then we don’t have to put in huge teams with deadlines to do all the data cleaning and data punching for the analysis. So we decentralized it.

Apart from that, also since the number of sample increased — if you see our National Election Study, the National Election Study’s sample increased — so what we started doing was that we had four or five questionnaires. Questions on elections and politics, we had ten or fifteen questions which were in all the questionnaires. But in different questionnaires we started asking people on different issues, like security issues, economy, and all that. That way, in fact, we had more spread of getting opinion about politics, economics, price rise, and all these things. So I think these are the two major changes that have happened in the last ten years in the Lokniti surveys.

But the common thread which runs right from the first survey until now, and that we haven’t changed, that is we still do face-to-face interview of sampled voters in their residence. That is one common thing that is there in the last forty years of CSDS surveys.

SS: Going back to the 60s.

PR: Yes. Starting with the 60s.

SS: My research is looking at sampling error and whether certain populations are more or less likely to be sampled and participate in a survey. I’d like to know whether, in your experience working on Lokniti-CSDS polls, certain populations were more or less likely to be sampled in surveys, because of cultural factors or socioeconomic factors.

PR: In fact, the best part of the opinion polling which you do on elections is that in the the last ten years, all the voter lists are available online. So your universe of the study is– what happens is that you get the list of the complete universe. And we have been using the random stratified sampling system. So what has happened is that we have tested it in the 90s. We have been using those and once we get our data then there are certain indicators. We have matched the profile of the voters which we have done in our survey with the profile that exists in the state or at the all-India level. We have government data on the gender breakup, on religion, on caste and community. By using our random stratified sample, by the end of the 90s we could see that in fact we had been including a very, very representative sample. I think the sampling method in fact does not exclude anyone and is quite representative of the population.

SS: Even with the sampling being properly inclusive, did you find that certain populations had a higher likelihood of non-response and not participating in the survey?

PR: Well, I think not. We don’t have data for that. No, in fact, we have data for that and there is no difference in the non-response based on caste, community, or socioeconomic [level]. Non-response has been for many reasons, like people not living in that area, or migrating somewhere, or the day you reach for the interview they’re out of their village, they’re not in their residences. But there is no difference in the sense that the non-response of rich people is more or the non-response of poor people is less. It’s more or less uniform.

SS: What about for men or women? For religious minorities?

PR: For that we also don’t see any differentiation because what we do for the Lokniti surveys — you know how the network operates — we have a network of different universities and they’re mostly students who are doing their master’s or their MPhil. They are trained in how to do interviews, how to handle questionnaires. There we keep a gender equilibrium, in the sense that 50% of the investigators are men and 50% are women. Female investigators mostly go and do interviews with females because in villages or even if you live in religious communities, like Muslims, for a male to go and interview is a problem. So we have been using female investigators for the interview so we don’t have any problem of non-response from a particular section like Muslim women. Even in north India where you have the purdah system, where a male cannot go and do a face-to-face interview, but definitely the females goes to do [the] interview. So we have taken care of this. I don’t think there is any high non-response.

SS: In your article for Economic and Political Weekly, “Fallibility of Opinion Polls in India,” you write about four types of errors: coverage error, sampling error, measurement error, non-response error. Which of these four types of error do you think presents the most significant challenge for Indian researchers? If you have specific examples you can refer to in your time working for Lokniti or CSDS, that would be most helpful.

PR: When I am talking about all these errors, then definitely it is there in most of the surveys. But you have to see how high it is. Even if you see a non-response error — if it goes to three or four or five percent — it doesn’t make much difference in the sample, because the non-response of people… It’s not like the non-response of male respondents are more or of female respondents are less. I think non-response error does not create much problem. I am talking specifically for the Lokniti surveys. After that, I’ll talk about the marketing or opinion polls. In fact, all these errors are within the limits and they cancel each other out and they don’t have [an] effect on the total sample.

But now as far as the market research opinion polls which are coming to the public domain in the last four or five years, because of which I wrote that article, they are not coming out with the proper methodology. They are not telling us exactly who are the people who did [the survey]. I think the main reason they are getting it wrong could be, one, is the sampling method. Because a face-to-face interview on a proper random sample is a very, very costly proposition in India with the kind of length and breadth of the country, and very interior areas. So I think one of the major areas where most of the polls are going wrong is on sampling.

Even if they get their samples correct and their field investigation correct, the second error could be the measurement error because getting the right vote share… [between] the time the vote share of a survey which you do and the election results are declared, the vote shares are within the limit of one or two percent. It doesn’t make much difference when you translate it into seats.

But the kind of results which we are getting — as I told you, five saying Aam Adami is going to win and five saying BJP is going to win in [the 2015] Delhi assembly elections. And it was a clean sweep. Just three seats for BJP. So there in fact it will be very, very hard to tell that because of these errors the surveys were wrong because with huge margins of the vote share, no survey can get it wrong. There comes in what I talk about, the vested interest of people in the media, their manipulating seats to create a bandwagon effect before the elections.

On the use of polls as tools of political communication

SS: In the article, you used pretty strong language, talking about how election surveys have been reduced to a “media gimmick.” If that wasn’t the situation in the past, why do you think that has happened and what can be done to change that?

PR: As far as the bandwagon effect is concerned, I think it was way back, around twelve or fourteen years back, when a series of surveys were done by CSDS and they used to come out with different report cards for MLAs [members of legislative assembly] and that was there in all the Hindi and English newspapers. A lot of people read it — we cannot say exactly what percent — but most of the people were aware of this survey. In fact, it came out daily for one month. So in the last round of surveys when they did all these MLA report cards, we had put a question, how many people have read about it? We were very surprised that just seven or eight percent of people had read about all those MLA report cards. And it was coming out in all the Hindi newspapers and English newspapers. But only six or seven percent were aware of it, the MLA report cards in the survey. And those who had read about it or heard about those polls, we asked them another question, what effect did those readings have on voting for a particular party? Did it change your original decision? They said no. Whatever decision they had taken was taken by us and the poll had no effect. The findings of our poll had no effect.

This is way back, I think, 2002. In the last eight to ten years — in fact, I am talking about pre-poll surveys just before the election — when you have a series of elections, say, OK, BJP is going to win, Aam Adami Party is going to win. Even Aam Adami Party, they also did a survey for the first [2013 Delhi assembly] election where they had spoken about getting something more than fifty seats. I think they got something like twenty-three or twenty-four seats. There also they had used survey findings done and Yogendra Yadav had joined the party at that time. They were saying that they were going to get more than fifty seats. And on the other hand, Congress was going into the elections. Because they were the incumbent party, they were saying they were going to get a majority. [The] BJP did their own polls and also said, “We are going to get a majority.” Now these are the three main parties and all three are doing that. But when the election results came out, the BJP had thirty-three seats, [AAP had] twenty-four seats, and Congress was completely wiped out [Editor’s note: In the 2013 Delhi legislative assembly elections, the BJP won 31 seats, AAP won 28 seats, and Congress won 8 seats, leading to an AAP-led government with the support of Congress].

So that means all the surveys that they did and all the findings which [were] put out, those were just to solicit votes. That was to create a bandwagon effect, that, “OK, now we are winning.” In fact, when all these reports were out, they were putting out banners and all those things. They were doing a lot of publicity.

SS: But based on that and what you just said, it seems like there isn’t a bandwagon effect then.

PR: No, this is something conjectural until the time we do a study to find out that people are aware of those findings and based on what they’re finding, “OK, Aam Adami Party is going to win and then all the traditional BJP supporters are voting for that.” So this is just conjectural because a study has to be done and definitely none of the polling agencies are going to do that. So I think the only one who can get into this and do a proper study to find out whether we have a bandwagon effect or not is the Election Commission of India. So that is what I have been telling them. Just ban seat predictions. Seat predictions. I’m not talking about banning surveys. Surveys are very, very important. Election studies are important.

The reason why I wrote in EPW is the fact that the purpose of doing election surveys from a instituit . Political parties have also been doing that. But the purpose is now to find out what is the vote share they are going to get. Will this candidate do better? So they are using polls to find out how they are placed, how they can do that, what kind of candidate should do that. So the purpose was that.

So I think until that stage, until the stage where the Vajpayee government was there, before that, in fact, they had a minister called Pramod Mahajan, the first to come out openly and admit, “OK, we also do opinion polls but the purpose of doing our polls is to find out how we are placed in a particular contest, what kind of candidates we have to put in.” He was the first politician to admit that they do their opinion polls.

Until that stage, that’s not a problem. But doing an opinion poll, fudging data, putting out reports to say, “We are going to win,” that is not a purpose of opinion polls. That is not a purpose of election studies. And that is basically telling lies. The purpose of opinion polls in India were to find out the mood of the voters, like what kind of assessment we have of the incumbent government, what are the issues they think are important, and whether they have formed voting decisions until now or whether they are going to make voting decisions once the campaign ends. That was how opinion polls developed since the 1990s.

But I think over the last five years now, whatever they are doing, the most important thing is to just tell the people who is going to win. So now they are using it as a political communication. That is giving an extremely bad name to the polling agencies. And though we [CSDS] do a different kind of work — our purpose is to do election studies, do evidence-based reporting of how a party fared, do post-mortem analysis, and [make] data available for researchers who come in — at the moment everybody is clubbed in. Lokniti did seat projections for a couple of years and we dropped them because it was not scientific and it was not working well. But still people feel that, “OK, CSDS is also saying that.” So people have now started looking at opinion polls, they think it’s all biased and being done by interests to further that.

On why seat forecasting should be banned

SS: You make an argument that seat forecasting should be banned. You think the reason is not just because there’s a lack of transparency with how they come up with the seat forecasts, it’s [also] that the parties are using these seat forecasts to generate some kind of bandwagon effect that will help the party. And it’s not being done using scientific examples.

I spoke to Rajeeva Karandikar, who used to do the seat forecasts for CSDS. He’s pretty open about how he does the seat forecasts. He doesn’t give you all the specifics, but he kind of walks you through the general principles. He wrote a big article for The Hindu Centre [on this].

If more seat forecasts were done in a way that a very detailed methodology statement was included with the forecast, do you think that would be okay? Or do you think it will still be a form of political communication that political parties would use to manipulate public opinion?

PR: What has happened with opinion polls is that seat forecasting has taken the center stage. If we get a report out in a newspaper on an opinion poll on elections, the only figure that people want to see is who is winning, how many seats. As I told you, whether it has a bandwagon effect or not, most political parties think that it is a part of a communication campaign. So they have started using this for doing it.

I have [said] that the Election Commission has taken a very, very harsh view in the last five years. Nothing much has been done. A time had come when they were talking about banning election surveys, in fact, before the elections. Now they have a time period. Once the polling process starts until it ends, that is a time you can do your survey but you not come out with any reports. But just before the election process starts, you can do as many polls as you want to do, as many seat projections. Instead of banning election surveys, what I have argued is that seat prediction should be banned or else the other alternative can be what happens in Japan.

In Japan, seat projections have been banned. What had happened is that earlier they used to do seat predictions but when it went wrong, people went to the courts, the political parties went to the courts and filed a case, and people who did wrong forecasting were penalized heavily. The courts there said that if you get your predictions wrong, you have to pay for that. So seat projection doesn’t happen. At the best, they do a lot of election surveys and opinion polls, but they talk like, “BJP is leading and Aam Adami Party is second or third.” They talk like this, but they don’t assign any specific figures about, “This is the number of seats they’re going to win.”

So I think that would also be one way to deter media. In fact, if a political party does something and then does it through media, you know, an India Today poll… What happens is you’re masking your poll. Ultimately [the] media has to be made accountable, because television is  a very, very important way of reaching the homes. And elections, politics in India, it creates more passion as compared with other countries. (laughs) People are political in that sense that they are interested. And when a media house puts out something it’s like putting a stamp on that survey. So either you put a huge penalty that if your prediction goes wrong — you cannot just say that BJP is going to win a two-thirds majority and when the results come out you find that the opponent side won [a] two-thirds majority. You can’t be so wrong. Either you impose a huge penalty on the polling agency and the media house which is showing it so that they stop doing all these gimmicks and all these bandwagon effects. Or the second and easier method could be that we just tell them which party’s vote share they can talk about, which party is leading, which is winning, but don’t give out those numbers.

SS: But don’t you think that the vote share would then become… It seems to me that even if you banned seat projections, the political parties still might try to gin up bandwagon effects by saying vote share: “According to this CSDS poll, we have 45 and these guys have 41.” I feel like vote share could also be a form of political communication, even if the seat share were banned.

PR: No, I agree with that. Once you ban the seat projection, people can come out with the vote share and they can talk about it, “This party is going to win the election and get a huge majority.” All those kinds of things will happen. So I think there has to be some body, the Electoral Commission of India has to set up one unit whose work would be to see what kind of media reports are coming out. And if somebody says that, “NDA is going to get a majority, and BJP will  get a majority but they will be the largest party,” but after the election you find the BJP gets a majority on their own. That does not amount to much tinkering of data or misuse of election surveys.

But you can’t do a projection that NDA is going to get a two-thirds majority, and when the results are out, you find that the opposite party has got that. So when they do that, I think the Election Commission has the right to get all the data and find out whether the survey or fieldwork has gone wrong, the vote share they have computed has gone wrong but the survey is correct, the vote shares are correct and they have manipulated the data, and based on that some punishment needs to be enforced if they want to restore the faith in election studies back. Because it has taken a very, very bad beating since the last 2004 national elections, they are getting it wrong.

And even if you see this election [2014 Lok Sabha elections], because I did not do a follow-up article on this, here also in fact nobody spoke about a BJP majority. They all said NDA will get a majority. But nobody could in fact that BJP is getting a majority. So that means the vote share which you got in your survey which you are showing, there’s some problem. And the problem can be of fieldwork, of data collection, wrong sampling. At the moment, we are just grappling in thin air. Whether the fieldwork is wrong or the fieldwork is right and people are manipulating at this stage, the media are manipulating. So I think the responsibility needs to be fixed, either with the polling agencies or the media houses.

SS: And do you think the responsibility for changing this, the culture around polling in India, that falls on the Election Commission of India?

PR: That’s the body which puts a stop to election studies because once elections are declared, most of the powers are with the Election Commission of India. The last one which we had, whatever orders they had were [carried] out. As I told you, the day the election notification takes place, from that day until the day the last vote is cast, that is a period they have said you cannot come out with a report of opinion poll. Even that is ambiguous in a sense. You can do your survey but you cannot come out with those reports. And I think that is being followed by most of the polling agencies and media houses.

So I think the Election Commission of India should get a study done to find out the bandwagon effect, so that it can tell the political parties, media houses, and polling agencies, “Your reports [don’t] make a difference.” Though I am just being conjectural. Maybe it makes an impact on the voters and some voters are changing because of the election surveys. And based on that the responsibility needs to be fixed so we have fairness and transparency in opinion polling to restore its image. Because at the moment it has taken a very, very bad beating.

SS: Have you heard about the Indian Polling Council initiative that’s supposed to be launched in the next month or so? There was an article in The Hindu about it.

PR: I am not aware of it.

SS: In the article, some pollsters — in the article, Yashwant Deshmukh and CVoter are the ones that are quoted — CVoter and five other polling agencies are thinking of coming together and starting their own group to come up with some standard guidelines for disclosure of methodology and transparency in terms of reporting on their polls. The idea is that if you belong to the Indian Polling Council, you have to report things like sample size, sampling methodology, etc.

PR: If this is happening, I think it is a very good initiative in a sense because what I feel is that instead of the Election Commission of India drawing guidelines for do’s and don’t’s for polling agencies, if they all come together and form their own thing where they share their things, and they can come out with reports, I think it would go a long way of restoring the credibility of opinion polls in India. So it’s a good initiative and if it happens, it will definitely help.

On the challenges of polling in India

SS: In your own experience — you’ve worked on fieldwork, you’ve worked on questionnaire management, you’ve worked on all the stages of opinion polls — what are the biggest challenges that you found in accurately measuring public opinion in India?

PR: As far as challenges are concerned, there are a lot of challenges in the sense that this whole face-to-face interview is a very, very hard task. There are villages where you have to walk for kilometers on the hills to reach them. Lokniti does it, but the cost involved of doing a face-to-face interview is very, very huge in this country. That is one of the biggest challenges, and I feel for most polling agencies they don’t get enough funds to do this.

Secondly, doing a telephonic interview in India is still not possible because the mobile phones’ reach has increased but still most people don’t have the telephones. So making a shift from face-to-face, compared with US and other European countries where face-to-face interviews are very, very less and most people do telephonic interviews and now using different platforms [the] Internet. So I think this presents a huge challenge.

Apart from that, as you know we have a very, very diverse country. Different languages and all that. So if you do a survey at an all-India level, like Lokniti, we translate a lot of questionnaires into state languages. For most of the states, we are doing that. But for other polling agencies, we are not sure whether they are employing, like if you want to do a survey in Karnataka, you need to have a researcher from Karnataka and you need to have the survey in the local languages. That is one of the big challenges and this is also [a] huge cost. So whether these polling agencies are doing it or not, that I am not very sure about it. At least Lokniti does it, but still it’s a major challenge because even if you see a language, even in a state, there are so many dialects, that translating the questionnaire into dialects and getting the investigators with those dialects to do the interview, that is a huge challenge.

In fact, these are the most important challenges of doing an opinion poll in India.

SS: Anything else you’d like to add?

PR: I’ll add an example of the kind of the challenges you face. As I told you, from 2005 until 2008, for three years Lokniti-CSDS, we did seat predictions. And that was mostly for assembly elections. So I think we had nine assembly elections during that time. We got seven right, the [election results] were exactly within the range. But we got one completely wrong and one partially wrong.

The election had taken place in Punjab. There also the fieldwork and everything went off well. We were using exit poll data. With the exit poll, what happens is that the moment the polling ends, say the polling ends at 5 o’clock, the television program has to go live exactly at 5 o’clock and by 5:30 you have to tell exactly which party is winning. So that means that you have to close your fieldwork by say, around 3 PM, because then you have to transmit data and all those things, and analysis has to be done, seat prediction has to be done. You get just two hours. So with Punjab we did that. It stopped at 3 o’clock and we got our data. We did the seat prediction and all.

But what happened is that the election was extended from 5 until 9 PM because there were heavy rainfalls that took place. Here all the predictions and television programs were on since 5 o’clock. We collected our data until 3 o’clock. And then the rain stopped at around 5 or 6.

Congress and the Akali Dal, they were in a very, very strong contest, just a margin of two or three percent. In fact, the Akalis were winning. So what happened is the rain stopped. A lot of Akali voters, they put their voters in their buses and they went and voted at 9 o’clock. Now for the exit poll we closed our field investigation at 3 o’clock. So our data was loaded in favor of the Congress. So we mentioned that Congress was going to win. But actually it was Akali Dal which won.

So we predicted something, it went wrong. We did a fair investigation to find out how come we got it wrong. Even for exit polls we do purposive sampling to make it representative. So we missed the whole, the Akali Dal voters go when the rain stopped at 4 o’clock. They went to the voting. And the voting was extended until 8 o’clock in the evening. That also, people didn’t inform us. So a huge section of people in a lot of places voted at that time. It’s a very strange thing. With all the Akali voters, what they do is that they get buses and tractors and all those kinds of things, where they go together and vote. So this is one challenge which nobody foresaw. And we got it wrong.

SS: What year was this?

PR: You have to check. This was Punjab elections. I think, Punjab elections 2007 or 2008. You will get that if you do a search [editor’s note: Punjab Vidhan Sabha elections were held on 13 February 2007].

SS: That was the one seat projection that you got way wrong, and there was another one which you said you got partially wrong.

PR: The other one where we couldn’t predict, I need to check. Was it Tamil Nadu? No, Tamil Nadu, we got it right. The other one I don’t remember. But two we got wrong. In the third, also, in fact with Tamil Nadu, what happened, as I told you, is that you have some error, the coverage error. There was some errors. But what happened is that the errors for non-response and the errors for non-coverage, both cancelled each other out. So we got the vote share correct. But if you see the survey, and if you see the initial data that was coming in, there was some problem in fieldwork. We couldn’t get the fieldwork done properly. It was a problem with investigators, with fieldwork, with supervision. There were logistical problems. We did not get the vote share. But one mistake, cancelled by the other mistake, and we got the vote share correct and we made the right prediction (laughs).

SS: And what was this for?

PR: This happened in Tamil Nadu, this was also between 2005 and 2009 [editor’s note: Tamil Nadu Vidhan Sabha elections were held on 8 May 2006].

SS: For Vidhan Sabha?

PR: Yeah, for Vidhan Sabha. Our prediction was perfect. But if we see our data, and what had come, and what were the errors, as we do a review of all our surveys once we do [them], and mostly during seat projections.

So two wrong, and third one we got it right by default. So we decided that doing seat predictions and vote shares, it’s a very, very tough situation in India. And so we stopped. I think 2008 we stopped. 2009 Lok Sabha elections, we did not make any prediction.

And then there are some interesting cases. In one of the Karnataka assembly elections — not the last one, the one before that — I very well remember that Congress had got 32% of the vote and [the] BJP got 30% of [the] vote, but [the] BJP got the majority of seats and they formed the government [editor’s note: In Karnataka’s Vidhan Sabha elections held over two phases in April 2004, Congress won 35% of the vote share and 65 seats, while the BJP won 28% of vote share and 79 seats].

SS: Because their vote share were spread out in a way that was more advantageous.

PR: Yeah. The Congress vote share was completely spread out and they were concentrated in different assembly constituencies.

I think that was also one phenomenon that we first saw in Karnataka, because before they we hadn’t seen in any elections, having more votes but getting less seats. Karnataka gave us that lesson.

And apart from that, if you see the case of Bihar, so [in] Bihar they were completely new political coalition formations. If you see Bihar, first of all, there were so many parties, new coalitions. When you have new coalition partners, you don’t have any record of the previous vote shares. You have nothing. And how that coalition works on the ground, the vote transfers and all that. That is very, very difficult to ascertain through an opinion poll. Until the time that you don’t have some quantitative study done to substantiate your quantitative thing, you can never get a seat prediction.

Where there is just two main parties in contest like it happens in Tamil Nadu, a couple of states, it happens in Rajasthan where [the] BJP and Congress is there, it happens in Uttarakhand, there once you get a vote share [which is] correct, like 45% for Congress and 38% for [the] BJP, you can easily do a projection based on that. And mostly you get it right.

SS: But it’s more difficult when there’s more parties, like Bihar or UP [Uttar Pradesh].

PR: These are the challenges, and how the model is going to take into account these things, for that, we say that until the time polling agencies don’t come on a single platform, they don’t share their prediction models, and if they’re not going to improve upon that, they are going to be more and more wrong.

SS: And for the Karnataka poll where Congress got a larger vote share, but BJP got a larger seat share, did CSDS do a seat projection for that election?

PR: I think we did that and maybe that was one survey we got wrong.

SS: You had the vote share right but the seat projection didn’t predict that [the] BJP would end up winning more seats with less votes. OK, I’ll check that.

PR: That you can check. I think we have it on our record though I don’t have it because we did a complete check of our seat predictions.

It started in 2005, exactly with the Bihar elections. We got it right.

SS: The seat projections.

PR: The seat projections. For all the political formations, we got it correct. We got the vote shares correct. That was the first seat prediction we had done that was really celebrated for getting it right. And then it went on for a couple of years. It was 2008 or something, I think, when we got two or three wrong.

SS: This was the year of Punjab and Karnataka.

PR: Yeah. Punjab and Karnataka.

SS: Wouldn’t you say, if you got seven out of nine right, that’s a pretty good record, right? Maybe you think statistically because of some of the errors, because some of the challenges of polling, you’re going to get a few wrong, but the overall track record of seven out of nine, or whatever it is, that seems pretty good. I guess it’s a matter of focus on the fact that we got seven out of nine right, or we got two out of nine wrong.

PR: No, but see, if you see the pre-poll surveys which we have done — at the moment we are just doing post-poll surveys — you can see the data that we have got in all the pre-poll surveys. You will not find a single poll where you find the actual vote share in that election — if Congress got 40%, then we have got 30% — it was never so wide. Our vote share would come to around 38%, or maybe 1% more. It has always been in the range of 2%. Plus or minus 2%. In fact, the vote share has never swung more than that.

SS: Is this for post[-polls] or pre[-polls]?

PR: Pre-polls. Just before the elections. In fact, we have got it within the range of 1-2%. So you can just imagine that we have got the vote share 95% of times right. So that means that the sampling method which we are doing, the field investigation we are doing, our systems are perfect in the sense that we get it right.

But even with those right vote shares, when you do a seat prediction — as I told you, we got six or seven right out of nine. One or two we got it right by default. By default, one error cancelling the other error, as I told you. That means that that system is still not completely developed. Either you do a lot of research and development, when you can put in all these variables, like concentration of votes and all these kinds of things and different vagaries different coalitions. Either statistically over a period of time, and if somebody does that, but at the moment I don’t see anyone.

Because even if you see a new polling agency which just came up, its name is Chanakya, Chanakya got it right in 2014. You will not believe. They got the seats right for BJP, okay, and all the polling agencies missed it. Everybody missed it. And I think everybody laughed also again. BJP will get a majority. Everybody was laughing. “Who is this Chanakya?” and all that. And then Chanakya got such a big name. And then again have started floundering. Delhi, they have got it completely– Bihar, they have got it completely opposite. Now people are laughing. They are saying that Chanakya by mistake, instead of putting UPA, they are putting NDA with some grammatical mistakes [editor’s note: Today’s Chanakya exit polls showed an NDA landslide. Following the announcement of election results, they said their prediction was off because a computer error had mixed up the names of the alliances]. (laughs)

SS: Yeah. I saw that. Interesting excuse, yeah.

PR: Prannoy Roy is one of the leading psephologists in this country. He was the one who started doing opinion polls through his channel, and all the swing votes, and all that, he made it so popular. So I think Yogendra Yadav and Prannoy Roy, not only are they the best political analysts but they are the leading psephologists. And the way Prannoy Roy got it wrong this time, he had to render an apology. So now this really calls for a question. In fact, people should ask. The media should go and ask him, “What kind of a survey [did] you [do]? Your survey was wrong.”

Because there are a lot of stories going around, and they were trying to show this, push this, that should [the] BJP comes in power, they have all these taxes left and these debts will be paid. They are getting pressures from above.

SS: It sounds like CSDS has a pretty solid track record. You’ve gotten a few seat projections wrong. And maybe that’s because of the methodology, because of the focus on post-polls rather than pre-polls. It seems like when you do pre-polls, you try to do them as close as possible to the election as you can.

PR: The whole purpose of doing an opinion poll for us is it’s a study. In fact, we call it election studies. Our purpose is that. The reason why we started going to the media is that there is a huge cost involved in doing this. Until the 80s, the government agencies provided funds, but after [the] 80s, there was no funds available for conducting election studies. It is still not a discipline in India. It is still a sub-discipline.

So when we had the tie-ups with the media, that helped us in doing our surveys and giving them the required data like they are just interested in the vote share, the popularity ratings of the leaders, those kinds of questions. But apart from that, using those funds, we have been doing a huge, a very, very in-depth study of that election. So the data is with us. So our purpose was that. And when we got into seat prediction, at that time [the] media said, “Since you all are experts in election studies, why don’t you get into this?” And we also went into that to see how can we refine, and the model which Rajeeva Karandikar, I think, is still using is the one that was developed at that time. Then we started doing the seat predictions. But once they started going wrong, we said we don’t want to do it. Because our purpose is not that. Our purpose is to have a set of data on each and every election that happens in the country.

SS: And to understand why the results are happening and not necessarily just predict them beforehand.

PR: Yes.

SS: Thank you very much.

Op-ed in The Hindu

Slowly returning to the blogosphere after a busy week and a half. I have an op-ed in The Hindu today on what pollsters and political analysts can take away from the Bihar polls. Today’s Hindu op-ed page also features a thoughtful piece on the long-term meaning of the Bihar elections by Yogendra Yadav as well as an examination of CVoter’s Bihar post-poll data by Yashwant Deshmukh and Manu Sharma.