Mukulika Banerjee on anthropology’s insights into politics, why India votes, and the upcoming West Bengal elections

Mukulika-head-shot

Mukulika Banerjee is the Director of the South Asia Centre and Associate Professor of Social Anthropology at the London School of Economics. Her published books and edited volumes include The Pathan Unarmed: Opposition and Memory in the North West FrontierThe Sari (with Daniel Miller), Muslim Portraits: Everyday Lives in India, and Why India Votes. She has received several prestigious grants and awards for her research.

Banerjee is currently working on a project entitled ‘Explaining Electoral Change in Urban and Rural India’ (EECURI) funded by the India-Europe research networking programme of the ESRC, ICSSR and ANR (2012-16). This project brings together a network of scholars at King’s, LSE, University of London, Sciences-Po, JNU, BR Ambedkar University (Lucknow) and the NGO Janaagraha to study changing patterns of electoral politics in contemporary India. Mukulika is directing a series of ethnographic studies of Panchayat and State Assembly elections between 2012-2015, including studies of elections in Delhi, Bihar, Chattisgarh, Gujarat, Jharkhand, Rajasthan and West Bengal. This research as part of a large-scale interdisciplinary research scheme (with partner institutions in Europe and India) that will uncover the reality of democratic governance in contemporary India.

Last year I reviewed Banerjee’s book, Why India Votes, on this blog. On February 9, I connected with her over Skype from her London office to discuss the unique insights anthropology offers to our understanding of politics, Why India Votes, and expectations regarding the upcoming West Bengal elections. The transcript of the conversation below has been lightly edited for length and clarity.


On political behavior through the lens of anthropology

Sam Solomon: I liked how your book used qualitative research methods — the team of ethnographers — but also incorporated quantitative data, including CSDS survey data. Why don’t we see more research like this? Or is this more research like this that I am not aware of?

Mukulika Banerjee: I don’t think it is arrogant to say that there isn’t research like this in anthropology. This was a real experiment. Because what you’ve had in anthropology in the last, say, fifteen to twenty years, is that political anthropology as a sub-field has really disappeared. It was seminal in the formation of anthropological theory in the 30s, 40s, 50s, 60s, in that period of the development of the discipline, when modern social anthropology really came to be. Discussions of politics and discussion of political behavior right up to the transactionalists in the 70s was absolutely central to the discipline itself.

Now, what happened? Partly because of the critique of transactionalism — and that’s interesting in light of what we are discussing, politics and elections — it was quite evident that transactional behavior in the way that anthropologists were describing it was really inaccurate when trying to understand how people actually behaved. And so that kind of pulled the whole sub-discipline of political anthropology down. Combined with the writing of people reading Said’s Orientalism, and the discovery of Foucault in translations, [this] meant that thinking about the world, and thinking about power and institutions, became absolutely organic to the way you thought about any institution in society. So you could think about power in the family and kinship as much as you did in political institutions and kingships or sovereign bodies. It was almost as if now that everyone understood how insidious and ubiquitous power was, you did not need a special sub-discipline to study politics as such because politics was everywhere. You had politics of the family, you had politics of kinship, and politics of the economy, of course. So what happened as a result is that the anthropological attention to political institutions as political institutions seems to have collapsed, and there was no systematic study of political institutions.

We were studying social movements. We were studying different kinds of interactions of post-colonial dynamics, of the remnants of indirect rule in sub-Saharan rule and South Asia. We were studying caste and politics. There was a lot of study of Indian politics, if you talk about India now specifically. But there was very little attention to political institutions, say, after F.G. Bailey’s book, who was working in Orissa. And Bailey, coming out of that Manchester school in anthropology, was interested in connecting his local village study to national politics. So he scaled it up. The way he did it was to study factions. Factionalism was the language of studying local politics in those days. He looked at how factions are formed, how leaders’ followings were maintained, what were the ingredients of good political leadership at the village level, then examined this phenomenon at the district level, and then took it up to the state level in Orissa. That was one model. I’m trying to think if there are exceptions to this, but within anthropology certainly the study of political institutions came to a grinding halt at that time.

When I started, I was very much trained in that tradition. I wasn’t a particularly keen election watcher. I was interested in politics, but my doctoral research had been in Pakistan on an anti-colonial movement against British colonialism among Muslim Pashtun men. That’s what had intrigued me. That’s what my earlier work was about. So I was in a completely different space. But CSDS was very central to the story. Lokniti was just formed, and the ‘99 survey showed this statistic which I talk about in the book. And people have rehearsed this many, many times. Now it has become so much part of our consciousness that we pretend that we have known it forever. But I remember the moment in ‘99 when the NES results showed, for the first time, that poor, rural, low caste voters are more likely to vote than their urban, male, high caste counterparts.

This was a startling moment. Yogendra [Yadav] had revived the whole survey tradition. He was building a team. He was reviving the longitudinal surveys that CSDS had been doing. And it was startling. There was a lot of excitement around this finding because we hadn’t realized this, that this was happening in Indian elections. And it was a conversation with Yogendra — I was just finishing my Pathan book at that time in ‘99 — and he said, “This is a really interesting finding. We have found this as political scientists. The survey’s done very well. We’ve sampled it well. It’s a good instrument. But we can’t explain it. We don’t know why we are getting this finding. Why are rural, low caste, poor voters more likely to vote? What is it that they invest in this process? That is something this survey simply cannot take on as a research question. And we need an ethnographer.” And I was just beginning to come out of the last project and looking for the next one. And I thought, “Yeah, that’s fascinating.” That is something which really animates my imagination.

The reason I’m telling you this background story is to explain how I came to study something so central to politics like elections as a result of the limits of survey data and survey methodology. It was a very organic relationship. I don’t think anthropology, brilliant as it is as a discipline — and I wouldn’t be anything else — has the capacity to pick up such a macro trend. That kind of macro trend where you can say in a vast country like India, we can say definitively that poor, rural, low caste people are likely to vote more. That anthropology can’t do. But once you find it, anthropology is very good in probing the questions. Because it’s subjective. It has to be long-term. It has to be qualitative. It has to understand people on the ground. Motivations can never be understood in compartmentalized ways. You can’t understand political motivations divorced from religious motivations divorced from familial motivations and so on. And anthropology is the social science par excellence to do that kind of holistic understanding.

SS: Something that struck me from the book — and from your answer as well — is that political scientists and anthropologists both study political and social phenomena, but the tools that they have and the vocabulary they have to understand these phenomena are so different. It seems to me that often times they are talking past each other.

I worked for a company doing survey research for several years. I worked on some qualitative projects where we were doing focus groups and in-depth interviews as well. So I got to see how different research tools provide different types of insights into whatever it is we are trying to study. And I see it at CSDS a little bit as well because CSDS has different research units. I’m working with Lokniti, and they’re numbers geeks, and they’re looking at SPSS outputs, and producing articles and journal pieces where they’re looking at trends. Then there’s Sarai [doing] media studies. Then there’s the Indian Languages department, and they have their own research agenda.

What struck me about your book is that it was an interesting meeting point where it seemed like it is possible for these different disciplines to talk to each other. Because I think they should. I have been to political science conferences that are very, very quantitative-heavy — this has been in the United States — and I’ve heard people say, “Sociology…what is that?” Or, “Anthropology? Come on!” And I’ve had conversations with people here — not anthropologists, but one person who is in film studies, and he was saying, “What can surveys really tell you, right? People change their minds all the time.”

All these different experiences have made me feel as though the different disciplines of social science are often talking past each other or not really trying to interact with each other. I wonder if there are some examples to counter that. I found your book was a way to bridge that divide.

MB: That’s really interesting that you should say that because I know the people you’re describing. And I can hear Ravi Vasudevan say that sentence that you just said: “What’s the point of surveys? People change their minds all the time” [editor’s note: I was not referring to Ravi Vasudevan in the above remark].

We are all wedded and committed to our silos at one level. The way universities work, the way associations work, you can’t go across. You can’t have dual membership. Forget multiple membership of different disciplines. You’re simply not allowed to do that. Anthropology always attracted me because when I moved to the Delhi School of Economics to study sociology… when I say social anthropology, in the Indian context it’s sociology. My counterparts in India are sociologists. The anthropologists tend to do more folklore of physical anthropology in India on the whole. Shail Mayaram at CSDS — Shail and I did our M.Phils together so I know her quite well — she’s an anthropologist in Britain and a sociologist in India.

I’ve thought about this a lot. I can speak only for myself. What I find most interesting is to be led by the question rather than by the discipline. It depends on what you’re interested in. Let me give you a completely off-field example. Have you seen my book on the sari?

SS: I haven’t.

MB: [walks to bookshelf, takes copy of The Sari down, and holds it up the screen] This is what it is. Can you see it?

SS: Yes.

MB: Then, if you open it… [turns pages of the book to reveal many beautiful photographs of women dressed in different styles of saris] It’s 300 pages full of color photographs and designs.

It’s always important what the research question is. I refuse to be wedded to any particular discipline. It is more a question of what is interesting to study and that’s why I’m a social scientist. Now, on the basis of a slightly personal experience of wearing saris to work in London and teaching British students, a student who actually taught at the London School of Fashion said to me, “How can a modern woman like you wear something that looks so antiquated and Victorian? And isn’t it interesting?”

So the question was: can the sari be modern? And then it was a question of how does one write something about this which is representative of India rather than of what the natural anthropological tendency would be: to go and focus on a particular location and try to understand it in-depth. At this point, I had already been doing about three or four years of research in these two villages of West Bengal where I’d been returning every year for the last fifteen years. I had a very good sense of what rural India’s engagement with the sari was, as opposed to the urban milieu in which I’d grown up. But rather than do what would be the classical anthropological study of going to one place, and then people say, “Well, that’s true of your two villages in West Bengal. What about Tamil Nadu? Or what about Gujarat?”, my co-author and I set up a random selection of women. It was randomized in that it was arbitrary, but it was very carefully designed to cover women from different professions, different castes, different classes, different regions, urban and rural. And on the basis of all that we wrote this book, making an argument for not only that the sari is modern but actually it’s the quintessential modern garment. That’s the twist we put on the story.

It’s a book that sold more than 10,000 copies, which for an academic book is being widely read. Even now it’s got a second life; the publishers got lots of course adoptions in the US, so now they’ve brought out a black-and-white version to make it more affordable for students. And it’s selling. People are reading it because I think it makes Indian modernity as an issue very accessible to a student or an outsider. You think clothing is a fairly accessible issue. Anyone can pick it up and think about it. So book clubs have adopted it. Within the discipline of anthropology, my colleagues wouldn’t be very persuaded that this is a scholarly piece of writing in the kind of standards we hold ourselves to.

Somewhere along the line, you’ve got to make a choice. There are two kinds of writing you can do: one is that the audience is entirely made of your peers in your discipline, which is scholarly in a very particular language, and I think inaccessibility to that writing by all surrounding disciplines as indeed the wider world is a hallmark of that kind of scholarship. It has to be a specialist language. We have to be incomprehensible in order to be taken seriously, in the way that economists don’t bother explaining to us, unless you ask them, what their language means. In the way that mathematicians don’t. I think social science aspires to that kind of inscrutability to be taken seriously. And there is a second category of writings which you can do, which is about communication and insight and understanding. Of being able to share the excitement of your genuine insight, which is gained through scholarship. I will never compromise the rigor of my research, but what you learn from it can be disseminated in so many ways. I think your blog is exactly that second category of communication, where you engage with writings really in-depth and seriously, but when you write about it, anyone — even your grandmother — can read it and learn something about what you’re doing in India, what you’re reading, and what kind of things you’re thinking about. That’s point number one about language and communication and audiences.

The second thing — when you were talking about political scientists in the US, I know many of them and our department of government of LSE is going more and more that way, it’s getting more and more quant — is that it depends on what you’re trying to understand. What is your object of study? As an anthropologist, I’m always, always interested in the ordinary person. So when I studied an anti-colonial movement, I was interested in their evolution. I lived in the house of the leader of the movement’s son. The leader had just died, which is what sparked my interest, but his son was alive. I stayed in his house, and I got into a real argument with him because I was not interested only in the story of his father. I was interested in the common Joe who never knew his father who was a part of the movement.

By the same token, the reason why the NES finding excited me so much was because this was not about political parties. This was about ordinary voters. And so much of Indian politics, as you know well, is dominated by politicians and political parties and factions within [them]. And the media are lazy as hell, so they think personality-driven stories are the most interesting. That’s all the discourse is about, whereas the whole system would be meaningless if it wasn’t for ordinary people engaging with it. So when this finding came out about the electorate, I said, “Great. Nobody else is interested in the electorate.” Or — actually, that’s inaccurate — the electorate becomes important to political scientists only in terms of who they’re voting for. So you need to count. So you’re counting votes, and you’re interested in what kind of person votes for BJP, what kind of voter is likely to vote for their caste party, or for women. But it’s always about who they are going to vote for, and I think political science goes a long way in that. I think the NES can be worked on a lot more than we do at the moment. We need to break this down much further and really interrogate the material, because I fear that actually the most interesting insights are counterintuitive. That’s why how you interrogate the material, the data itself, is so important. And I don’t think the NES data has been worked enough by a distance, but more and more people are doing it.

What I was interested in as an anthropologist was the finding that people were voting. Not that a majority of rural Indians were voting Congress. That we kind of knew. But the fact that they were voting at all was a question that, as far as I could tell — and you will tell me, because you have time to read and look around more — I didn’t see anyone else looking at that question. And that was the most startling, most interesting thing. In a sense, I started thinking about this in ‘99, and it’s been vindicated as a research question because as you know the turnout goes up every single election we have. Whether it is national, whether it is the Bihar elections, now women are outstripping men. Bizarre things are happening. It’s in such contrast to the state of the country in so many ways that it’s a real puzzle that we need to understand. So that’s what animated this book.

Therefore, if you are going to probe why people are voting, this is where an anthropological sense of ordinary people amongst whom one tends to live and work really helps. A lot of human activity would be classified by anyone working with quantitative data as irrational. Most of our human activity is that and that’s why things work. People are not maximizing all the time. That is why the world continues to function. I think voting in India is the irrational activity par excellence. It is not about maximization of ends. It is certainly not about increasing any tangible influence. It is not about a zero-sum kind of transaction where if I vote somebody else can’t vote; it’s not that kind of thing. All the instrumental reasons that are cited again and again — the assumption we make about poor people in India all the time that somehow people vote because they’re going to get something out of it — more and more research is showing to be completely untrue. Yes, people get a lot of things, but that has nothing to do with who they’re voting for or the fact that they’re there at all. All of those things kind of go out of the window, and all of the things which we unearth in the course of this study was about a whole set of ideas which nobody had frankly taken seriously up to this point. But it’s taking people seriously because it’s entirely inductive. I’m not making this up. It’s what’s happening on the ground and it needs to be reported.

On why India votes

SS: You chose a certain number of states to put your ethnographer teams at for the 2009 Lok Sabha elections.

I was talking about your book and your research at the CSDS office before I came here. In some way I was almost fishing for questions: “I’m going to talk to Dr. Banerjee. I’m trying to think about good questions to ask.” Someone put up a good question. They said, “There’s a certain reality of Indian elections for the states that her book has chosen.” We looked at the turnout data from different states for 2014.

Your work was looking at the 2009 Lok Sabha election. 2014 was obviously a very different election; a lot of people were talking about it as an election that changed India. The first question is: Do you think that you would revise or reconsider any of your findings in light of the 2014 election results?

The second question is related to the fact that the state that consistently has the lowest turnout for national elections is Jammu and Kashmir. The states with the highest turnout are the states of the Northeast. Do you think there might be a different reality of Indian elections for people in those states? Would you need to put an ethnographer team there and do a study? Based on your own impressions, how expansive are these findings from the 2009 study that became Why India Votes?

MB: The reasons that we found for voter enthusiasm or engagement to my mind are not election-specific. The short answer is I can’t see the reasons changing from election to election. What I can see is that certain elections, when they have an especially high buzz as the 2014 [elections] did, more people vote. It’s like the Obama effect in the US. There was a higher turnout then than ever before. It’s in the air so much; it’s always in the air when there’s an election in India, but there was an extraordinary amount of money spent on media images and the overwhelming presence of the election meant that it’s likely that more people are going to vote.

I think the Jammu and Kashmir and Northeast contrast is a very interesting one to think about. I will obviously add the proviso that it’s always good to do more research to find out whether this is true or not, but on the basis of what we found in 2009 that makes perfect sense. That’s exactly the kind of interaction I want between ethnography and surveys. Now if you tell me that the survey data shows this, I will try to use my insight from ethnographic work to explain it, with the proviso that we do more ethnographic work now that we know this finding.

So much of people’s desire to engage is about being counted as a citizen of the country. Of being counted, basically. It’s a sense of selfhood. It is an ethical action. It is about themselves, not about the corrupt politician. And therefore it is a moment in which they are able to express the sense of belonging, and desire to belong, and desire to be part of the polity in very concrete terms.

The Northeast is so horridly absent from the national imagination at most times that I can see why anyone living there would want to use this opportunity to say, “We also exist.” This is not dissimilar to this man in the Sunderbands — I think I quoted him in Why India Votes or it’s in the previous paper, “Sacred Election” — where he was literally in the last village before the Bangladesh border. And he said to me — in ‘99, this was — he said, “Look at where I live. I’m surrounded by water. I’m on the edge of this country. Over there is Bangladesh. If I don’t use this moment to register my presence, I’m going to be totally forgotten.” So the very fact that we now know that Northeast voters vote in large numbers makes us remember to count them. In a sense, it’s a vindication of their agenda.

How that plays out in Jammu and Kashmir, I don’t know. There have been J&K elections where voter turnout has been very high; in others, it’s been low. Jammu and Kashmir anyway is a difficult one to explain. And in that case, partly because the relationship to the Indian state is so radically different at every election — depending on what’s happening with the army, where we are with the talks — that it’s constantly shifting, unlike in the rest of the country where there is a certain stable relationship of neglect or occasional interest that we can take for granted. So I would put J&K aside precisely for that reason, because of the bearing nature of the presence, the ham-fisted nature of the state, how much people are willing to place their hope in being part of the Indian democratic polity. All of these things change from election to election.

But the rest of them — when you go back to the 2009 study — we have a very good balance of North and [South]. We have a very good balance of East and West. We have a good balance of rural and urban. I don’t see those reasons changing from one election to the other. What I just said about voting being the exemplary act of citizenship where you are allowed to register your name. Voting because of peer pressure, because of the indelible ink where it’s really uncool and inconvenient not to have voted — you get pestered so much by questions.

Voting because the experience of the polling station — and I can’t underscore this enough — I have now, since we did this research, been talking to so many people. Every Indianist I talk to I pose this question to: can you think of any other example of a public moment in Indian life which is comparable to the polling station queue? Anything else where you get a genuine mix of class, a genuine max of caste, gender, poor and rich? It is random. And it is orderly. Nothing, absolutely nothing, in the country even remotely comes to that.

And it’s funny; just the morning I was on the London Underground coming into work, and partly because you never stop being an anthropologist there was nowhere to sit. I was standing near the door. I just looked around the six or seven people around me, and I had this wonderful, “Gosh, I love living in London” moment. Because there was an Eastern European guy with paint on his trousers and dirt under his fingernails — he was clearly a builder — who takes out a smartphone, takes out his headphones, wrestling with his roll out tobacco pouch, he gets his headphones out, pulls out his smartphone, listens to music; he’s leaning against the door. There’s a woman in a barber jacket, immaculately made up, a city worker. There’s a student next to her. And then next to her there’s an Indian woman. The social mixing that you get — and the London Underground is not cheap — this kind of mixing simply doesn’t happen in India, even in that partial way. The polling station is the only one. And I’m not surprised that everywhere in the country people talked about this. And especially the poorer, the lower castes, the marginalized; they mentioned this much more.

I saw it in Bandra, in Bombay. I happened to be there on the day of elections in 2014, and I went to various polling stations across the suburbs in Bombay. And in Bandra where I was staying you could see this. There was literally a guy with a gold Rollex watch next to a man without shoes. Everybody understands this is India with all its contradictions, but it’s almost as if those contradictions are made visible on that one day. And it is orderly and everybody is treated equally. Now this is a big deal. And I don’t think our social scientists — if you don’t spend enough time outside your offices, not just traveling to India to conferences and training researchers but actually getting out into where there is no sanitation and where people have one set of clothes and where they don’t have any security of existence, those kinds of situations. This is a big deal. They are treated with respect. They are humanized for that day.

So that’s why I spent a whole chapter in the book talking about the culture of the polling station. That’s where an anthropological frame really helps because it’s only anthropologists who are able to break down every element of a ritual or of a space and reap the meaning in it. Not in some anal academic way but through the lens of the people there who are often thinking it but not articulating it. It’s the engagement of my questioning and being with them and chatting with them in the evening, or chatting with them seven days latter, or chatting with them at the next election — that kind of longitudinal, in-depth engagement that ethnographic work brings — is the one that brings out why it is meaningful for them.

Initially, when I ask people in my village, they say, “I like voting.” “Why do you like voting?” “Oh, it’s nice. I like it.” That’s all you get if you ask people questions. But after six or seven years of being in and out of that village and spending several months at a time sometimes, you begin to see why it is meaningful. And you can’t understand that meaning by confining it to their understanding of politics. I don’t know whether you read “Sacred Elections,” but do you remember the story of the sari?

SS: I don’t.

MB: This is a good shorthand way to explain to you what I’m saying about the ethnographic method. The reason I even called it “Sacred Elections” — but people now misunderstand that. And if you can in your blog help me dispel this misunderstanding, I’d be so grateful. Because I used the word “sacred,” they think it’s religious. Whereas what I mean is that it’s sacrosanct. It’s inviolate. And the reason I arrived at this rather bizarrely is because the women in the village where I worked on the whole have two saris, and they have one good sari which they wear for special occasions. I had been there one year for Eid, for Qurbani, when they were sacrificing animals — it’s a mostly Muslim village — and they were all in those best saris. We talked about that. I was told my best sari was not good enough, I should have worn something else, etc. Next time I saw them wearing their best saris was a wedding. And the third time I saw them wearing their best saris was an election day.

That gives you a clue how people approach elections. And for the voter, election day is contiguous and similar and analogous to all other special festival days. You see that in their choice of sari. It’s a visible clue. And then you can begin to probe questions on why the hell it is so special. Yes, Eid I know is special. A wedding I know is special. Why is it elections are special? But you wouldn’t know it is that special unless you were there for all those different occasions. If I just kept going on election time, as journalists do and often political scientists do, then you’d never know it’s their best sari.

I had political scientists on my researchers’ team because I was so desperate for researchers and I couldn’t find enough people in India — young people, paid well, to go and spend one month somewhere without budging. People were simply not willing to do it. This is the state of our social sciences. We are internationally known, but when it comes to being in touch with what’s going on in India on the ground, nobody was willing. So it was really hard. But one of them was trained as a political scientist, fancied himself as a political philosophy kind of guy, and his take on what he was hearing was extraordinarily paternalistic. He said, “Oh, they think it is important. They think their role is really important. Because people are not educated, they don’t realize that really it doesn’t matter. One vote doesn’t really count.” Whereas if you put that to a side and listen to what people are saying, and really trying to understand what they are saying, and often realize that what they say is very different from what they actually do, you realize that when it comes to the doing it is invested in the whole range of meanings that we need to create the academic social science language to capture. And I think ethnography actually is the only way to do it.

SS: Are there any other similar studies for other countries?

MB: You know, I don’t know. I certainly haven’t seen comparative ethnography like this. Because what was discussed a lot, partly because of sexy American anthropological theorizing about multisited ethnography, and that was a very different thing where the same ethnographer — if you wanted to study a city — went to four or five key different kinds of spaces and tried to capture the city through that. So you went to a school, you went to a playground, you went to a church. This made-up model, I so far haven’t found a close comparison with it.

But, having said that, I would love, just on this question of why people vote, to see us doing studies like this elsewhere. Especially where there are democratic traditions that flourish where we worry about voter turnouts. I would love to do a study in Britain where I live, and I engage with British politics as deeply as I do with Indian politics. I’d like to see it done in the US, where so much is said in the name of voters. It’s happening in the US now. It happens in Britain. Every time an election comes round, the voter is spoken on behalf of.

Today, they said in New Hampshire, “Despite the gale and the wind and the snow, a lot of people are going to show up and vote.” My first question was why are they going to do this? Why are they feeling so invested in this? Is it only about outcomes? Or is it about, “Gosh, this might be a milestone election and I want to have been part of that story? I don’t want to be sitting at home if there is a really interesting moment in American democracy.” And the reasons will be different in different countries obviously. In India’s deeply socially unequal society, the chance of equality and respect and dignity — I think dignity is the key word here. Most of our population in India is robbed of its innate human dignity. Forget civil, social dignity. And the absence of civility in public life. But there might be very different reasons in Britain.

On West Bengal politics

SS: For your fieldwork, you spent time with [West Bengal chief minister] Mamata Banerjee and the Trinamool Congress. What are you expecting to happen in the elections that are coming up this year?

MB: I’ll go back to my village. I’ve found that rather bizarrely my village has been a rather good bellwether for what happens at the state level. I couldn’t manage to go back last year. But I will in May this year when the elections happen.

I know the BJP made a big push and failed miserably. I also know that quietly a lot of Trinamool work is getting done in West Bengal. Little changes which matter to people of electricity and roads and infrastructure and so on. Anyone who does anything to make the state better will be rewards. If Mamata has made even this much change [hold up her hand to show a little space between her thumb and forefinger], she will be rewarded for it because she is a stable option, as opposed to the squabbling Communists and the chauvinistic ascendant BJP, which simply doesn’t chime. They tried. Narendra Modi after all campaigned in West Bengal and in his wake there were riots. So that kind of division is there. Bengal after all higher than the national average Muslim population. That kind of divisive politics I’m not sure will get anywhere. Everybody I talk to who’d know seems to think Mamata’s going to win.

But I must say, since you mention my work, my interest in Mamata started when she first won her first Lok Sabha seats a month after forming Trinamool Congress in ‘99. I spent a month shadowing her. It was very interesting because I just thought, “Gosh, all these people voting for her… what do they see in her?” Again, I was interested in the voter. And I wanted to understand what was the source of her charisma and her authority. I just wanted to see her more closely. I got chastised by social scientists up and down the country, in India, abroad. “How can you take this woman seriously? She is completely without culture. She looks like a maidservant. She cannot be taken seriously.” I said, “I’m interested in the people who vote for her, not so much in her. I want to talk to the people who come to her rallies.”

In a sense, it’s been again a vindication of that populist pulling power that she clearly had always. She has an ability to identify with the common man and woman which is unparalleled in Bengal politics. It’s unparalleled. And Yogendra used to have a good line about this; we used to talk a lot when I first started my research about this. He said, “If you put Mamata next to [former West Bengal chief minister] Jyoti Basu and you didn’t know who they were, and asked, ‘Who do you think is the communist leader?’ People are likely to say Mamata.” I remember saying to him, “Yeah, but you’re working under the assumption that Communist leaders are always close to the ground, whereas as we’ve seen in Communist Russia and elsewhere, often that idea of the intelligensia vanguard elite leading the masses is so built in to the ideology of communism that as a people’s leader, yes, it’s Mamata. But that’s not synonymous with communism as we know.” There’s lots of fantastic CPI [Communist Party of India] leaders who are much more ascetic and simple and austere in a way that Indian communism has thrown up repeatedly that kind of example. But Mamata’s fight was exactly with that elite political establishment. The parallels with [Delhi chief minister Arvind] Kejriwal were evident.

That paranoia and that intolerance of dissent that all these guys have — whether it is Mamata, whether it is Kejriwal, whether it is Jayalalithaa, whether it is Mayawati, and Narendra Modi of course par excellence — it is an intolerance of democratic dissent which is common to them.

So I don’t know, but it looks like there might be an erosion of her vote share, but I can’t see an alternative formation winning the election. Don’t forget that it’s not 2011 we should be comparing it to so much as 2013, when they had the panchayat elections. That was the last time I was there during elections. People were telling me that they thought 2013 was the proper consolidation of power. They said, “You’ve got to win the state elections. Then a national election is coming up. But really, unless you win the panchayats, you can’t make any changes.” So it is really her record between 2013 and 2016 that she’s going to be judged for. Because people think that unless you have the panchayats, you can’t actually do anything on the ground because the budgets are devolved to panchayats. It’s because she won the panchayat elections so convincingly in 2013 that that’s what she’s going to be judged by.

But already between 2011 and 2013, the moment you got out of Calcutta and stopped reading English newspapers… In 2013, I was being warned by people in Delhi and Calcutta saying, “Don’t go to West Bengal. Don’t go to your research village. There’s so much violence going on.” It was like there was a civil war on. And you went out there and yes, there’d been incidents of violence, and we normalize these incidents of violence. But those apart people were actually quite chuffed with how things were progressing. So it didn’t surprise me at all when she won the election because people felt that they could see changes that were better.

Part of the challenge of studying Indian politics is also to get beyond the headlines. Because headlines are being produced by people who, especially in English, simply don’t have the sense of what’s going on.

SS: Thank you very much.

Advertisements

Sanjay Kumar on the challenges of survey research and CSDS’ role in Indian society

lok_people_sk

Sanjay Kumar is the Director at the Centre for the Study of Developing Societies (CSDS). He is also the Co-Director of Lokniti, a Research Programme of the CSDS. His published books and edited volumes include Indian Youth and Electoral Politics: An Emerging Engagement, Changing Electoral Politics in Delhi: From Caste to ClassMeasuring Voting Behaviour in India (with Praveen Rai),  Rise of the Plebeians? The Changing Face of Indian Legislative Assemblies (with Christophe Jaffrelot)and Indian Youth in a Transforming World: Attitudes and Perceptions (with Peter R de Souza and Sandeep Shastri). He has contributed chapters for several edited volumes, written various research reports, published articles in both international and national research journals and writes regularly for popular newspapers.

As an expert in survey research, Kumar has directed various national and state level studies conducted by the CSDS, which include the National Election Studies (NES) 1998, 1999, 2004, 2009 and 2014, various round of bi-annual ‘State of the Nation Surveys’ and various state level studies. While his core area of research is electoral politics, he has also been engaged in survey based studies on Indian Youth, State of Democracy in South Asia, State of Slums of Delhi, State of Indian Farmers and Issues of Electoral Violence.

Sanjay Kumar is also the supervisor for my research project. Last Tuesday, we sat down in his office at CSDS to discuss his background in survey research, the challenges of survey research in India, and how he defined CSDS’ goals and mission. The transcript of the conversation below has been lightly edited for length and clarity.


On his background in survey research

Sam Solomon: Please tell me how you got into the field of survey research.

Sanjay Kumar: I won’t say it was very pre-planned. It was kind of an accident.

I had enrolled myself as a PhD student in the Department of Political Science at Delhi University. But I also wanted to get hands-on experience of how you collect data, how you analyze data. So I came to this place [CSDS] to work along with the team that was headed by Yogendra Yadav and VB Singh. This was 1993, October. They were initiating a project to study assembly elections which was being held in five states: Delhi, Madhya Pradesh, Rajasthan, Chhattisgarh, and some other state. I came to know about the project. So I came and [told] them that I would be happy to work with them on the project. They said yes, this is a small project, we would be happy to associate [with] you for a few months. Maybe two, three, four months.

So I started working on the project. Got more and more interested in the kind of analysis which they were trying to do with the data, the fieldwork, etc. With my initial… not contract but kind of informal understanding that I would be working for three, four months, I also got a chance to the go to the field. I went to Madhya Pradesh to do a study of one constituency. I extended my research to making a comparison of four or five constituencies, looking at the Muslim voting behavior, which is my first article in EPW [Economic and Political Weekly], “Myth of Muslim Vote Bank.”

So I continued. I worked for three, four months. Then the project was getting winded up. But still VB Singh and Yogendra Yadav were looking forward to bigger studies in Bihar and in Lok Sabha elections, 1996. So they checked with me if I would be willing to be part of the project for the next six months or one year’s time. Since my initial experience of working three months, I got very excited about the kind of work which they were trying to do. So I said yes, I would be willing to work for the next six months or so, or one year. Then I think we moved into some kind of formal contract.

And then time passed. After that, we had the assembly elections in Bihar. I was deeply involved in that because I belong to Bihar. So my home in Patna was kind of a central place from where we were running the project. I went around to several constituencies to do the study. I especially studied one constituency, Mokama. But along with that, I was also coordinating the entire study in Bihar.

And by that time, I had almost decided that now, it’s enough. I should actually go back, do my PhD, etc. So that’s the time I gave a hint to Yogendra Yadav, with whom I was closely working, to say, “I think it’s enough. I should go back to my PhD, etc.” And then, I think, this was a conversation I think we were having at the railway station when I came to see him off. He said, “Let’s discuss this when you’re back in CSDS. Why do you want to take a decision immediately?”

And when I came back, we had [a] long chat. He said, “If you look at the history of CSDS, people who have worked here, their careers have gone up. Their careers haven’t gone down. So why do you want to leave? You can continue your PhD. You can continue working with the projects if you want. We have a space for you.” Space, in a sense, space in a place to work with me. I thought several times, and then decided okay. Then he said, “Now we’ll work out another arrangement. In a sense, it would be kind of a lecturer-level arrangement now. You have been working with us the last two years as a research officer.” The initial six months was a research assistant, then it was kind of a research officer. After the Bihar assembly elections, they said, “Now we are heading to the big Lok Sabha elections. Now you have gained experience. We value your association. And you would gain by working with us, and I don’t think your career is going to be on a downward slide. And if you agree to work, then we can work another arrangement. Give you a lecturer’s level position. So you will be a research associate now.”

I decided to continue and then finally after the Lok Sabha elections of 1996, we had a detailed discussion about how I [saw] my career moving. And they said, “Now we want to take this election study forward. We don’t want a break in this. And we would like a person like you in a permanent position. Are you interested in that kind of thing?” The institution advertised for that position. I applied along with several other applicants, and I was selected by the selection committee. Formally, I would say I joined CSDS faculty in April 1998.

[For] my entry into survey research, this is the kind of journey I took. But election as a theme has always interested me. It’s not that it was something not interesting for me. Even when I was in my school days — I remember class 12 — I was so fascinated about elections that I used to do, if I recollect, my kind of psephology. I used to read newspapers and prepare charts on my own. Who is likely to win from which place? Though I miserably failed because this was 1984 elections, and ‘84 election was a wave election for Congress. That was the time, if I remember correctly — and I do remember correctly — that I was kind of anti-Congress. Really did not like Congress. So when I was doing my own kind of psephology, I think I used to be a little biased. When I was making a judgment about who was going to win, I would actually defeat most of the Congress candidates on paper. “Oh, he will get defeated…” I remember defeating on paper [Bollywood superstar] Amitabh Bachchan because he was a Congress candidate contesting against H.N. Bahaguna, who a very tall leader from another party. That’s my entry into survey research.

SS: You joined CSDS informally in 1993, so you’ve been with CSDS for twenty-three years. During that time, which aspects of survey research at CSDS have changed the most and which have remained consistent?

SK: I think the big change is the timelines. When we started working in 1996, ‘96 was not with TV. It was with newspapers. But when we are working with television as a medium, the analysis has to be very quick and very sharp. If one is writing an article in the journal, then you have all the space. You can write 5,000 words. When you’re writing in a newspaper, you have to restrict to 800 words, 1,000 words. But when you are actually providing stuff to the television, it is very little words but it has to be very, very sharp. So I think our rules of analysis, trying to do deep analysis, that’s one thing that has changed. [In] 1996, we would do broad analysis maybe. And it would take a lot of time. That has changed.

The second thing is about the sample. It is also connected with the analysis. Earlier we used to do more aggregate analysis. We would come down to a few states, bigger states where we had bigger samples. But now we know that we not only do analysis for big states, we have been able to do analysis even for small states. Even within the states we actually disaggregate the analysis. We try to go down the level of analysis to figure out what’s happening at the ground level itself. That is a big change.

The other is that, initially, ‘96, ‘98, ‘99, even up until 2004, the survey as a tool we applied only for election studies. Now we have diversified a lot and we apply the survey method to do studies on various other themes. We have done the study on farmers. We have done the study on youth–two studies on youth. We have done studies on Naxals. And various other issues: economic reforms, Muslims, women, eating habits, all kinds of issues we have touched upon. That’s a big change. The initial years, we were only looking at using surveys as a method to study elections, voting behavior. Now we have used that tool to study various other kinds of issues.

On the challenges of survey research in India

SS: What are some of the greatest challenges you’ve encountered during your time at CSDS when it comes to 1) accurately measuring public opinion and 2) effectively communicating the findings from your research to a media outlet or the public?

SK: I think the biggest challenge has been to keep up the reputation CSDS has got. Because as soon as something is put out in the public, that this is a study conducted by CSDS, moreso with regards to elections — vote percentages, which party is likely to win — a large number of people believe it is a truth. And they would say, “If CSDS is saying Party A is winning, definitely Party A would be winning, because they never go wrong.” So that’s the biggest challenge. Maintaining that credibility in the field and trying to do it as accurately as possible.

SS: How do you address that challenge?

SK: Being neutral all the time. We try not to be biased, not to be swayed [by] our likes and dislikes for leaders and political parties. And the big advantage when we are trying to do the analysis, when we are trying to assess which party is likely to win the election, we take advantage of the huge datasets we have in our hands. The historical data.

Suppose we see the Bihar elections. For Bihar, we would have at least eight or ten survey datasets. So we try and look at what has been the pattern when we did surveys in the state. In order to try to be as accurate as possible, we try to be neutral. We try to collect data in a neutral manner, do the analysis in a manner as neutral as possible, plus take advantage of the past data to figure out if at all we made any errors when collecting data in a particular state. That’s how we try to make our findings as accurate as possible.

SS: When it comes to the surveys themselves, have you found that certain types of populations — gender, caste, class — are more likely to be oversampled or undersampled? If so, which populations have those been?

SK: If you look at our sampling technique, our sampling technique is very different compared to the sampling technique of other survey agencies or organizations. We not only sample the constituencies, the polling stations, but we also sample the voters whom we want to interview. All of these are done randomly, systematic random sampling. If we have adopted a good sampling method, a scientific sampling method, and if investigators actually follow that method — when they go to field, they go with a list of people who have been sampled, and they have to interview only those people on the list — if they actually follow that, there is very little chance of oversampling or undersampling any segment of the population. Unless we are looking for a very small sample.

Normally, we take a sample of say, 3,000 or 4,000 in a state. The chances of the sample being overrepresentative [of a demographic segment], over or under, is very [small]. That has been the record of our surveys for the last twenty years. We have put all this data on our website to figure out when did we go wrong with a sample that is representative.

But still, if we look at the chances — still there are chances — normally, we might expect women to be slightly less in our sample because it also has a relationship to the investigators you have. If there are some states where we don’t manage to get an adequate number of women investigators– it becomes extremely difficult to interview Muslim women if you don’t have woman investigators.

SS: Muslim women investigators or just women investigators?

SK: No, just women investigators.

Normally, there would be a lower completion rate in the urban areas because the urban population is more mobile compared to the rural population. Sometimes we also get samples which are underrepresentative in terms of the urban population.

These are the two elements of the sample which I have noticed that sometimes we get wrong, in a sense, underrepresented. All the time they would be underrepresented. Hardly overrepresented. Urban and women.

SS: When it comes to SCs (Scheduled Castes) and Muslims, you have census data that you can consult to get the proper proportion of SCs and Muslims in the states. So it’s not as much of an issue.

SK: I don’t think that we had issues with regards to the Muslims and the dalits, SCs, when it comes to getting a sample.

Unless we are doing an exit poll. We used to do exit polls earlier but now we have stopped. Because if we are doing exit polls, there is always a chance of heavy overrepresentation of young, urban voters. And also this issue of dalits, Muslims, or tribals not being adequately represented in the sample, that’s the case when we do exit polls. That’s largely the case. And that’s why since we are not sure of getting the right representative sample or not, that’s one of the reasons we stopped doing exit polls.

That’s a big problem in exit polls, but not in the pre-poll or post-poll surveys which we conduct. Only because we actually sampled the respondents from the electoral rolls.

SS: What about OBCs (Other Backwards Classes) or intermediate castes?

SK: There are very rough estimates about the proportion of OBCs in different states. It ranges from 40% to 55%, 60%. And it is such a large group that getting them not adequately represented in the sample is rare. But since you don’t have an exact estimate, it’s not like Muslims, it’s not like dalits or tribals, [where] you have data to compare [to the sample]. OBCs are only an estimation. In Bihar, we have roughly about 52%. We don’t know whether they’re actually 45% or they’re 55% or they’re even 60%. So that has not been an issue in the past.

SS: I find the issue of caste to be really interesting, maybe because it’s a foreign phenomenon for me, but also because it’s so complicated. Last week, [the Lokniti survey research team] were trying to figure out how to code the different castes in Tamil Nadu. Everyone in the office is north Indian, and the caste system in Tamil Nadu seems really complicated. We were struggling to figure out which caste gets categorized as upper caste, and which as intermediate, and so on.

How do you do that for every state in India? Do you have the lean on the local knowledge of the Lokniti network?

SK: You will notice that we use a codebook to code several open-ended questions, caste being one of them. That codebook has not been prepared overnight. If you get sometime a chance to look at our codebook [from] ‘96, ‘98, ‘99, [you can see] how it has progressed, how it has improved, how it has become more comprehensive now compared to the past. The 1996 codebook was one sheet, one page, for all the codes: education, occupation, caste, etc. It was just one sheet. Now if you look at our codebook it is a booklet. Six or seven pages have been devoted only to caste.

This has been possible because, as you rightly mention, of inputs from our local coordinators. At one point in time, we actually took it as a project to engage our local coordinators, and requested that they engage sociologists at the local level to figure out where each caste should be placed, and what is the parallel of caste A to castes in other states. Caste coding is really complex, and we have been able to develop [it] after a long, long period of experience.

Still, we won’t say that it’s completely perfect. Because every time we do a survey, we encounter this problem in varying degrees. And we try to develop it as [neatly] as possible.

On the goals and mission of CSDS

SS: Which factors do you weigh when deciding which research projects you are going to go forward with? Naturally, there’s always going to be an audience for election surveys. People like to read about politics and elections. But how do you decide to do a survey on farmers, or youth, or Naxals?

SK: You rightly pointed out that elections keep happening every year, and CSDS is a place that is known for its research on elections. So election studies we keep continuing. On other issues, whether we want to do a survey on Topic A, B, C, or D, the big consideration is: is this a topic [about] which people want to know, people would have curiosity? Is it topical? Do people have concerns about that?

So if you look at the kind of research which we have done on youth, there’s so much focus on youth, not only in India but in South Asia and other parts of the world as well. Their anxieties, aspirations, employment, unemployment. So we have done studies in the past on youth, and also we are in the process of doing another round of the youth study to look at the change over a period of time.

Farmers — if you actually look at the debate going on in the country for the last couple of years, [there is] so much focus on the crisis of farming in India related to land acquisition bill, etc. When somebody approached us with the idea that there is a need to do a proper study of farmers in India, though there are other studies — but mainly done by economists who look at the economic condition of farmers mainly. So we pitched in. We did the study on farmers. Along with looking at the economic conditions, we focused more on anxieties, aspirations, expectations, more on opinion rather than on fact. That is how we picked up that study.

On Naxals, [there is] so much focus on this issue of Naxals. Is it an issue of economic deprivation or is it a law and order problem? That’s the big question people want to address. Some people see it as a law and order problem and they say, “The government should use hard force to just crush them.” The other side says, “It is an economic issue. These are poor people whose land has been taken away by the government. It is an issue of their survival.” So we wanted to do a proper study [to] figure out what is the situation. We are not saying whatever we would say would be the truth. But at least we would try and bring out what is their situation on the ground.

So that’s how we pick up the themes. But we have also been approached to do studies about consumers’ perceptions about the new airports which have been developed: Bangalore, Hyderabad, etc. We were quick to say, “No, we don’t want to get into this.” This is none of our business. If we want to do a study, it has to relate to [the] social sciences. We don’t want to get into airports. We were approached to do a study about something related to electricity. Supply of electricity, are people happy, should prices go up, etc. This was in Madhya Pradesh. And we said, “No, we don’t want to do this.” This is not social science.

SS: Aren’t those issues — airports, electricity — related to issues of economic development as well? I suppose that if an electricity company approaches you and says, “We want you to do this survey, so we know how to price our electricity.” That’s one thing. But how do you draw the line between what is social sciences and what is not?

SK: The prime concern is that when we do the survey, we also want to be clear about how we are going to make use of this data in the future. If it is for the client, just to do the survey and give them the data, is that the end? CSDS is not there to make money off of surveys. So we figure out if this data is going to be useful for us in our future research. Will this data be useful for other social science researchers?

When we thought of the electricity [study], we thought this would be a kind of client servicing. You take money, collect data, and give it to the client. Airports — it would make news in the newspaper for one day, but later on social science researchers won’t be able to make any use of the data. That’s the concern.

CSDS is primarily a social science research institute. This is not a survey agency. We have to be clear about that. We keep reminding ourselves that if a study has something to do with social sciences, society at large, people will make use of the data in the future, then we will get into that.

SS: How do you envision the role of CSDS vis-a-vis Indian society today? What is CSDS’ overall mission for itself, and how does Lokniti’s research fit into that mission?

SK: CSDS’ mission of research is very, very difficult to say. We have been debating for the last several years to figure out what is the core research of CSDS. We figured out that we can’t say, “No, this is the core of CSDS’ research.” We can only say the core of CSDS’ research is to do research in social sciences which would cut across several themes: media being one, urban being another, elections/democracy/politics being an important theme. Also now youth has become an important theme for the last five, ten years because as soon as anybody talks about a good study on youths in India, they would refer to CSDS. Political philosophy. Secularism. So there are six or seven themes which we can say are the themes on which CSDS does research. We won’t be able to say, “This is what CSDS research means.”

Lokniti’s position vis-a-vis CSDS — in CSDS, people not only do research on various themes, they also adopt different methods of doing research. Within that, Lokniti fits into one of the core themes of CSDS, because CSDS was established for doing research on Indian politics. Lokniti now is continuing that tradition of doing research on Indian politics. When CSDS was formed in 1963, the initial studies of politics [were] conducted through survey research: doing surveys in Kerala when the Kerala assembly elections were taking place, then immediately after that we had national elections. CSDS conducted surveys in the 1967 election [and] 1971 elections. Lokniti is trying to the take the CSDS tradition forward of doing studies on Indian politics using surveys as a method. That is why I would say Lokniti is core to CSDS’ functioning.

SS: CSDS is like a tray with different foods and the foods don’t really get mixed together. I’ve been here for more than six months now, and I’ve really gotten to know the Lokniti team well. But I don’t know anyone from Sarai. I don’t even know the other programs that are here at CSDS. Do the other teams interact with each other? Is there ever any talk of working together or does everyone just do their own initiative?

SK: People are always encouraged to do collaborative projects. But we can’t imagine a situation where twenty faculty, twenty-two faculty, will all be working together on one project. This happens only in small institutions, with less than ten faculty, then maybe a large number of faculty will be working together. But in an institution where you have a faculty of, say, twenty-five, you can’t imagine a situation where everybody would be working on the same theme.

But yes, we have forums where people are expected to interact with each other. All the programs make a presentation to the faculty about their work. Every month we have a faculty meeting, but that’s kind of administrative. We don’t discuss academic issues. Every Friday all the faculty members try and sit together to have lunch. That was designed in such a way that people can have exchanges about their work. We have also instituted a system of annual faculty seminars which a very intensive exchange of each other’s work and ideas.

So we have tried to have forums where faculty interact, know each other’s work, and also anybody is welcome to make a presentation whenever someone has conducted a study. Now we have also instituted a system of monthly faculty seminars, which began only last month. Every month one faculty member is going to present a detailed work. We have kept two hours for that. You present your work for an hour, then a one-hour discussion.

So there are platforms available, but I can’t think of a situation where everybody would be interacting on a daily basis with each other’s work. That kind of a situation is only in small institutions, because in small institutions sometimes the director assigns work. Suppose the institution gets three projects, the director assigns these three or four faculty to this project. In CSDS, we don’t have that system. We are very, very autonomous. But still you will find Lokniti working as a team, Sarai working as a team in which three or four faculty members are associated. Four faculty members are working on the Indian Languages program. Then there is a project on violence. Three colleagues are taking that project forward. And then you will find another five to seven faculty members who are not collectively working but they are working on their own projects.

SS: What do you think have been some of CSDS’ contributions towards understanding Indian society?

SK: If you look at the various kinds of studies which have been conducted by CSDS during the last fifty-three years, the kinds of issues on which we have touched upon — Rajeev Bhargava’s work on secularism; Sarai’s work on media and urban space; Hilal’s and Sanjeer’s work on marginality, Muslims, deprivation; Lokniti’s work on elections, youth, farmers — If you look at the range of issues which have been touched upon, the range of methods which have been adopted by various researchers, all this has been in the public domain.

We use four platforms. The media, I would say, print and electronic. Research journals. Books. And these days we also have three workshops which CSDS organizes every year. A workshop in which we try to train young researchers in data analysis; we have been doing that for the last ten years. Another course on researching the contemporary which is in the sixth year. And last year we started this new workshop only for students who belong to the marginal communities: SCs, STs, and Muslims. I think there is a lot of input to the social science community by way of publications and training workshops.

SS: Which topics do you think are in need of additional research here in India? Which topics do you think are underexamined?

SK: There is a need that in social sciences people should try to pick up the themes which have a direct connection to society. Not necessarily with policymaking, but directly with society. There is maybe a little more focus on the historical method of research. I would personally want people to be doing research more on current themes. There are issues [related to] youths. There is so much anxiety about farmers. It is an important issue. Minorities. Reservations is such a hot debate in this country. Secularism. And similar such issues. Issues on which the government is trying to formulate, there is a need to debate those issues. There is a need to formulate rules, or revise the already existing rules and regulations, which relate to the common man.

Also, there is a focus these days on India-China relations. But as you know, CSDS is not a place where we actually do foreign relations. Maybe some research on — not exactly on foreign relations, but something related to relationships with neighbors, which would fit into the big ambit of foreign relations.

SS: Thank you very much.

VB Singh on survey research in India since the 1960s

lok_poeple_vbsingh

One of the privileges of conducting research at the Centre for the Study of Developing Societies is the presence of so many accomplished scholars of Indian politics. One such scholar is VB Singh, a Senior Fellow. Professor Singh joined CSDS in 1966, and worked on the institution’s first National Election Study in 1967. He has conducted research on electoral politics for many decades and his publications include a five volume series on elections in India giving detailed results, including by-elections from 1952 to 1985 for all the states in the Union of India. From 1987 to 1990, Professor Singh directed a program on survey research and training, which conducted research methodology courses in the social sciences. He served as the director of CSDS from 1997 to 2003.

On December 19, I met with VB Singh in his office at the CSDS campus to talk about his experience working on election studies since the 1960s. The transcript of the conversation below has been lightly edited for length and clarity. The transcript starts abruptly because I did not turn on the audio recorder at the start of our conversation.


On CSDS’ origins and the first National Election Study in 1967

VB Singh: …building an organization where the problems are not studied only in the classroom situation, but the area is much wider. Go to the people. Understand their views. Understand their problems and their reflections on various issues confronting this country.

So the first two studies undertaken by Rajni Kothari at that time where one was a values in active community, that was an international project headed by Philip Jacob and his wife. In that, they were trying to reach the district-level and block-level leaders. By that time, the three-tiered government was already in existence. So how the values of these active people at this lower level is being inculcated to understand the problems. So that was the project which was done here in India in three states: Uttar Pradesh, Maharashtra, and Gujarat.

Sam Solomon: And what year was this?

VBS: This was 1965.

SS: 1965. And when was CSDS started?

VBS: CSDS was founded in 1963.

SS: By Rajni Kothari?

VBS: By Rajni Kothari.

The first election study was undertaken by Rajni Kothari and his team. At that time, his team constituted Rajni Kothari himself, DL Sheith, Ramashray Roy, and Bashiruddin Ahmed.

SS: And what about you?

VBS: I joined a little later. I joined actually, some time, early ‘66.

SS: So that was before the ‘67 [national election and election study].

VBS: Before ‘67. I came to CSDS through one project which was [a] cross-national project on social and political change. Sidney Verba. Have you heard of him?

SS: I haven’t. Sidney Verba?

VBS: Sidney Verba. Civic Culture [is] his book. Civic Culture. He was from Stanford University. And he had a cross-national study of social and political change. That study was conducted here in India, in Yugoslavia, in Japan, in Canada, and I think America as well.

CSDS was the India partner. And that was the first major fieldwork in which we traveled [to] almost every major state of this country. That was a kind of cross-section [of] people: the panchayat leaders, the block-level leaders, and then district leaders. To cover the three layers of administration, and their views with regard to various issues, and their perceptions, and their evaluations. There is one book based on that data, published by Verba, Bashiruddin Ahmed, and Anil Bhatt. That book may be available in our library [gets up and looks through bookshelf in his office]. You will get it in the library.

SS: This study was in ‘66?

VBS: That was 1966. And then, [following] that study, the ‘67 general election came. So part of that Social and Political Change Studies resources were used in that study. And some new grant from other interested scholars came. We had partial funding from various sources. SRC [Survey Research Center at University of Michigan], America, and then this cross-national project money. This transnational studies fund was utilized to study voting behavior in 1967.

SS: So that wasn’t part of the social and political change study. It was a separate thing.

VBS: That was earlier than this. But that spilled over. The same team went to the field again.

SS: So the same team that did that study did the study for 1967?

VBS: Yeah. By that time the kind of foundation of the survey team was already made. Basiruddin Ahmed happened to be the project director on both the projects, that cross-national social and political change [study] as well as this election study.

SS: What was your role for the 1967 national election study?

VBS: I came from a very rural background. Trained in psychology in Lucknow University and just happened to be here. So I was at that time a research assistant in that project. But I was quite enthusiastic, so at that time Rajni Kothari and Basiruddin Ahmed, they gave me a brand [new] name, “Field Marshal.” It means carrying out, supervising the fieldwork, recruiting and training the researcher team, sending them to the field.

But not alone. At that time, whenever the training of the investigators used to take place, all these go, the senior people: Rajni Kothari, Ramashray Roy, Bashiruddin Ahmed, DL Sheith. [They] assisted them because everyone was interested to know the problem from the ground. They used to visit some of the field along with the field researchers, had their own experience. Their point was that the data were collected by the field investigators, but we have to give the meaning to the data. So we must also go to field, and interact with some of the local people then we will be able to understand the responses from the people and will be able to give meaning to that.

So I associated myself from that. Questionnaires used to be conceived in English. I used to translate that into Hindi.

SS: The original questionnaire was written in English?

VBS: Yeah. Because the basic thing is that with survey research, we had borrowed this technology from some of the established research centres. Mostly, actually, in America. So many of the questions on motivations, on other larger issues, theoretical issues, those questions were in English. So that we brought into [an] Indian English version. And then that had to be translated into the local language, that language which illiterate people can also comprehend. So I did a kind of translation of those and going to the field, testing whether the translation is done properly, whether we are getting the kind of responses we intended to get. Pre-testing of the questions. So that was done.

And then the survey used to be conducted and at that time we didn’t have this kind of mechanism of state-based transferring answers from questionnaires to the computer. At that time, questionnaires used to be quite lengthy. There used to be a large number of open-ended questions and that was the period actually… You c[an] have pre-coded questionnaire questions only when you have a kind of paramount idea of what type of answers you are going to get. Since that was the early stage of our research, we were not very sure. We were asking this question and what can be the answers? But now we can very easily, based on our earlier experiences, we can pre-code. We can pre-conceive the type of answers we are going to get and then you can provide these spaces in your questionnaire.

Earlier everything used to be recorded, the answers [for] open-ended questions. So the filled-in questionnaires used to come, then a team used to sit together. All those open-ended questions’ answers used to be jotted down with a kind of frequency marking. And then a team of seniors, with the project director being the head of that team, the coding category used to be constructed, be it caste, be it problems, be it aspirations, be it expectations, type of leadership. So that used to be done.

And then we used to have the large 80-column codesheets, and we had IBM punchcards. I’m sorry, actually, I have to describe all those things. So the data used to be punched by a punching machine on those IBM cards that used to be of this size. [holds up a large book] Of this size. And there used to be columns 1 to 80, and they used to be filled [with] 0 to 9. And then data used to be punched and then these cards used to be fed to the computer.

SS: So you had a computer here at CSDS.

VBS: We didn’t have a computer. We used to actually go to Delhi University and for some bigger allowances, at that time IIT Kanpur had a very big computer. So the team used to go with that data to IIT Kanpur computer.

SS: So was the data in the punch cards?

VBS: Yeah, punch cards. We prepared a kind of box, in which [there were] actually two thousand cases because of our first study, after election study, and the cross-national study. Tthe total number of completed interviews were less than 2300.

SS: So you had a card for each respondent?

VBS: One card for each respondent. That data used to be run into several cards for one person. So eight cards, nine cards. Card one, card two, card three, card four, card five, card six. So there was variable 1 to, say, 20 in one card. 21 to 40 in other card. 41 to 80 in other card[s]. So that is how this data used to be handled.

You used to get a table, you could ask [for] cross-tabs, bivariate tables, control tables. So the data handling was a big problem. Now whatever you conceive, you feed in the computer and get the out. At that time, getting one table worked out, it used to take days! Even with computers.

SS: Because you had to put all these cards into the machine.

VBS: And then the machine read [the data]. You had to write programs. SPSS was not available in those days. SPSS came into being some time in the ‘70s. So the programmer used to write: this variable, this field, compare this with this variable. So we had the programmers. So that was the case, actually.

At that time our fieldwork was a very vigorous thing, actually. We used to be in field for months. These days actually sending a person to the field for four, five days becomes very, very tiresome. We used to travel. I have myself, actually, in 1966 and 1967, I mean, two months in the field! And we had four or five teams going in very different directions, spending one, two days in one village. Complete the interview during the daytime, most of the time, actually, since it is a kind of working population, you had to be there during the night. For light, we had torches. We used to interview with that. That was a kind of very vigorous thing.

SS: How did you assemble your field team for ‘67?

VBS: Not through newspapers, but we had contacts in different universities. So suppose we have to do fieldwork in Uttar Pradesh. Lucknow University, Allahabad University, Aligarh University. We are looking for these kinds of persons.

Suppose we want to recruit thirty investigators. We used to give training to fifty during all-day class–coming, going, staying, and training them. And out of that fifty, we select thirty, [and] put some persons in the reserve field. Suppose somebody gets ill or somebody decides to leave in between. Always through personal contact, so that the investigator who is selected, he is directly responsible to me only, he is responsible to the person who has recommended his name, and through him, to the person who has recruited. So we used to have this kind of very, very committed team.

SS: For each state?

VBS: For each state. For each state.

SS: The 1967 national election study, was that fielded in every state?

VBS: Yes, and there were state in charge. In 1967, by that time, I had already acquired some experience. So I was made in charge of [the] Madhya Pradesh team. I went to Madhya Pradesh in 1967. In Uttar Pradesh there was another colleague who left us, Mr. Ayep, and one Dhirendra Vajpeyi, who is maybe still alive actually, at Ohio University. These two persons were in charge of Uttar Pradesh. And the training used to be done, generally, [with] two or three states together.

At that time, we used to get a letter from the Election Commission and from the central government. “Such-and-such research team is going to such-and-such area. Provide them basic help and assistance.” We used to approach the district headquarters. From there we used to book our stay in some circuit house, guest house, and the local officials of the state or the block development officer were asked to coordinate our fieldwork. So generally they used to provide local transport from wherever we stayed.

SS: The Electoral Commission would transport you?

VBS: Sometimes, actually, because the letter was from the Election Commissioner and the Home Secretary. They used to transport us from headquarters where we were staying to the village and thereafter they disappeared. We would be there in the field for one day or two days and then they would come and take us back. At some times, if there was no staying facility in that village, then we would do that fieldwork until late and then we would come in that Jeep to the headquarters.

So it was quite tiresome at that time and we traveled, actually, to remote areas where Jeeps can’t go. Even the carriages run by horses, they [would] not. Sometimes bullock carts, you had to hire. [laughs] One field, we had gone by elephant. Riding the elephant, we crossed.

People actually by and large were very, very committed. There used to be team of four or five persons. One person used to just roam around and see if things are doing done properly or not. If somebody is interviewing somebody and that person is surrounded by a large number of spectators, somebody used to go, “Please come and talk to me. Let him do his interview.” So this was our earlier practice.

SS: Was the fieldwork done before election day or after? How was the fieldwork timed to coincide with the election?

VBS: Good question. You see, if it is a kind of voting behavior study, at that time we used to do post-election. Post-election means the process used to start before the election, but the preparing of questionnaire used to take something like [a] fortnight, or a month. Because there used to be four or five scholars, pulling their minds, preparing questions, deliberating, fighting with each other. So finalizing a questionnaire itself was very, very time-consuming. And then the field investigators go to the field after the results are more or less declared. Then we used to ask all these questions: To whom did you vote for? Why did you vote for this person? All other questions.

Later on, we realized that there are some kinds of gaps. Gaps in the sense that once you are going to the field when the results are already announced, then at that time, if you ask the question, “To whom did you vote for?”, there is more likelihood of the person responding in favor of [the] winning candidate. Because it’s human psychology to side with the winning person. So we used to get much more responses in favor of the winning candidate and the winning party. Then you had to test the validity of that response along with the other questions. He said that this party was good, that party was bad on this count, how far the response of that person is in favor is correct. Number one.

Number two, another thing was when you ask a person, “Whom did you vote for?” That respondent used to answer for, “I voted for Mr. X.” Either that person’s name, or she will name the symbol, or name the party. Later on we realized that this is not a very correct and honest response we are getting. Why? Because there are other people who are also hearing that. So suppose I am supporter of Congress Party, and I am being interviewed by you. You ask me this question, “To whom did I vote for?” My natural response would be, “I voted for Congress Party.” But I am not saying that. Why? Because there is one boy standing there whose father is the landlord of that village, or is the pradhan of that village, or is the leader of one political party. And he has come to me and asked me to vote for some other party, not the Congress. Suppose he asked me to vote for BJP. At that time, it was Jan Sangh. Even if I did not vote for that party, to communicate to that person, through that boy, that I voted for BJP, so that that boy will go and he will tell his father, “I was listening and he was asked this question, ‘To whom did you vote for?’ Papa, he did vote for BJP.” So instead of answering my question, he was answering and satisfying that person. We used to get wrong information.

So there were two changes in our survey technique that were introduced. One, asking this question, not to respond openly, but we created a kind of dummy ballot box and dummy ballot paper. Asking them to mark your vote here — don’t tell me, don’t show me — and put it here. And the other was that since the results are announced, there is a kind of bandwagon effect on the respondent. To reduce that, we started doing pre-poll studies. But by learning this technique, the two general elections were already gone. In ‘71, we didn’t do it. In 1980 election, we didn’t do it. But later on, in the 90s, we started doing this pre-poll — pre-poll means pre-results — and through ballot paper and ballot box.

SS: That wasn’t introduced until the 90s.

VBS: And that’s why the answers used to be biased in favor of the winning party, at that time.

 

On sampling methodology, and CSDS studies during the 1970s and 1980s

SS: How did you do the sampling for the ‘67 survey, and how did it change from ‘71 to ‘80?

VBS: Sampling actually, we have not deviated a lot. Because that was quite perfect during that period.

In 1967, we selected constituencies based on their competition typology. The different types of competition between the parties. The competition between parties where Congress, which used to be considered a kind of centrist party, where there is a right-wing party as well as a left-wing party — triangular contests. The constituency where there is only Congress and right-wing parties. The constituencies where there was only left-wing parties. It was a direct contest, or triangular contest, or four-cornered contest. Since the survey was being done after the results were announced, we knew what the different political parties got and in what type of contest typology that constituency [fell]. That was applied only in 1967 and through that we are not getting the real representation of our whole universe. Because certain kinds of constituencies fell only in a few areas. So that area used to get less representation.

Soon after ‘67, in ‘71, we made a kind of very universal sampling design in which we decided how many interviews would be good enough for the country. So that was at that time 2,500 to 3,000 responses, would be good enough to do a kind of analysis at the national level. Then how that had to be distributed. At that time, there was 400 some odd constituencies. It was enhanced in 1974 to 543. So this many constituencies are there. So we had to select this many parliamentary constituencies, about — I don’t exactly remember the percentages of those constituencies — suppose it was decided, 15% of parliamentary constituencies will be studied. So depending on the size of the state and other things, 15% of constituencies were selected. So from some states, twelve constituencies. I think it was more than 20%, because a state like Uttar Pradesh always got more than fifteen constituencies. From smaller states, two parliamentary constituencies. From middle-level states like Kerala, four constituencies. Like that. You decided that this many constituencies would be studied from this state, this many from this state.

Then how [were] those fifteen constituencies from Uttar Pradesh selected? For that selection of the parliamentary constituencies was done on PPS [probability-proportionate-to-size] basis. You take the electorate size of all the constituencies, get the cumulative total, and then divide it, get the mean, mode, median like that. Then you select one — you always allow a larger chance to the larger constituencies. So once the parliamentary constituencies were selected, we decided to select two assembly constituencies from that parliamentary constituency.

SS: So just to be clear, the assembly constituencies are located within the parliamentary constituencies.

VBS: Each parliamentary constituency, two assembly constituencies.

SS: This was at that time, in 1971.

VBS: Yes. To select those two constituencies was again the same basis.

SS: PPS, random starting point with a step.

VBS: Yeah. Total of that parliamentary constituency, divide it by two, then draw a random number from the first half and add the half to that and get the second constituency.

SS: So they typically had more than two assembly constituencies in the parliamentary constituencies. And you would pick two by doing PPS.

VBS: By doing this PPS. Then from each assembly constituency, again two polling stations. And those two polling stations again on the same basis.

SS: On the same basis. Most of them are pretty much the same size, but you’d still do PPS.

VBS: The larger polling stations used to get a little larger chance of being found in the sample.

SS: How did you get all this data on the size of the parliamentary constituencies?

VBS: Parliamentary constituency and assembly constituency data were available at the national level. Below that, one had to go to the field. By that time, you know which are the assembly constituencies, which are the parliamentary constituencies you are going to cover.

SS: And you got this data from the Electoral Commission, or the Census?

VBS: The electoral data was Election Commission. We used to go to the Election Commission here for the parliamentary and the assembly constituency [data]. And once the assembly constituency was selected, then to select the polling station we had to go to the district headquarters of that assembly constituency. And there the polling stations were selected. At that time, once the polling station is selected, we used to take the whole list of voters at that polling station and draw a random number.

SS: And so the lists were also at the district headquarters. The list of every registered voter in the polling station.

VBS: Yeah. Team leaders’ work used to be going in advance. Somebody [went] to the district headquarters first before the team arrived. By that time, you are ready with the sampling of polling station and the list of respondents whom you are going to interview. So that list used to be prepared in advance before the survey team reached that area. So that would be handed over to that team, and the person would move to another district, make arrangements for the team’s stay — where they will come and stay and get the data, and how to reach the polling sample area.

The only difference which also was later introduced in [the] 90s, I think a little later — not the first survey but the second survey — earlier we used to say that the distribution of respondents [was] done based on the electorate size of the assembly constituency. Suppose a number has to be interviewed in Uttar Pradesh, and you have fifteen parliamentary constituencies. How [was] that number distributed? Based on their PPS size. So they will get their number, and that number has to be divided in assembly constituencies. That used to be done [with] PPS. Then how many respondents had to be interviewed from the polling station? That was also done on the basis on PPS.

Suppose one polling station is of an electoral size of 1,200 and the other is 700. So the 1,200 polling station will get a little more and this 700 will get a little less. Which was later made equal. Actually, thirty interviews needed to be done from each sampling point.

And then earlier, it was only a kind of random number. Suppose we have 1 to 700. So the three-digit column was taken from the random table. You select, you may get 413. That will be your first respondent. Then you get another one. That may be 102. Third may be 435. So you are from the same area getting this third respondent. There used to be a chance actually in that period that sometimes you got two respondents from the same household.

SS: Sorry, I didn’t follow. How did you draw the respondents from within the polling station? Did you do a random start and step?

VBS: Random sampling, but the random number, actually, have you seen the random tables?

SS: No. What are random tables?

VBS: [searches bookshelf] That book is not there. There used to be a random number table prepared. Where there is only figures. There would be actually two digit rows. So there would be three-one, two-one, one-three, zero-two, zero-five, seven-one, nine-one. All these random tables were prepared randomly by the machine. And that table used to be available. We only had to decide that, we had to draw from how many digits. If it is a kind of two digit number, then you have to have only first row. And then suppose the first number was 71. Second number was 03. Third number was 75. You go on doing that until you reach the end. The random number was prepared in such a manner that the same number cannot fall twice in the same poll.

But the danger was that to get two respondents from the same family at the same locality.

SS: What would you do then?

VBS: We used to interview them. But later on we realized that there is some flaw in this technique.

What to do then? Then we decided that we had to interview thirty persons from this polling station, and we had a sample size of 900. 900 has to be divided by thirty, so you get a quotient of 30. You draw a random number, which is less than 30. Suppose you get 14. So the fourteenth will be your first respondent. Then keep on adding 30 to that. And you will get thirty in your list. Doing this, you get the representation of all the area of that polling station.

SS: When was the first survey that you used that?

VBS: I think this was done some time in… ‘95? I think it was done some time in ‘96 or ‘98.

SS: So for the early projects, ‘67, ‘71, ‘80–

VBS: That was a kind of random number where there was a danger of getting the proximal respondent in our sample and some areas were getting not properly represented.

SS: Because of the way you drew the respondents within the polling station. It seems like you did that circular sampling for the parliamentary constituency, then the assembly constituency, then the polling station. But then you didn’t do it for the respondent.

VBS: Not in the first two or three studies. ‘67, ‘71, ‘69 — ‘69, there was a kind of mid-term poll in four states.

SS: Which states?

VBS: Uttar Pradesh, Bihar, West Bengal, and Punjab. We did that ‘69 study as well. That was [a] mid-term poll. And then ‘71 study. Then ‘80. ‘77 was done — only a few constituencies were studied.

SS: There were a few studies in constituencies in ‘77?

VBS: ‘77, we didn’t do that survey research. It was a few case studies.

SS: I’m wondering about the timing of it. Why you chose to do it for ‘67, and then the mid-term [in] ‘69, and then ‘71, and then you didn’t really do a full study again until 1980. Why not do 1977?

VBS: Two things. One was the funds. And the other was a kind of lack of faculty support. Because I was the youngest in the first generation of researchers. So by the time we reached [the] 70s, this first generation of scholars, they lost interest.

SS: They lost interest? Really?

VBS: Not that enthusiastic, yes. But not energetic enough to undertake this kind of rigor of survey research.

Because that was quite, as I said, that was quite time-consuming, unlike today actually. [CSDS Director] Sanjay, suppose today he conceives that [he] has to do a nationwide survey, he can plan and do it, implement in seven days’ time. Because the team is already there. There are trained people, actually. You don’t have rush experts to train Tamil Nadu team or Karnataka team. There are local teams, people available. Every person of our network has some team of committed field workers who take charge and recruit how many are required, and they send and collect the data. And they deliver it. And the questionnaires are now in such a manner that they will send the data directly to you.

Earlier it was not possible. As I said, you had to prepare the questionnaire. And then you had to quote the data, check the data, clean the data, get it punched, get it on the cards, take it to the computer. The computer will take so much time. There used to be a long queue in the computer. Your data will be processed tomorrow. So you have to wait. And then tables used to come. Sometimes there need[ed] to be a kind of merging of various categories. Somebody had to sit and manually add two columns together and then work out the percentage once again. It was quite time-consuming.

So they thought that they had got sufficient information to theorize, with the support of whatever they had got. Number one.

Number two, I was still young and I was not that competent to theorize, so I needed to divide this activity. So they used to say, “Okay. Go ahead. Get it done.” I did try but at that time, actually, in 1971, the Indian Council of Social Science Research (ICSSR) was already founded by 1971. It was founded some time in 1969. So they funded a large number of election studies in 1971. They gave grants, small, big, to various professors located in different universities. They got the project and they never submitted the report. So ICSSR decided to not fund any more.

SS: Because they just gave money away and didn’t get anything for it. Professors just took the money.

VBS: Yeah. So the funds were not available. At that time I was not that competent enough and known enough to get funding from some other sources. So the funds were not available. That’s why in 1977 I got a kind of very small grant from Indian Council of Social Science Research so we had four case studies. One in Kanpur, one I did in Azamgarh, one my colleague did in Muzaffarnagar. In 1986, we got a small grant to do a study of Haryana assembly elections. But not a large-scale survey project because the money was not available.

SS: It was in ‘80 though? In ‘80, you could get it?

VBS: ‘80, actually, we didn’t do directly. In 1980, we didn’t get a direct grant. De Costa, actually, [of] the Indian Institute of Public Opinion (IIPO), he managed the grant from, I think, ICSSR or some government [body]. But he didn’t have the expertise, so we helped them. IIPO got the grant [and] did the study, but we helped them to prepare their instrument, sampling, and team. The promise was that, “You’ll [IIPO] get the money, we’ll [CSDS] save the data. We will get access to the data.”

And then the first major study we did was in 1995, the Bihar assembly election study.

 

On the establishment of Lokniti, and survey research in the 1990s and 2000s

SS: How did that come about? In ‘80, IIPO did the study. The last one CSDS was 1971. So what happened after twenty-four years, that you decided to do a study of Bihar?

VBS: The interest was always there actually. Yogendra [Yadav], he was spotted by DL Sheith in one of the seminars in the Institute of Advanced Study, Shimla. He was making one presentation there. At that time he was teaching in Punjab, Chandigarh. DL Sheith was very much impressed by his presentation there. Then he invited him to make another presentation here in CSDS. He made [a] presentation here and then all of those people who were there at that time persuaded him to join us.

SS: You saw him and knew, “We want this guy.”

VBS: So he joined. He joined and by that time he had already seen my writings and my books were there on elections. So he said, “Singh saheb, let’s start once again.” I got a kind of good support in him, then we got a kind of small proposal for this Bihar study. And it was granted. How much money? The quantitity was 1 lakh, 11 thousand… or 91 thousand? Less than 1 lakh rupees. It was a post-poll survey. So we went there to Bihar, and the poll was to take place two or three days after that. We thought that we would be training our investigators two days in advance. They will take two days to prepare themselves to go to the field. By that time, the election would be over. At that time, the idea was to do a post-poll but before the results were declared.

We went there and we completed the training. The second day morning, when we were to take our investigators to have field experience, the news came that the elections had been postponed. Then we were really in a fix. Subrata Mitra was there, Yogendra was there, myself, and Sanjay [Kumar]. Then we sat together. What to do? We decided that suppose we ask our investigators to go home and then again come and do it after the poll, then we don’t have that kind of money. So what to do? I don’t remember who was the first to forward this idea: why not make this [a] pre-poll survey? Instead of asking, “whom did you vote for?” let us ask, “For whom you are likely to vote supposing the elections take place tomorrow?”

So we jumped on this idea, and a few questions needed to be altered. Other questions, any time they can be asked. Before the election, they can be asked after the election. But about the decision-making, and what happened during the voting day period, those kinds of questions were altered [to include] “suppose the elections are going to take place tomorrow.” And we did it. We sent it out. And we ourselves also got two days’ time. We roamed around. Somewhere we went to drop our team members to different places and came back. Talking to the people.

And that was going to be over Lalu Prasad Yadav. He was fighting for the second term. By that time actually, the Bihar intelligentsia, the opinionated people, the journalists, everybody had already written off Lalu.

SS: They thought he was going to lose.

VBS: In 1995. That he is going to be defeated. And we used to be ask people, “How will he be defeated? Because there is no single front against him.” People used to argue, those who are informed: “No need to align against him. The voters have made up their mind to defeat him and they will choose their own candidate to defeat Lalu. They have already made up their mind to vote Lalu Prasad Yadav out, so whoever candidate they find in the constituency competing, they will vote for that person.”

When the data came, actually, then we got a fair amount of idea that Lalu — although he had entered into some kind of understanding with Left parties, some seat-sharing — people were saying that at the most he will be able to get a kind of minimum majority and with the support of the supporting parties he will be able to form the government. Our data showed that he is going to get single-handed the majority. In this month — this was February — we were deliberating what to do. We had got the data, but none of us, we didn’t have that amount of courage and we had never done it advance. And we didn’t want at that time to publish that Lalu is going to win before the elections were completed.

SS: Because this was such against conventional wisdom. Everyone thought he was going to lose.

VBS: Yeah. But we trusted our data. We had very good faith in whatever we had been able to see. We were very confident that Lalu is returning to power. But we couldn’t get that courage to publish it before the elections. One, we didn’t want in the eyes of the Election Commission that we ha[d] done something wrong. And the other, we didn’t have that kind of courage also.

But we couldn’t hold it. The day the counting was scheduled, we gave handouts to three, four newspapers and only two of them published. One is Indian Express, the other was The Statesman. Times of India didn’t publish and Hindustan Times also didn’t publish. And we said, “We have done such survey, this and that, this is the sample size, this is the conduct, and this is the result.” And in the morning, the counting started and those people who read the newspaper started phoning us: “You people have also turned out to be admirers [of Lalu].” There is one saying at that time, Lalu chalisa. It means, those who are praising Lalu like Hanuman. “You people also started praising Lalu Prasad Yadav.” We said, “No, actually our data is showing that.” They said, “This time your data is going to fail you!”

By the afternoon, the trends were available and the same people started phoning back. “Sorry, your data seems to be right.”

And soon after that, a fortnight after that, there was a director’s meeting at the ICSSR. All directors of different institutes in a year, once or twice, they meet in ICSSR. Sometimes in Delhi, sometimes outside. At that time, Jiram Reddy was the chairperson of the ICSSR. By that time this report came in the paper and the results were there. Jiram Reddy asked me, “Why don’t you give the copy?” I made an almost nine-page article, “Why did Lalu win?” The elections were staggered in four, five phases. And every phase forward, Lalu’s chances improved. This has happened this time [in the 2015 elections] also. He asked [for] the copy. I sent that. He got something around one hundred copies of that article and he circulated [it] to relevant places. In that director’s meeting, while he was speaking to the different directors that use public money and decide what kind of research we used to do: “Here is the example. Dr. Singh has done this study.”

SS: So it was great publicity.

VBS: He was so… I knew him earlier also. He had visited us earlier, actually. He was also carrying out some study. For that, he came to Bashiruddin Ahmed and then took our help. So he was just kind of ga-ga about our findings and our quality of research. And I became a kind of darling of Jiram Reddy.

And at that time in that meeting during the lunchtime he used to talk to me. At dinner time, he spent time. And then he said, “Singh sahib, you do [a] similar study for the Lok Sabha elections.” I asked him, “Will it be possible for ICSSR to fund such a mega-budget project?” He said, “How much?” I said, “Something around 20, 25 lakh rupees.” He said, “That’s all?” I said, “Yes.” “Okay. Done! Start working on that.” And he said this thing in front of the members secretary.

SS: And this was the head of ICSSR.

VBS: After four, five months, then we started working on the proposal and other things. Yogendra Yadav really prepared a very good research proposal. The model research proposal, actually. He submitted that proposal to ICSSR.

It was going to be considered. Meanwhile, [Jairam Reddy] was in a meeting in Oxford University and there during the dinnertime, something happened, and he collapsed and he never returned. So we lost all support.

SS: [gasps]

VBS: But the member secretary, he was very loyal to [the] chairperson, because he brought him there. And our conversation took place in his presence. So he said, “Singh sahib, don’t worry. We’ll do it. But we are not able to fund the same amount.” But he funded something. 17, 18 lakh rupees.

Subrata Mitra also managed some funds and then we polled together and we did all sorts of studies in 1996. We did exit polls. We did pre-polls. We did post-polls. Because we had funds.

And thereafter, that exit poll was done for one Nalini Singh and associates who were working with Doordarshan, Delhi Doordarshan. [They] funded that. And we had that data. That time was the first time I saw [a] mobile telephone.That was double the size of this [holds up landline phone on his desk and drops it]. And very heavy weight. So we got most of the data, some of the data we used to get on telephone. Yogendra used to be sitting on the screen and we used to supply data to him for the deliberation. All of us, actually, one after the other, appeared. And that happened to be a 15,000 [respondent] sample. For all India!

And we did predictions and it was 99% correct.

SS: That was the election where–

VBS: 1996.

SS: Where the BJP got the largest vote and formed a government for thirteen days.

VBS: We went wrong only in Karnataka because of our sampling. In there, some tribal constituencies were overrepresented. The share of Karnataka should have only half the constituencies of [the] tribal belt, but we had two constituencies. I don’t know how that happened. So we went wrong in Karnataka, but nationally — since the sample was national[ly] drawn — we were very correct.

Thereafter we were so far in media and other thing[s]. And then we were very, very critical of ourselves [in] predictions of seats and doing research for media. It is not a kind of serious research. We collect data and we make presentations on electronic media, and we do some small writings for newspapers, and then we forget about our data. So it [was] a very great loss of our academic caliber.

We should disassociate ourselves from the prediction of seats. But we have to monitor and do the surveys of all elections so that our database for all these stages are available, so that we can understand the political processes operating at different points in time. Based on that, actually, we can very easily — if someone is serious to do — we can predict what is going to happen in the next ten years.

That should not be last. Therefore, we keep on doing surveys but not for seat predictions. But for [understanding the] parameters changing, considerations changing, support bases of different political parties changing, what that change indicates and why that change is taking place, whether it has moved or going back. All these kinds of things. That interest is there. Since we don’t do predictions — we are happy enough to do post-polls but before the election [results are announced] — we get the real picture.

Apart from this, still there has been surveys… there have been Ashis Nandy’s survey of displaced persons in the party, and block-level planning projects, and some other projects. But when Lokniti was constituted in CSDS, that was the period when CSDS was going through a very, very critical financial constraint. For doing these election-related surveys and other things, the funder used to approach us. We [didn’t] have to go the funder. So There was a large amount of money available. Ashis Nandy at the time was the director. He said, “Today we have a large amount of funds for this. Tomorrow we will not have. So let us have a kind of separate programme for these electoral studies.” They decided that we should have an Institute of Comparative Democracies. That was the English name, and the Hindi name was Lokniti. Later on we decided that it should not be Institute for Comparative Democracy but [a] Programme for Comparative Democracy so that we can block this money for the rainy days. Suppose we don’t get a grant. There should be something we can invest from our own. And therefore there was a need of a separate programme and [to] block this money for some institutional purpose.

Any more questions?

SS: Yes. How it evolved after that, after the 1996 National Election Study and the formation of Lokniti– It’s become a very regular thing now. For every state election, every national election, CSDS does an election study. It seems like it’s shifted between doing some pre-polls, and then they did exit polls, and they had a relationship with CNN-IBN and they were doing projections for CNN-IBN. How that whole process unfolded… It seems like there’s been a kind of constant evolution in CSDS’ mission and its approach to survey research. It seems like it’s something that CSDS is still even now trying to develop.

For this last Bihar election, they did a pre-poll study but they also did a post-poll study. I remember when I first arrived here talking to Sanjay Kumar about it. And he was saying, “We used to do seat projections, but we don’t like to do them anymore.” Because it’s exactly as you were saying. It’s not as scientific a process, and you’re kind of playing this media game where if you project it and you get it right, then people think you’re a genius. And then you get it wrong, even if the principles behind it were good, people are going to criticize you.

In terms of your own involvement, are you involved with the surveys these days? It sounds like you were very involved in the 1990s and the reconstitution of survey research. What was your involvement from the end of the 90s going into the 2000s with Lokniti and surveys here?

VBS: I am still with Lokniti. I am still with Lokniti. I have been a very active partner of Lokniti. Formally, I retired in 2006. Until that, I had a very, very formal role to play in Lokniti. We are partly to take a collective decision with regard to what to do and what not to do since, at one point in time, fund-raising was kind of our goal.

You see, actually, why we fell into that media trap. Because we wanted to get some money. Because at that time this IDRC funding was not available and ICSSR was very, very minimal, only supporting up to 60% or 70% of our salaries. So to pay the salaries and get some money for our in-house research, somebody who is not working on projects or not on funded projects. If that person wants to go abroad or go to some other library in some other state, from where can that money be got? That was the kind of consideration we wanted to [address], and we were able to sell ourselves better at that time. So the funds were available.

But at the same time actually, there was always a kind of unease among the collectivity. What is the difference between CSDS and the other media, these poll companies? They do it, and we do it, and then there is a kind of bickering in the institution. If we say something favorable for one party or unfavorable for another party, our people say that, “You are pursuing this kind of ideology.” Or: ”You have preference.” Or: “You are misrepresenting. This is not the collective thinking of the institution.”

This conversion of votes into seats in India is really very, very difficult because [of] the complexities of electoral politics. Competition typologies. Regional parties. There are some parties which cover only three districts in a state. And you are doing a pre-poll survey. For seat predictions, suppose somehow that that area doesn’t come into the [sample]. So predicting vote is scientific as far as the Indian situation is concerned. Converting that vote into seats has a lot of problems. So chances of your being right and chances of your being wrong is almost 50/50.

Suppose there is a kind of direct contest, as happened in Bihar, it was very easy. And this has happened in the history of Bihar since 1952, until date, no election was as directly contested as this time. 1952 onwards, we always have a three-cornered, four-cornered [contest]. Independents in different constituencies. And you see here, the chances of independents winning elections was quite high. [It] used to be that one-third, one-fourth of candidates selected in the house were independents. But later on they had been marginalized. They had been sidelined by the elected [officials] saying, “Independents can’t play any role, any constructive role in [the] house.”

So a large amount of discussion took place in our Lokniti group to restrict ourself to telling about the mood of the people, but not converting them into seats. Not spending too much time on this rigmarole of survey research. We should do what we require for our understanding of the subject. So that our understanding is not hampered by the lack of data to analyze and understand the democratic institutions, and the problems confronted by them in this country, and where we are heading, and what problems we confront. So for that whatever is required, we should do. But we should not enter into this kind of entertainment competition of different media. Number one.

Number two, [because of] the kind of method we apply in our research it is very difficult to get that kind of funding from this media. Our surveys, actually, we spend something around, not less than 500 rupees per interview. And these media people, they give them results with 120, 125 rupees per interview. Because they don’t have to spend so much, they will just increase [the] number only on computers. They will do interviews of 10,000 people and they will say they have done it for 60,000 people based on that. They have their network of commercial people in different areas. They will select people. They will send them. Say, “Go to this district, to this constituency. From the headquarters you go three kilometres west and interview one village. And you go five kilometres south to that headquarters, and visit another village there and do thirty interviews. And you are paid per interview completed.”

If we sent our people with a randomly selected list of names and ask[ed] them to meet people and interview only those persons, if out of thirty you are able to meet only three persons — you turn back with an explanation for why you have not been able to interview twenty-seven — we are more than happy. It won’t affect your payment. We don’t pay the type of investigator… we recruit through their teachers. So for them, it was not only a type of earning proposition, it was there for learning. They come here to learn. Only at the end of the day would we take care of their expenses and give some money at the end so that they can have some pocket money. Saying, “I spent one week outside my class or my home and I have three thousand, four thousand rupees in my pocket.”

The same survey, if they want to get it done from us, that will be five times, three times costlier than that. So they’re not ready to do [it]. But still there are some people, some media, they still want [us] to do [it]. But we say that we can’t predict the seats. And for them, actually, the greatest entertainment is only to cover the votes-into-seats. That made the change, when we stopped our pre-poll surveys. Because in pre-poll, whatever stage you do — suppose you do it three minutes in advance — even then you will be asked to predict. Predicting seats based on a survey conducted three months earlier, and then what happens during that three months — God knows — but some people will remember that, “You said that the BJP will get this much seats in Bihar. But they got only 53.” If they are not aware of what you have done. So we have slowly kept ourselves out of that and we are happier for that. We have no complaints. It was a collective, wise, well-deliberated decision.

SS: When was that decision?

VBS: This was… You see, actually, we decided not to do that prediction but we helped them to get one. Rajeeva Karandikar, located here in Bangalore, he used to get data and used to do something, and used to convert that. But people were not able to differentiate between Rajeeva Karandikar and CSDS because the survey was done by CSDS, Rajeeva Karandikar made the prediction, but Rajeeva Karandikar’s name is lost. So it happened from 1998, ‘99, 2004… I don’t remember whether [it was] 2009 we stopped it, or ‘11. It was decided, I think, three general elections before. It must be 2004, or ‘09. But this you can ask [someone else].

 

On reflections

SS: So you’ve worked with CSDS then during the 60s, 70s, 80s, 90, 2000s, 2010s. How has the public profile of surveys and quantitative methods as a means to study politics in India changed — or not changed — over the decades?

VBS: There is not much qualitative change, but in terms of approaches, time and energy spent, with the media exposure, technological development, means of transportation, these things have made surveys much easier, number one.

Number two, I would say that this younger generation who are engaged in survey research, there [are] some sort of shortcut methods. Very few survey researchers who are doing major writings based on survey data have much exposure to the pre-poll. Earlier, the people who used to analyze data — it was always analyzed by the technical hands — but use that analyzed data and give meaning to that, used to spend some time in field to understand the nerves of the people.

Number three, you save survey data for long-term theoretical considerations and reflecti[ons]. That kind of research. We had very, very great hopes from Yogendra Yadav. We were expecting something will come from Yogendra. What Rajni Kothari did in the 60s and 70s, Yogendra will do in the 2000s, the 21st century. That kind of dissatisfaction. The kind of writings we are seeing in the form of reports and other things, that depth is missing. We are sitting on a mine of data and I am pained to say that we are not able to use more than 75% of the data which we have collected.

SS: Why not?

VBS: Because this is the kind of thing, there is a lack of strength in our CSDS as well, apart from Sanjay. There are hardly any persons actually. Earlier there used to be at least more than three persons, senior-level, using this empirical data and doing some long-term work. But Sanjay has too [many] tasks. He is now holding the post of director of this place, so there is institutional responsibility. Elections are too frequently coming here and there. So that kind of research is missing.

SS: More theoretical, long-term, systematic looks at the data across elections, rather than one election study after the other.

VBS: Suhas Palshikar. We requested [to] him several times, “Why don’t you come here? Spend a few years just concentrating on using the data.” We need that kind of thing. Yogendra has already gone into active politics.

That is the kind of problem, or sadness, that we face. I am ready to say [it is] my responsibility, but not able to create that amount of time, but now actually we have money, we have time. But [we need] to find the right kind of people.

The change that has taken place in the current generation of academicians, no one has time to do this: long-term using cross-sectional data for two decades, three decades, digging out information. It takes time. If you are working on this, you are not going to produce a book after six months. You are going to produce a book after three years. That kind of patience in the younger generation is missing. There we have some expectations and hope from people just like you [laughs].

SS: Thank you very much.

Yogendra Yadav on Lokniti’s origins, the challenges of polling in India, and the alternative politics of Swaraj Abhiyan

yogendra-yadav_650x400_81425277261

For many Indians, the name Yogendra Yadav is synonymous with polling and psephology. In nearly all my previous interviews, his name has been repeatedly mentioned as an unparalleled authority on polling in India. Well-recognized from his days of forecasting election results on CNN-IBN, he is a widely respected scholar of political science who has written countless articles and edited several volumes on Indian politics.

After eight years of teaching political science at Panjab University in Chandigarh from 1985 to 1993, Yadav joined the Centre for the Study of Developing Societies (CSDS) in Delhi and brought the Centre back into the field of election studies after a two decade hiatus. In 1997, these studies were installed at the Lokniti Programme of Comparative Democracy, where they have continued ever since. The founder-convener of the Lokniti network of professors who fielded and analyzed these studies, Yadav was the director of Lokniti from 1997 to 2003. Since 2004, he has been a Senior Fellow with CSDS (full disclosure: CSDS is my primary research affiliate, and I am working with Lokniti on my research).

In 2012, amidst the India Against Corruption protests shaking Delhi, Yadav left psephology as he became gradually more involved in politics. He co-founded the fledgling Aam Adami Party (AAP) and contested the 2014 Lok Sabha elections on an AAP ticket. Yadav was a senior leader in the National Executive when the party swept the Delhi assembly polls in February, winning sixty-seven of seventy seats in the Delhi Vidhan Sabha.

Shortly after this historic victory, Yadav and fellow AAP co-founder Prashant Bhushan wrote a joint letter to the National Executive questioning Delhi Chief Minister Arvind Kejriwal’s unilateral decision-making approach and expressing concern about compromising the party’s core principles. This started a series of allegations and counter-allegations that ended in March with the expulsion of Yadav, Bhushan, and two other senior AAP leaders, Anand Kumar and Ajit Jha, all for “anti-party activities.”

Yadav and Bhushan continued their involvement in politics through Swaraj Abhiyan, a grassroots movement for an “alternative politics in India” that they started earlier this year. In June, Swaraj Abhiyan launched Jai Kisan Andolan, a movement focused on bringing the plight of Indians’ farmers into the national discourse. Swaraj Abhiyan has since conducted a number of protests and yatras, or marches, to focus on farmers’ issues.

Last Tuesday, I met with Yadav at his apartment in Patparganj to talk about what he terms his “previous life” in polling and his present life in politics. The transcript of the conversation below has been lightly edited for length and clarity.

On public opinion research in India and starting Lokniti

Sam Solomon: You spent several decades doing public opinion research in India. I’d like to know how the methods of research changed during that time.

Yogendra Yadav: I spent two decades doing that. I was lucky to be at CSDS because I think the foundations were already laid there. While people remember the work that Lokniti team did, I did, my colleagues did, unfortunately not many people remember the foundational work done in the 1960s and 1970s. I think we really built on that. We were not doing something radical or innovative. In the 60s, people like Bashiruddin Ahmed, DL Sheth, and VB Singh, along with others, were involved in laying the foundations of political opinion, attitude, behavior research in India. And in many ways we simply built on that. There is an article which traces this entire journey which I had written a lot but is under a generic team name. It’s in the EPW of 2004, I guess? Which is “National Elections Study, An Introduction.” And this piece traces first generation, second generation, third generation of survey research in India. So I guess we made some innovations. We made some improvements. But much of what we did was by way of fine-tuning an already existing model from CSDS.

What we did was to expand it, expand it in terms of the sample, expand it in terms of reach, and expand it in terms of taking it to the public and making it more popular and visible. We made it more frequent. We made it more of an archive, a systematic archive rather than an episodic thing. And we tried to integrate that with an ongoing political commentary. I guess that’s what we did.

Unfortunately, much of that work is known for its forecasting, which is how we got the money to do whatever we wanted to do because we moved away from that foreign-funded research grant model. And naturally people remember the forecasting part of it, which wasn’t bad, which on balance I think was a satisfactory experience. But that’s not why we did it. That’s not really what we wanted to do.

I think the part that we did somewhat better was the post-mortem of electoral verdicts. That was our forte, and I’m glad we managed to do that. And I think that is the part that CSDS has continued after I left. And I feel happy that they are doing it in many ways better than when I was there. I looked at this Bihar thing and the previous election coverage. I felt that this was much better than what we would have done if I was around.

SS: So what were you trying to do then?

YY: This was continuity of the CSDS tradition which is to understand public opinion, attitudes, and behavior.

When I turned to research on Indian politics, which was not my original area, I discovered that the field was shockingly bereft of any serious evidence. I was trained as a student of political philosophy and theory. After ten years of research on that, on intellectual history, on political theory, and such like, I shifted to making sense of Indian politics. And I thought, I came to CSDS in the hopes that much of the evidence lies there, and I just have to go, understand it, and try to out it. And then I discovered how not only did we have a dearth of evidence, but the overall temper, overall style, overall mood, and mode of doing research on Indian politics was something which looked down upon any evidence at all. So much of this was hearsay. Much of this was pure speculation, and I didn’t know how you could research on empirical questions without gathering systematic evidence.

So the entire effort in that decade or two was to create an archive of public opinion and political behavior. And I do think that now we have something of an archive to go back to. I feel bad that we didn’t use that archive as well as we could have used in terms of coming up with theory, in terms of coming up with an understanding which would be different, distinctive. So I think people like me are guilty of underutilizing the data that we created. But frankly the task of creating the data itself was so huge that we were just snowed under.

SS: And the archive is there for scholars who do want to do that.

YY: For future generations, who I hope would do a better job of it than we did.

SS: What were the greatest challenges that you encountered when you were with Lokniti in terms of accurately measuring public opinion?

YY: Let me say what the challenges were not. There is a misimpression that somehow Indians are particularly immune to being measured by survey research. I find that attitude silly. I have never encountered it. If anything, doing surveys in India is easier than doing it in other parts of the world. Especially in the First World, in one respect, that the refusal rate is very low. If you go, especially in rural areas, not only do people give you interviews, they actually welcome you with a cup of tea. Because the survey penetration is so low that actually you can conduct very high quality face-to-face surveys in India and the costs are reasonable, of doing face-to-face surveys. These are not impossible costs. So India is actually still one of the few places in the world where you have the conditions permit a face-to-face survey where the costs allow that, people welcome it, and there is a trained scientific manpower which can actually carry it out. There are very few places in the world where the combination of all these things exists.

So this is really not a problem. Unfortunately, whenever some survey goes wrong or some forecast goes wrong, people fall back to this stupid excuse, that they say, “It’s impossible to do that in India. Indians are very different. Indians are unique.” I don’t think there’s anything of that kind.

What is also not a challenge is sampling. Your piece indicates that there is a problem in India, which is, I mean, we don’t have that kind of stratified data available at the micro-level. But if you go by strict random sampling, you overcome all of that. And because strict random sampling is possible in India, given the census data and given the electoral rolls and so on and so forth, we have been able to circumvent it. And the quality of samples, especially in the polls done by CSDS, has been quite good. I think by international standards these are very high-quality surveys.

The difficulties, the challenges we faced were the following: Number one, the academic culture of Indian social science really looked down upon survey research. So if the theorists were the Brahmins of Indian social science, survey researchers were the coolies. No one thought this was worth respecting. And this occupied such a low position in the hierarchy of Indian social science that the business of gathering evidence, especially on politics, was simply so underdeveloped. Fortunately, in other disciplines like history, for example, you could get up and ask a question on sources, which is to say, “What is your evidence?” But in political science seminars it was rare twenty years ago for anyone to ever ask a question about evidence. What is the basis on which you are making these grand generalizations? No one ever asked this. So that was one problem, a lack of research culture.

[Number two], there was lack of trained personnel in political science. India has a great tradition of statistics. India has a great economic traditions. But in politics, people who were trained in doing surveys and handling quantitative data, it was very, very rare. It still is unusual to find people who can handle data with some degree of sophistication in political science. Otherwise, Indians are spectacular when it comes to data and mathematics. This was the second challenge.

As a result of these two, we did not have a community where there would be a lively exchange of methodological and substantive issues in survey research. There was no methodological innovation and frankly no one actually understood. No one cared. Just as ordinary newspaper readers pick up the newspaper and say, “Oh, you got your forecast wrong. Your survey must be fraud.” Similarly political scientists picked up these things. Most political scientists could not even read bivariate tables in any sensible way, meaningful way, intelligent way. So methodological things, people simply didn’t understand.

While there has been a very intense culture of debate on politics in India, the culture of debate among political scientists was not careful on issues of middle-level generalizations. Most of the Indian debates have been at the very abstract level of generalization. So the lots and lots of generalizations that you need to build a hard theory of politics are simply unavailable. And unfortunately then much of these models came from a readymade repertoire of democratic theory in the West, which came largely from North America. And those theories simply don’t work. It’s a very different context, very different country, where you can learn a thing or two from constitutions. You can learn something in a very grand, general way. But middle-level generalizations about politics are so deeply anchored in the specifics of politics of that locality that to pick North American theories and to apply that to India was so weird.

So on the one hand, we felt we suffered from that complete deficit in our academic culture. On the other hand, we suffered from this excess export from North America. And it was a struggle for us to try and handle that situation there in Lokniti.

And I think we were very, very lucky in the mid-90s when we brought together that group which is called Lokniti today. It was an exceptional group in many ways because the quality of political scientists who agreed to join that, and the simple collegiality which you must have experienced, is rare. Academics don’t work with each other normally. Academics suffer from more envy than ordinary human beings do. But in Lokniti somehow we managed to create that collegiality and this is a group that has now worked for… more than twenty years? Twenty years, it’s just completing. So we were very lucky in terms of collegiality. We were very lucky in terms of intellectual integrity of many of the colleagues in [the] Lokniti group. We were lucky in having administrative and sheer hard work of many of the younger colleagues who we interacted with, who really laid the foundations of this work.

We were not so lucky in having leaders who could connect theory to empirics. And that’s where the limitations of people like me came into [the] fore. Some of us had to do that work. I don’t think we did it sufficiently well. So those were some of the issues and limitations.

The other big issue was lack of committed funding. Every time… I don’t know of any other research archive in the world which had to go through such financial stresses as we did and which had to do all kinds of things just in order to keep afloat. All kinds of things, exit polls, this poll, that poll, because we had no research grant. And it was astonishing that perhaps now Lokniti archives should be one of the biggest archives in the world on public opinion and attitudes and it is a shame that a large experiment of this kind did not have any sustained funding for what it was doing. And we approached everyone in the Indian establishment for that and no one was willing to support that.

What that did was we ended up investing… a great deal of our energy was exhausted in keeping the ship afloat than in steering it. That’s a challenge we had.

SS: Do you think that the existence of Lokniti effected any change in the culture of political science or among academics here?

YY: I dare say it did. Although it’s sort of for future generations to judge. For people like you to read and check and come to an independent conclusion.

I think it did, and in the following ways: Twenty years ago, you could comment on Indian elections, Indian political parties, Indian public opinion without bothering to dirty your hands with any evidence at all. Today it is far more embarrassing to do that than it was then. So we have made it embarrassing for people to do things without evidence. People do it grudgingly. People [do] it while pretending that they’re not doing it. People do it by way of taking potshots at us. But they do it. That’s important.

I must say it was partly facilitated by something which people don’t readily acknowledge, which is that our work gained acceptance in North American and European academica. Because academics working on India were desperately looking for some evidence. They at least knew that evidence matters. And there was none. So very early on in our work, people started appreciating the quality of data that we were generating. They were probably not into our perspective. But they understood the value of data. They started referring to it, and so did media. So Indian media and Western academia started respecting our data. Initially, Indian political scientists and Indian social scientists were indifferent and almost defensive and aggressive. But I think unfortunately given the unequal relationship between Western academia and Indian academia, Indian academics gave in. After, I think, the first five or ten years, it became very hard for them to ignore it because the people they cited cited us. Strangely, by this ironic twist, that is what gave us much greater acceptability. So I guess that has changed.

I also see now, wherever I go in the country, I see young teachers who come to me and say, “I was part of Lokniti workshop. I have done a National Election Study. I have been a researcher or trainer.” In the discipline of political science, there [were] so few opportunities of doing field connect that Lokniti and CSDS work came as a window of opportunity. And lots and lots of youngsters availed of that opportunity. And I guess that may have changed their orientation. There was very little research training available for political scientists. Basically, you read theoretical books and you picked up whatever impression you did from your rickshaw-wallah and from your taxi-wallah, and then you wrote books. But in this there was the first time an opportunity that people actually got seriously trained. So over the last twenty years, we must have trained a few thousand people. And that, I think, is an everlasting contribution which is not acknowledged because we are not seen as a teaching institution. But we actually have been a bigger teaching institution than many universities put together. So that, I think, has changed the culture. Insufficiently, but it has changed.

But, as I said, this is a biased opinion coming from the person who has every interest in overestimating the impact. It is for someone else to measure.

SS: My research is looking specifically at whether certain populations are, because of cultural or socioeconomic factors, more or less likely to be sampled or participate in a survey. I’d like to know over your career, throughout your experience, which populations you found to be … I’ll say, less likely to participate — I realize the question of whether groups are less likely to be sampled or participate is different — less likely to participate in a survey.

YY: Unfortunately, that’s exactly the kind of research I would have liked to see more in India which doesn’t exist. I should not be answering this question with reference to my impressions. Ideally, there should have been fifteen articles on this question. I should have been trying on those articles. But they don’t exist. That’s the problem. The research culture in Indian political science is so limited. So I speak with my impression.

My impression is that first of all it is a function of the method you choose: do you do face-to-face [or] do you do telephonic interview? Telephonic interviews are most exclusionary in India, and unfortunately many of the polls — not the election polls, but most other opinion polls that you see in the newspapers which are not in the time of elections and where there is no fear of being contradicted — most of those are simply telephonic polls. Worse, some of them are Internet polls. Now, that’s just plain unrepresentative. Even though we have telephone penetration which is substantial, even then, I think anything other than face-to-face interview is very unrepresentative. And it’s unrepresentative in simple class terms. It massively overrepresents the upper classes. That’s one.

It’s also a function of method in a different sense. Do you do an exit poll in the context of elections? Or do you do a post-poll survey, which is to say it’s a site? Do you do your survey in a public place? Or do you do it in the setting of their home? The surveys done in public places, especially exit polls, one of the most popular things — the trouble is as you picked up from that quotation… [my quotation of Yadav in The Hindu op-ed] although survey researchers like to believe that they have selected the sample, in reality the sample is selecting himself or herself. They walk to you. If you see the real life context of Indian elections, there is no way you can pick every nth person.

SS: I did see. I saw some exit polls for Bihar.

YY: You did? You went there?

SS: I went with CVoter and I saw some exit polls.

YY: It’s actually impossible. There is no way, if someone says, “Pick every thirteenth person,” how would you do that? There is just no way you can do it. We tried a few exit polls and we realized this is impossible. The bias of the researcher, and the attitude of the respondent, and the willingness of the respondent are far more important than any randomness or any method. So that is bound to overrepresent those who are more articulate, those who are more resourceful. And therefore, in terms of an Indian context, it is straight a function of your class and caste. In places like Bihar, I mean, almost all over the country, because there is a considerable overlap between class and caste, so every one of these things ends up oversampling the caste– the upper castes get oversampled.

First is the question of face-to-face versus telephonic or otherwise, second is exit poll versus this, and third, of course, is random versus any non-random technique. I think given the fact that we do not have sociodemographic information at the precinct level, I normally don’t trust anything other than a strictly random sample. Random sample with a reasonably large size gets you everything. But anything other than random sample almost invariably oversamples again the more articulate, the more well-to-do, men, urban. The slope of privileges is such that sampling tends to flow in the direction of socioeconomically more powerful groups: rich, men, urban, upper caste, media-exposed, educated. And since of these, except the gender, has a significant correlation with each other upon another, there is a very substantial overrepresentation of one section of society.

These three things put together account for the bias in sampling. And therefore over the years we went by the route, which I should say was established by the first generation in CSDS, which is only face-to-face interview, only at home, and strict random selection. Otherwise, you don’t get it.

SS: Something that I’ve seen in your writings, and this is something other researchers have also called for, is greater transparency of pollsters. So the people who are consuming this information can have a better idea of how this research is being done. Who do you think needs to take the lead in making those kinds of changes? Does it have to be the pollsters themselves? Or does it need to be media sponsors, the people who are actually paying for this research?

YY: Ideally, the pollsters themselves. If not, the media. If not, the officials, which is to say the Election Commission. And if not, the government.

Someone has to step in. This cannot go on infinitely because the industry has flourished without any regulatory framework. Now this is not an industry where regulation should come from outside. So yes, ideally, a council of pollsters or something should come into existence.

The only problem is that in India the record of such professional agencies to self-govern has been exceedingly poor. Whether it is the Medical Council, or it is the Media Council, the Editors’ Guild, and so on. Their willingness and capacity to defend the profession against outside onslaught in never matched by their capacity to reform within, and that’s why I remain skeptical that the pollsters themselves would be able to do [this].

The trouble is that the thing called the pollsters has two very different animals: there are places like CSDS, these are non-profit places; and there are these crass commercial operators who would not like to bind themselves down to very serious, rigorous protocols of transparency.

So sometimes I feel maybe the media and those who pay for it should come into it. Their problem is that they are ignorant. They simply don’t know what they are paying for. I rarely come across editors who understand even the basics of what they’re dealing with. So they are like [those] customers in the market who judges things by its package weight. Can the Election Commission do it? Don’t know. They have shown very limited appetite for regulating the media and they usually like very ham-handed methods like bans rather than any sophisticated regulation. And the government… God save us. That’s the last thing I would want to happen. So I don’t know, realistically speaking.

But yes, I continue to believe what you could do is have an agency which has representatives from pollsters and media advertisers which is sanctioned by the Election Commission to do this. And what you need is not a ban. I have repeated it, so much so that I now get tired of saying the same things over and over again. A ban would be a very silly way; it would actually push the industry under the carpet. It would actually complicate the matters even further. What you need are very strict disclosure protocols. I have written [in] several places about what needs to be disclosed, and everyone in the world knows what needs to be disclosed. I don’t have a particularly innovative solution to that.

The only thing I have added to that list is to say in the Indian context you must also disclose the social composition of your sample. You cannot tell me you have a sample of 1,900 and it was picked up from so many cities, so many this, it has this many men and women. Do tell me how does it correlate with or fail to correlate with the known social demographics of the area that we are talking about. That should be a mandatory thing in India. I need to know how many upper castes you have in your Bihar sample, and how many OBCs, and how many SCs, and how many STs. Unless I know that, how do I begin to trust your sample?

The second thing that I have added to that is to say the disclosure should be a three-tiered disclosure structure. One, certain basics should be self-declared, proactive self-declaration. Second, if someone asks you for some cross-tabulations, that should be made available on demand. Probably for a price, but it should be made available in principle. And third, if someone challenges the veracity of the survey itself, someone says, “The whole thing is a fake, you didn’t do it,” there must be a place where the raw data is stored and made available for a group of experts to come and check. It is not impossible to tell fake data from real [data]. If you have any experience of survey research, you should be able to make it out. And the data needs to be stored, and yes, a committee should be able to get access to your raw data, to go into allegations of fake data.

It’s not merely disclosure and transparency. It’s also regulation of quality. If you can set up something like this, then it’s all right.

SS: Have you heard out about the Indian Polling Council that’s being proposed? There was an article in The Hindu about it.

YY: I’ve just heard about it. If it comes about, it will be a good idea. And if everyone agrees to abide by it.

As I said, the record of professional self-regulation in India cutting across professions have been very, very poor: medical profession, legal profession. The only exception to that is auditors. Chartered accountants, for some reason, have managed to hold their bar reasonably high and they have not allowed it to fall. They actually give degrees and they have ensured that the degree does not fall below the quality standard. But otherwise it’s not been a great experience.

On entering politics and Swaraj Abhiyan

SS: I don’t really know, aside from you, of any psephologists or political analysts — I can’t think of any in the American context, I wouldn’t know as well about the Indian context — who decided to enter politics, after spending many years or decades of doing psephology. I’m curious to know what motivated your decision to enter politics.

YY: One exception. There was someone called Biplab Dasgupta. If you recall, one of the first books on Indian elections, voting trends, etc. was written by Biplab Dasgupta and Morris-Jones. I think it’s Trends and Patterns in Indian Politics. Something of that kind, it’s called. Biplab Dasgupta went on to become a member of parliament from the Congress party. Not that there was any relationship between his research and his politics. But he did. So I’m not really the first one.

In my case, it got much more noticed. I do think that [there are] many social scientists who went for active public or political life in India. In my case, it just happened to be more noticed, partly because I was somewhat known on television and partly because of the dramatic context in which I entered. I don’t think I was doing something else exceptional, but the notice that I got was exceptional.

Also, in my life, actually it’s not much of a transition, as colleagues in Lokniti will tell you. All those years when I was in Lokniti, when I was at CSDS, a full-time academic, I wasn’t there in the sense that I was actually quite torn. I used to spend quite a lot of time even then traveling in different parts of the country, meeting social movements. I was with a very small political group even then called Samajwadi Jan Parishad, which was led by my political guru, Kishen Pattanayak. So that was very much a significant slice of my life even then. It’s just that it was below the radar. No one quite noticed it. Not that it was anything secret. Everyone knew about it. But that politics was so spectacularly unsuccessful that no one cared to know more about it. But I was very much involved in a certain form of social and political action all these years of being in Lokniti.

And, as some of the colleagues would tell you, I was always very, very uneasy. I never wanted to be a psephologist in my life. I was, if anything, awkward and embarrassed about it. It wasn’t my life. All this might sound odd because I am known to be a somewhat successful one. But this is what I never wanted to be in my life. And I used to say to my colleagues… initially, they thought this was all pretension; academics have all kinds of pretensions. I used to say, “I am a political animal who has strayed into political science. And I want to get out.” People didn’t believe me because they thought this is posturing. I also used to say that I shall not retire as a professor. This is my nightmare. My nightmare was to retire as a professor at the ripe age of sixty-five or something. I never wanted to do that.

What I had not planned was, of course, the Anna movement and all the developments that happened. But, in fact, my very entry to CSDS in ‘93 was a bit of an accident. Because that is the moment when I had made up my mind to quit academia. I had done political theory. I had done intellectual history. And I was so unhappy with academia that I wanted to quit and go to my village. So in my journey from Panjab University, Chandigarh to my village, I got stuck in Delhi. I got stuck for twenty years. But I saw it as a diversion. So strange though it might sound, psephology was never, never number one priority one in my life. It was always number three or number four. Even in my intellectual pursuits, psephology was number two for me. Number two or number three. In my life pursuit, it was number four or number five. It so happened that it took substantial time and, in terms of public perception, this was the only thing I was known for. But I never associated myself with that.

So when I finally stepped out, I felt relieved. And actually I had said six months before joining politics, I had said, “No more forecasts ever for me in life.” This Uttar Pradesh assembly election in 2012, which we fortunately got right. And that evening on television, I said, “Thank you very much. End of my election forecasting. Never again.”

This business of election forecasting is something which attracted me and it did not attract me. It attracted me because it’s a kind of an itch. There’s a mountain and you want to climb it because it’s there. So election forecasting was like this. This is a technical riddle. Someone needs to solve it. And I knew at least two-thirds of it, of how to solve it. The remaining one-third never came. And I thought, if it’s actually possible to do it, why not do it? So there was a sort of technical riddle-solving part in me, a bit of mathematical curiosity, and a bit of political common sense, which I wanted to use.

The only thing I regret is that ideally, I would have liked to spend five years on this, solve it, and move beyond it. I spent fifteen, and didn’t manage to solve it fully, because there wasn’t any… I wish there was a group of model-builders that I could work with and a lot of background research on survey methodology. I think we were really one or two years away from having solved it properly. But we couldn’t do it fully. We did somewhat better. It wasn’t a record to feel ashamed of. But I think we were on the verge of a technical breakthrough which never happened. That’s the only thing I regret. But I don’t regret moving beyond that at all.

I was a political animal. My principal identity is that of a political animal. And what I’m doing now is what I really always wanted to do with my life. So no regrets.

SS: Was there a moment in 2011 or 2012 where you said, “This is my opportunity to get back to my original calling and step away from psephology?”

YY: Yeah. I mean, in a sense, 2011, the Aam Adami movement and thereafter was… as I said, I was very much in politics theoretically, I mean, notionally, because I was with a political group. I was very much writing. All those years of being in Lokniti, I ran an activist magazine in Hindi. I was the editor of a political magazine, which was read by very few, but it was an ideological magazine. I ran that.

I always wanted to be 24/7. Something in that context told me that yes, this is the moment to do it 24/7. So that’s the difference.

SS: I read a little bit about Swaraj Abhiyan online. I was talking to some of your assistants about it and trying to understand it. Why try to change India’s politics through a grassroots organization like Swaraj Abhiyan as opposed to through a political party?

YY: There’s no opposition between Swaraj Abhiyan and a political party because Swaraj Abhiyan is a political movement. It is not an NGO. It is not simply a social movement. It is a movement which very much aims at creating and being a political alternative, doing alternative politics. It’s just that at this stage of our evolution we are in no position to offer that alternative. We have neither the experience nor the critical mass nor the carefully-developed policy perspective. And that assurance that we would be able to create a democratic organization. There are several prerequisites which we are unable to fulfill.

But in principle I see no reason why we should shy away from doing party politics. Doing party politics is extremely important work. And although I would say that it’s important to do things other than party politics as well, party politics needs to be supplemented.

I keep saying there are five elements of politics: there is electoral politics, about forming governments; there is politics of struggles, movements, agitations; there is politics of constructive work, social, constructive, positive, good work; there is politics of ideas; and there is politics of the inner self. All these five need to be done. And the best politics is something that combines all these five. Unfortunately, we have reduced politics to parties, parties to elections, elections to government. We need to move beyond that. We need to think of politics in a more comprehensive way. And that’s what we would like to do through Swaraj Abhiyan.

So it’s not in opposition to a political party, but, in my ideal universe, Swaraj Abhiyan would do a politics that does not shun but subsumes electoral politics.

SS: When you talk about a “new politics” — this is something that’s all over the website and your assistants were talking about it as well. An “alternative style of politics.” What kind of politics are you talking about?

YY: As I said, a politics which is multidimensional, which subsumes what political parties, what NGOs do, what universities do, what agitations do, and what spiritual gurus do. All of that should be subsumed in politics. The best of politics has always subsumed all these. Mahatma Gandhi’s politics subsumed all these five elements. Which is what we need to do. Politics has to be a much bigger activity of transformation, including transformation of the self.

SS: I don’t know if you’re comfortable speaking about it, but why is Swaraj Abhiyan not working with AAP? I know a little bit about your history with–

YY: With AAP?

SS: Yes. Another way of–

YY: What went wrong? Why are we not with AAP?

SS: Yeah. Exactly. It seems like you–

YY: That’s too well-known a story for me to recount again. Not because I don’t want to talk about it. It’s just that I’ve spoken hundreds of times about it. And the story is so much out in the public domain. All I can say is that there’s nothing which is hidden from [the] public. There’s no hidden story here. Every single thing is in the public domain. When it started, why it started, where it reached. I mean, it’s so much there that it’s a waste of time to talk about it.

SS: I’d like to know how Jai Kisan Andolan fits into the broader vision that Swaraj Abhiyan has of a new politics.

YY: As I said, new politics has to have not just electoral politics in it. It has to have movements and constructive politics. And politics of idea. In all these three respects, the question of the farmer, the land question is the biggest question in India today. Land understood not merely as the land, the piece of land, but land as in the original economic sense of the land, which is to say all natural resources. That is the biggest political question, and farmers are the most neglected social group in our country. Neglected by governments. Neglected and despised by intellectuals. By social scientists and academics as well. There is a deep, deep apathy, if not hostility vis-a-vis farmers in the ruling establishment and among the academics as well.

So in terms of generating ideas, in terms of creating constructive work, and creating political movement, [the] farmers’ question is the biggest question. So we thought we must pick it up. Whether it is electorally viable or not, doesn’t matter. So it is a beginning of a very long-term engagement.

And for me particularly, I come from a village. As I said, twenty years ago, I wanted to go back to my village and be a farmer. That’s what I wanted some twenty-five years ago. So it is now, in terms of my life trajectory and life mission, it is invigorating to connect to that mission and to be able to articulate [it]. The trouble is that farmers cannot articulate farmers’ voice. You have to some middle class connect and some middle class articulation to be able to articulate farmers’ voice. And I happen to have that. And I thought that’s the best use of my time.

SS: [In] your article in The Hindu yesterday, you urged caution about celebrating too much the victory of the mahagathbandhan [in] Bihar. You say, “Much abused anti-Congressism is replaced by a vacuous anti-BJPism.” You describe this in this final line of the editorial as a “call to action.” I’d like to know what kind of action, specifically, you mean.

YY: That’s what Swaraj Abhiyan is all about: creating an alternative politics.

Anti-BJPism is dreadful. It is vacuous. It is poor politics. It is dangerous ideational activity. It is laziness. [The] BJP and Mr. Modi represent a very serious danger to the very idea of India, but that cannot be met by anti-BJPism. We have to understand why Modi came where he did. And we have to overcome that. Going back to the very same forces which are responsible for Modi’s ride in the first place is not a solution to that. You have to create an alternative. Alternative in terms of ideas, alternative in terms of political energy, alternative in terms of organizational protocols and principles. Which is what we want to do in Swaraj Abhiyan. So that’s the long-term solution.

And these short-term ways of trying to combat [the] BJP are suicide. So yeah, you read me right. I think this is suicidal politics. Not just poor, it’s suicidal politics of the Indian seculars. This is an admission of defeat, an admission of their incapacity that they are doing what they’re doing. This is certainly no way to respond to the very significant challenge posed by Mr. Modi. This complacent belief that somehow he will self-destruct and make way for the same old forces to come back is a lazy belief. It’s a dangerous belief. And even if this were to come about, it would do no good to India. That’s exactly what Swaraj Abhiyan is about.

SS: Thank you very much.

Praveen Rai on the challenges of polling in India and why he thinks seat predictions should be banned

Praveen Rai

Praveen Rai is the Academic Secretary at the Centre for the Study of Developing Societies (CSDS) in Delhi. Prior to working as Academic Secretary, he was a Project Manager at Lokniti, Programme for Comparative Democracy, a research programme of CSDS that specializes in election studies (full disclosure: CSDS is my primary research affiliate, and I am working with Lokniti on my research). While at Lokniti, he handled more than fifty election studies and opinion polls between 2005 and 2009.

His research interests include electoral politics, monitoring election opinion/exit polls and political participation of women in India. His seminal work “Electoral Participation of Women in India: Key Determinants and Barriers” was published as a special article in Economic and Political Weekly in 2011. He also co-authored a book, Measuring Voting Behaviour in India, with CSDS director Sanjay Kumar that was published by Sage in 2013. His articles have been published in Indian academic journals, edited books, newspapers and blogs. In much of his writing, he has been critical of the use of opinion polls as tools of political communication.

Last Wednesday, I sat down with Praveen in the CSDS lounge to talk about his experiences managing polls with Lokniti and his criticisms of media reporting on opinion polls. The transcript of the conversation below has been lightly edited for length and clarity.

On his involvement in public opinion research

Sam Solomon: Please tell me about about your background conducting public opinion research in India and how you got into this field.

Praveen Rai: I am basically an LLB. I have done my master’s in history from Delhi University and my law [degree] from Delhi University. I have worked in different private companies, small and big, looking after legal affairs. But somehow that job didn’t interest me, being a very routine thing. So I ha[d] been trying to make a career switch.

I just got a chance to come to Lokniti while I was looking for a job. I was just working here as a research assistant. But while working on that slowly I developed interest in opinion polling.

SS: You didn’t have the interest before though.

PR: No. In fact, I had been interested in politics and looking into election surveys on television and newspapers. That used to interest me. But I had never thought of getting into this area.

So I came here as a research assistant on a very temporary contract looking for a job, making a job change. And slowly I got into this. So I started right from the ground by doing the first time exit poll, which went wrong for me, as a field investigator. And then I did a few more fieldworks, and then slowly I got into questionnaire designing. So with a period of time I got into this field more by default in the sense that that was not my job. But I started learning , right from questionnaire designing to conducting fieldwork to managing large projects, and then data analysis and all that. So slowly I got into this. I started writing about it. I got an opportunity at Lokniti. And over a period of time I developed an interest in this.

So I think I worked almost four years where I was the project manager to Lokniti, and I think I conducted more than fifty or sixty opinion polls. That included election studies and studies in democracy. And different kinds of opinion polls. The Prime Minister’s survey, to find out what are the opinions that people had of Prime Minister Manmohan Singh after his first term. In fact, during his term in three years. So slowly I got into that but then I learned the whole thing and it became an interest area for me.

SS: Over what time period is that? When did you start getting into Lokniti and CSDS?

PR: I have a very long association with Lokniti. As I told you, the first one when I came here in 2002 as a research assistant. So I worked there for six months, and as I told you I did field investigation, I did field supervision, about how interviews are done, how student field investigators do. And after that I again went into the private sector where I was handling corporate communications. But there also I conducted some election studies for some political parties also. That was a learning stage for me.

In 2005, when Lokniti got into an informal contract with CNN-IBN for doing all the election studies and different opinion polls, that was the time I got the invitation from Lokniti. In October 2005. Then I came here. But by that time I was managing projects, I knew how to do field investigation, supervision, and all those things. And from 2005 to December 2009 when I was here, I got a chance to get into questionnaire designing, data analysis, and then I started writing. Now I’m all about opinion polling right from field investigation to putting the reports out.

SS: You have an office at CSDS. Are you still affiliated with Lokniti?

PR: No. In 2009, I got into my current assignment, that is like work as an academic secretary. So my role is a little different. To look about academically and liasoning with funders. The reason why I got here was this is a permanent job. In Lokniti, that was a contractual thing. Every year it used to be renewed. So at the moment I am not actively involved in opinion polls in this sense.

But still I write. If you see over the period of the last four or five years, it has changed in a sense. Now I don’t do much quantitative writings. I think everything is finished. All my writings and book chapters. And then there was a book also which I wrote with Sanjay [Kumar, Director of CSDS]. it’s called Measuring Voting Behavior in India.

Since last year I started tracking opinion polls. The way opinion polls and findings– in fact, now ten or fifteen agencies do [opinion polls]. So five or seven agencies say this is winning, and five or seven agencies are saying the other parties is winning. That confused me a lot. So then I got into writing about opinion polls. I tried to find out, I tried calling them up to find out what is their methodology, where was the fieldwork done, do you have the questionnaires. But the response from the different polling agencies has not been good.

SS: Were some better than others at responding?

PR: No, except for a few agencies like AC-Nielsen and all, which are market leaders in a sense. They are market research agencies who do it in a very, very proper methodological way. In fact, the quality of their surveys are, I would say, as good as Lokniti-CSDS. But apart from that, most of the agencies won’t even spell out who are the people who did the field investigation, what exactly was their sample, was it in residences or on streets. So these are things which are not coming out into the public domain.

Opinions polls are mostly on elections. Apart from elections, you get very few opinion polls. Eight to ten, you’ll get, to see in country which are on other issues. That is mostly done by Lokniti. The agricultural studies, the youth study. Apart from that, most of them are doing [surveys] during elections. Round after round — pre-polls, post-polls, exit polls.

SS: You’ve been involved with Lokniti-CSDS for almost fifteen years, since 2002. I’d like to know how measurement of public opinion has changed over that time, for CSDS and more generally in India.

PR: I told you that since things are not in the public domain, we don’t have a forum where the oldest people who were involved in public opinion polling get together and share those things. Nothing of that sort has developed in the last ten years. Though there have been a lot of people who have been trying to take initiative, I think it is not going. So at the moment I can just speak about CSDS surveys, Lokniti surveys. So I will just speak about the time when I joined and how we changed. And before that we have the history which you can find out from our book where me and Sanjay, we have written a very long chapter on how the surveys in Lokniti have changed.

During my period, in fact, the changes which happened in Lokniti surveys [were] at the stage of getting all the questionnaires here, cleaning it up, data punching, and then doing the analysis. From 2005 onwards, slowly we started decentralizing data entry and data cleaning. We developed it in different states. So now what’s happened is that it’s not centralized in a sense. All the questionnaires from India don’t come here and then we don’t have to put in huge teams with deadlines to do all the data cleaning and data punching for the analysis. So we decentralized it.

Apart from that, also since the number of sample increased — if you see our National Election Study, the National Election Study’s sample increased — so what we started doing was that we had four or five questionnaires. Questions on elections and politics, we had ten or fifteen questions which were in all the questionnaires. But in different questionnaires we started asking people on different issues, like security issues, economy, and all that. That way, in fact, we had more spread of getting opinion about politics, economics, price rise, and all these things. So I think these are the two major changes that have happened in the last ten years in the Lokniti surveys.

But the common thread which runs right from the first survey until now, and that we haven’t changed, that is we still do face-to-face interview of sampled voters in their residence. That is one common thing that is there in the last forty years of CSDS surveys.

SS: Going back to the 60s.

PR: Yes. Starting with the 60s.

SS: My research is looking at sampling error and whether certain populations are more or less likely to be sampled and participate in a survey. I’d like to know whether, in your experience working on Lokniti-CSDS polls, certain populations were more or less likely to be sampled in surveys, because of cultural factors or socioeconomic factors.

PR: In fact, the best part of the opinion polling which you do on elections is that in the the last ten years, all the voter lists are available online. So your universe of the study is– what happens is that you get the list of the complete universe. And we have been using the random stratified sampling system. So what has happened is that we have tested it in the 90s. We have been using those and once we get our data then there are certain indicators. We have matched the profile of the voters which we have done in our survey with the profile that exists in the state or at the all-India level. We have government data on the gender breakup, on religion, on caste and community. By using our random stratified sample, by the end of the 90s we could see that in fact we had been including a very, very representative sample. I think the sampling method in fact does not exclude anyone and is quite representative of the population.

SS: Even with the sampling being properly inclusive, did you find that certain populations had a higher likelihood of non-response and not participating in the survey?

PR: Well, I think not. We don’t have data for that. No, in fact, we have data for that and there is no difference in the non-response based on caste, community, or socioeconomic [level]. Non-response has been for many reasons, like people not living in that area, or migrating somewhere, or the day you reach for the interview they’re out of their village, they’re not in their residences. But there is no difference in the sense that the non-response of rich people is more or the non-response of poor people is less. It’s more or less uniform.

SS: What about for men or women? For religious minorities?

PR: For that we also don’t see any differentiation because what we do for the Lokniti surveys — you know how the network operates — we have a network of different universities and they’re mostly students who are doing their master’s or their MPhil. They are trained in how to do interviews, how to handle questionnaires. There we keep a gender equilibrium, in the sense that 50% of the investigators are men and 50% are women. Female investigators mostly go and do interviews with females because in villages or even if you live in religious communities, like Muslims, for a male to go and interview is a problem. So we have been using female investigators for the interview so we don’t have any problem of non-response from a particular section like Muslim women. Even in north India where you have the purdah system, where a male cannot go and do a face-to-face interview, but definitely the females goes to do [the] interview. So we have taken care of this. I don’t think there is any high non-response.

SS: In your article for Economic and Political Weekly, “Fallibility of Opinion Polls in India,” you write about four types of errors: coverage error, sampling error, measurement error, non-response error. Which of these four types of error do you think presents the most significant challenge for Indian researchers? If you have specific examples you can refer to in your time working for Lokniti or CSDS, that would be most helpful.

PR: When I am talking about all these errors, then definitely it is there in most of the surveys. But you have to see how high it is. Even if you see a non-response error — if it goes to three or four or five percent — it doesn’t make much difference in the sample, because the non-response of people… It’s not like the non-response of male respondents are more or of female respondents are less. I think non-response error does not create much problem. I am talking specifically for the Lokniti surveys. After that, I’ll talk about the marketing or opinion polls. In fact, all these errors are within the limits and they cancel each other out and they don’t have [an] effect on the total sample.

But now as far as the market research opinion polls which are coming to the public domain in the last four or five years, because of which I wrote that article, they are not coming out with the proper methodology. They are not telling us exactly who are the people who did [the survey]. I think the main reason they are getting it wrong could be, one, is the sampling method. Because a face-to-face interview on a proper random sample is a very, very costly proposition in India with the kind of length and breadth of the country, and very interior areas. So I think one of the major areas where most of the polls are going wrong is on sampling.

Even if they get their samples correct and their field investigation correct, the second error could be the measurement error because getting the right vote share… [between] the time the vote share of a survey which you do and the election results are declared, the vote shares are within the limit of one or two percent. It doesn’t make much difference when you translate it into seats.

But the kind of results which we are getting — as I told you, five saying Aam Adami is going to win and five saying BJP is going to win in [the 2015] Delhi assembly elections. And it was a clean sweep. Just three seats for BJP. So there in fact it will be very, very hard to tell that because of these errors the surveys were wrong because with huge margins of the vote share, no survey can get it wrong. There comes in what I talk about, the vested interest of people in the media, their manipulating seats to create a bandwagon effect before the elections.

On the use of polls as tools of political communication

SS: In the article, you used pretty strong language, talking about how election surveys have been reduced to a “media gimmick.” If that wasn’t the situation in the past, why do you think that has happened and what can be done to change that?

PR: As far as the bandwagon effect is concerned, I think it was way back, around twelve or fourteen years back, when a series of surveys were done by CSDS and they used to come out with different report cards for MLAs [members of legislative assembly] and that was there in all the Hindi and English newspapers. A lot of people read it — we cannot say exactly what percent — but most of the people were aware of this survey. In fact, it came out daily for one month. So in the last round of surveys when they did all these MLA report cards, we had put a question, how many people have read about it? We were very surprised that just seven or eight percent of people had read about all those MLA report cards. And it was coming out in all the Hindi newspapers and English newspapers. But only six or seven percent were aware of it, the MLA report cards in the survey. And those who had read about it or heard about those polls, we asked them another question, what effect did those readings have on voting for a particular party? Did it change your original decision? They said no. Whatever decision they had taken was taken by us and the poll had no effect. The findings of our poll had no effect.

This is way back, I think, 2002. In the last eight to ten years — in fact, I am talking about pre-poll surveys just before the election — when you have a series of elections, say, OK, BJP is going to win, Aam Adami Party is going to win. Even Aam Adami Party, they also did a survey for the first [2013 Delhi assembly] election where they had spoken about getting something more than fifty seats. I think they got something like twenty-three or twenty-four seats. There also they had used survey findings done and Yogendra Yadav had joined the party at that time. They were saying that they were going to get more than fifty seats. And on the other hand, Congress was going into the elections. Because they were the incumbent party, they were saying they were going to get a majority. [The] BJP did their own polls and also said, “We are going to get a majority.” Now these are the three main parties and all three are doing that. But when the election results came out, the BJP had thirty-three seats, [AAP had] twenty-four seats, and Congress was completely wiped out [Editor’s note: In the 2013 Delhi legislative assembly elections, the BJP won 31 seats, AAP won 28 seats, and Congress won 8 seats, leading to an AAP-led government with the support of Congress].

So that means all the surveys that they did and all the findings which [were] put out, those were just to solicit votes. That was to create a bandwagon effect, that, “OK, now we are winning.” In fact, when all these reports were out, they were putting out banners and all those things. They were doing a lot of publicity.

SS: But based on that and what you just said, it seems like there isn’t a bandwagon effect then.

PR: No, this is something conjectural until the time we do a study to find out that people are aware of those findings and based on what they’re finding, “OK, Aam Adami Party is going to win and then all the traditional BJP supporters are voting for that.” So this is just conjectural because a study has to be done and definitely none of the polling agencies are going to do that. So I think the only one who can get into this and do a proper study to find out whether we have a bandwagon effect or not is the Election Commission of India. So that is what I have been telling them. Just ban seat predictions. Seat predictions. I’m not talking about banning surveys. Surveys are very, very important. Election studies are important.

The reason why I wrote in EPW is the fact that the purpose of doing election surveys from a instituit . Political parties have also been doing that. But the purpose is now to find out what is the vote share they are going to get. Will this candidate do better? So they are using polls to find out how they are placed, how they can do that, what kind of candidate should do that. So the purpose was that.

So I think until that stage, until the stage where the Vajpayee government was there, before that, in fact, they had a minister called Pramod Mahajan, the first to come out openly and admit, “OK, we also do opinion polls but the purpose of doing our polls is to find out how we are placed in a particular contest, what kind of candidates we have to put in.” He was the first politician to admit that they do their opinion polls.

Until that stage, that’s not a problem. But doing an opinion poll, fudging data, putting out reports to say, “We are going to win,” that is not a purpose of opinion polls. That is not a purpose of election studies. And that is basically telling lies. The purpose of opinion polls in India were to find out the mood of the voters, like what kind of assessment we have of the incumbent government, what are the issues they think are important, and whether they have formed voting decisions until now or whether they are going to make voting decisions once the campaign ends. That was how opinion polls developed since the 1990s.

But I think over the last five years now, whatever they are doing, the most important thing is to just tell the people who is going to win. So now they are using it as a political communication. That is giving an extremely bad name to the polling agencies. And though we [CSDS] do a different kind of work — our purpose is to do election studies, do evidence-based reporting of how a party fared, do post-mortem analysis, and [make] data available for researchers who come in — at the moment everybody is clubbed in. Lokniti did seat projections for a couple of years and we dropped them because it was not scientific and it was not working well. But still people feel that, “OK, CSDS is also saying that.” So people have now started looking at opinion polls, they think it’s all biased and being done by interests to further that.

On why seat forecasting should be banned

SS: You make an argument that seat forecasting should be banned. You think the reason is not just because there’s a lack of transparency with how they come up with the seat forecasts, it’s [also] that the parties are using these seat forecasts to generate some kind of bandwagon effect that will help the party. And it’s not being done using scientific examples.

I spoke to Rajeeva Karandikar, who used to do the seat forecasts for CSDS. He’s pretty open about how he does the seat forecasts. He doesn’t give you all the specifics, but he kind of walks you through the general principles. He wrote a big article for The Hindu Centre [on this].

If more seat forecasts were done in a way that a very detailed methodology statement was included with the forecast, do you think that would be okay? Or do you think it will still be a form of political communication that political parties would use to manipulate public opinion?

PR: What has happened with opinion polls is that seat forecasting has taken the center stage. If we get a report out in a newspaper on an opinion poll on elections, the only figure that people want to see is who is winning, how many seats. As I told you, whether it has a bandwagon effect or not, most political parties think that it is a part of a communication campaign. So they have started using this for doing it.

I have [said] that the Election Commission has taken a very, very harsh view in the last five years. Nothing much has been done. A time had come when they were talking about banning election surveys, in fact, before the elections. Now they have a time period. Once the polling process starts until it ends, that is a time you can do your survey but you not come out with any reports. But just before the election process starts, you can do as many polls as you want to do, as many seat projections. Instead of banning election surveys, what I have argued is that seat prediction should be banned or else the other alternative can be what happens in Japan.

In Japan, seat projections have been banned. What had happened is that earlier they used to do seat predictions but when it went wrong, people went to the courts, the political parties went to the courts and filed a case, and people who did wrong forecasting were penalized heavily. The courts there said that if you get your predictions wrong, you have to pay for that. So seat projection doesn’t happen. At the best, they do a lot of election surveys and opinion polls, but they talk like, “BJP is leading and Aam Adami Party is second or third.” They talk like this, but they don’t assign any specific figures about, “This is the number of seats they’re going to win.”

So I think that would also be one way to deter media. In fact, if a political party does something and then does it through media, you know, an India Today poll… What happens is you’re masking your poll. Ultimately [the] media has to be made accountable, because television is  a very, very important way of reaching the homes. And elections, politics in India, it creates more passion as compared with other countries. (laughs) People are political in that sense that they are interested. And when a media house puts out something it’s like putting a stamp on that survey. So either you put a huge penalty that if your prediction goes wrong — you cannot just say that BJP is going to win a two-thirds majority and when the results come out you find that the opponent side won [a] two-thirds majority. You can’t be so wrong. Either you impose a huge penalty on the polling agency and the media house which is showing it so that they stop doing all these gimmicks and all these bandwagon effects. Or the second and easier method could be that we just tell them which party’s vote share they can talk about, which party is leading, which is winning, but don’t give out those numbers.

SS: But don’t you think that the vote share would then become… It seems to me that even if you banned seat projections, the political parties still might try to gin up bandwagon effects by saying vote share: “According to this CSDS poll, we have 45 and these guys have 41.” I feel like vote share could also be a form of political communication, even if the seat share were banned.

PR: No, I agree with that. Once you ban the seat projection, people can come out with the vote share and they can talk about it, “This party is going to win the election and get a huge majority.” All those kinds of things will happen. So I think there has to be some body, the Electoral Commission of India has to set up one unit whose work would be to see what kind of media reports are coming out. And if somebody says that, “NDA is going to get a majority, and BJP will  get a majority but they will be the largest party,” but after the election you find the BJP gets a majority on their own. That does not amount to much tinkering of data or misuse of election surveys.

But you can’t do a projection that NDA is going to get a two-thirds majority, and when the results are out, you find that the opposite party has got that. So when they do that, I think the Election Commission has the right to get all the data and find out whether the survey or fieldwork has gone wrong, the vote share they have computed has gone wrong but the survey is correct, the vote shares are correct and they have manipulated the data, and based on that some punishment needs to be enforced if they want to restore the faith in election studies back. Because it has taken a very, very bad beating since the last 2004 national elections, they are getting it wrong.

And even if you see this election [2014 Lok Sabha elections], because I did not do a follow-up article on this, here also in fact nobody spoke about a BJP majority. They all said NDA will get a majority. But nobody could in fact that BJP is getting a majority. So that means the vote share which you got in your survey which you are showing, there’s some problem. And the problem can be of fieldwork, of data collection, wrong sampling. At the moment, we are just grappling in thin air. Whether the fieldwork is wrong or the fieldwork is right and people are manipulating at this stage, the media are manipulating. So I think the responsibility needs to be fixed, either with the polling agencies or the media houses.

SS: And do you think the responsibility for changing this, the culture around polling in India, that falls on the Election Commission of India?

PR: That’s the body which puts a stop to election studies because once elections are declared, most of the powers are with the Election Commission of India. The last one which we had, whatever orders they had were [carried] out. As I told you, the day the election notification takes place, from that day until the day the last vote is cast, that is a period they have said you cannot come out with a report of opinion poll. Even that is ambiguous in a sense. You can do your survey but you cannot come out with those reports. And I think that is being followed by most of the polling agencies and media houses.

So I think the Election Commission of India should get a study done to find out the bandwagon effect, so that it can tell the political parties, media houses, and polling agencies, “Your reports [don’t] make a difference.” Though I am just being conjectural. Maybe it makes an impact on the voters and some voters are changing because of the election surveys. And based on that the responsibility needs to be fixed so we have fairness and transparency in opinion polling to restore its image. Because at the moment it has taken a very, very bad beating.

SS: Have you heard about the Indian Polling Council initiative that’s supposed to be launched in the next month or so? There was an article in The Hindu about it.

PR: I am not aware of it.

SS: In the article, some pollsters — in the article, Yashwant Deshmukh and CVoter are the ones that are quoted — CVoter and five other polling agencies are thinking of coming together and starting their own group to come up with some standard guidelines for disclosure of methodology and transparency in terms of reporting on their polls. The idea is that if you belong to the Indian Polling Council, you have to report things like sample size, sampling methodology, etc.

PR: If this is happening, I think it is a very good initiative in a sense because what I feel is that instead of the Election Commission of India drawing guidelines for do’s and don’t’s for polling agencies, if they all come together and form their own thing where they share their things, and they can come out with reports, I think it would go a long way of restoring the credibility of opinion polls in India. So it’s a good initiative and if it happens, it will definitely help.

On the challenges of polling in India

SS: In your own experience — you’ve worked on fieldwork, you’ve worked on questionnaire management, you’ve worked on all the stages of opinion polls — what are the biggest challenges that you found in accurately measuring public opinion in India?

PR: As far as challenges are concerned, there are a lot of challenges in the sense that this whole face-to-face interview is a very, very hard task. There are villages where you have to walk for kilometers on the hills to reach them. Lokniti does it, but the cost involved of doing a face-to-face interview is very, very huge in this country. That is one of the biggest challenges, and I feel for most polling agencies they don’t get enough funds to do this.

Secondly, doing a telephonic interview in India is still not possible because the mobile phones’ reach has increased but still most people don’t have the telephones. So making a shift from face-to-face, compared with US and other European countries where face-to-face interviews are very, very less and most people do telephonic interviews and now using different platforms [the] Internet. So I think this presents a huge challenge.

Apart from that, as you know we have a very, very diverse country. Different languages and all that. So if you do a survey at an all-India level, like Lokniti, we translate a lot of questionnaires into state languages. For most of the states, we are doing that. But for other polling agencies, we are not sure whether they are employing, like if you want to do a survey in Karnataka, you need to have a researcher from Karnataka and you need to have the survey in the local languages. That is one of the big challenges and this is also [a] huge cost. So whether these polling agencies are doing it or not, that I am not very sure about it. At least Lokniti does it, but still it’s a major challenge because even if you see a language, even in a state, there are so many dialects, that translating the questionnaire into dialects and getting the investigators with those dialects to do the interview, that is a huge challenge.

In fact, these are the most important challenges of doing an opinion poll in India.

SS: Anything else you’d like to add?

PR: I’ll add an example of the kind of the challenges you face. As I told you, from 2005 until 2008, for three years Lokniti-CSDS, we did seat predictions. And that was mostly for assembly elections. So I think we had nine assembly elections during that time. We got seven right, the [election results] were exactly within the range. But we got one completely wrong and one partially wrong.

The election had taken place in Punjab. There also the fieldwork and everything went off well. We were using exit poll data. With the exit poll, what happens is that the moment the polling ends, say the polling ends at 5 o’clock, the television program has to go live exactly at 5 o’clock and by 5:30 you have to tell exactly which party is winning. So that means that you have to close your fieldwork by say, around 3 PM, because then you have to transmit data and all those things, and analysis has to be done, seat prediction has to be done. You get just two hours. So with Punjab we did that. It stopped at 3 o’clock and we got our data. We did the seat prediction and all.

But what happened is that the election was extended from 5 until 9 PM because there were heavy rainfalls that took place. Here all the predictions and television programs were on since 5 o’clock. We collected our data until 3 o’clock. And then the rain stopped at around 5 or 6.

Congress and the Akali Dal, they were in a very, very strong contest, just a margin of two or three percent. In fact, the Akalis were winning. So what happened is the rain stopped. A lot of Akali voters, they put their voters in their buses and they went and voted at 9 o’clock. Now for the exit poll we closed our field investigation at 3 o’clock. So our data was loaded in favor of the Congress. So we mentioned that Congress was going to win. But actually it was Akali Dal which won.

So we predicted something, it went wrong. We did a fair investigation to find out how come we got it wrong. Even for exit polls we do purposive sampling to make it representative. So we missed the whole, the Akali Dal voters go when the rain stopped at 4 o’clock. They went to the voting. And the voting was extended until 8 o’clock in the evening. That also, people didn’t inform us. So a huge section of people in a lot of places voted at that time. It’s a very strange thing. With all the Akali voters, what they do is that they get buses and tractors and all those kinds of things, where they go together and vote. So this is one challenge which nobody foresaw. And we got it wrong.

SS: What year was this?

PR: You have to check. This was Punjab elections. I think, Punjab elections 2007 or 2008. You will get that if you do a search [editor’s note: Punjab Vidhan Sabha elections were held on 13 February 2007].

SS: That was the one seat projection that you got way wrong, and there was another one which you said you got partially wrong.

PR: The other one where we couldn’t predict, I need to check. Was it Tamil Nadu? No, Tamil Nadu, we got it right. The other one I don’t remember. But two we got wrong. In the third, also, in fact with Tamil Nadu, what happened, as I told you, is that you have some error, the coverage error. There was some errors. But what happened is that the errors for non-response and the errors for non-coverage, both cancelled each other out. So we got the vote share correct. But if you see the survey, and if you see the initial data that was coming in, there was some problem in fieldwork. We couldn’t get the fieldwork done properly. It was a problem with investigators, with fieldwork, with supervision. There were logistical problems. We did not get the vote share. But one mistake, cancelled by the other mistake, and we got the vote share correct and we made the right prediction (laughs).

SS: And what was this for?

PR: This happened in Tamil Nadu, this was also between 2005 and 2009 [editor’s note: Tamil Nadu Vidhan Sabha elections were held on 8 May 2006].

SS: For Vidhan Sabha?

PR: Yeah, for Vidhan Sabha. Our prediction was perfect. But if we see our data, and what had come, and what were the errors, as we do a review of all our surveys once we do [them], and mostly during seat projections.

So two wrong, and third one we got it right by default. So we decided that doing seat predictions and vote shares, it’s a very, very tough situation in India. And so we stopped. I think 2008 we stopped. 2009 Lok Sabha elections, we did not make any prediction.

And then there are some interesting cases. In one of the Karnataka assembly elections — not the last one, the one before that — I very well remember that Congress had got 32% of the vote and [the] BJP got 30% of [the] vote, but [the] BJP got the majority of seats and they formed the government [editor’s note: In Karnataka’s Vidhan Sabha elections held over two phases in April 2004, Congress won 35% of the vote share and 65 seats, while the BJP won 28% of vote share and 79 seats].

SS: Because their vote share were spread out in a way that was more advantageous.

PR: Yeah. The Congress vote share was completely spread out and they were concentrated in different assembly constituencies.

I think that was also one phenomenon that we first saw in Karnataka, because before they we hadn’t seen in any elections, having more votes but getting less seats. Karnataka gave us that lesson.

And apart from that, if you see the case of Bihar, so [in] Bihar they were completely new political coalition formations. If you see Bihar, first of all, there were so many parties, new coalitions. When you have new coalition partners, you don’t have any record of the previous vote shares. You have nothing. And how that coalition works on the ground, the vote transfers and all that. That is very, very difficult to ascertain through an opinion poll. Until the time that you don’t have some quantitative study done to substantiate your quantitative thing, you can never get a seat prediction.

Where there is just two main parties in contest like it happens in Tamil Nadu, a couple of states, it happens in Rajasthan where [the] BJP and Congress is there, it happens in Uttarakhand, there once you get a vote share [which is] correct, like 45% for Congress and 38% for [the] BJP, you can easily do a projection based on that. And mostly you get it right.

SS: But it’s more difficult when there’s more parties, like Bihar or UP [Uttar Pradesh].

PR: These are the challenges, and how the model is going to take into account these things, for that, we say that until the time polling agencies don’t come on a single platform, they don’t share their prediction models, and if they’re not going to improve upon that, they are going to be more and more wrong.

SS: And for the Karnataka poll where Congress got a larger vote share, but BJP got a larger seat share, did CSDS do a seat projection for that election?

PR: I think we did that and maybe that was one survey we got wrong.

SS: You had the vote share right but the seat projection didn’t predict that [the] BJP would end up winning more seats with less votes. OK, I’ll check that.

PR: That you can check. I think we have it on our record though I don’t have it because we did a complete check of our seat predictions.

It started in 2005, exactly with the Bihar elections. We got it right.

SS: The seat projections.

PR: The seat projections. For all the political formations, we got it correct. We got the vote shares correct. That was the first seat prediction we had done that was really celebrated for getting it right. And then it went on for a couple of years. It was 2008 or something, I think, when we got two or three wrong.

SS: This was the year of Punjab and Karnataka.

PR: Yeah. Punjab and Karnataka.

SS: Wouldn’t you say, if you got seven out of nine right, that’s a pretty good record, right? Maybe you think statistically because of some of the errors, because some of the challenges of polling, you’re going to get a few wrong, but the overall track record of seven out of nine, or whatever it is, that seems pretty good. I guess it’s a matter of focus on the fact that we got seven out of nine right, or we got two out of nine wrong.

PR: No, but see, if you see the pre-poll surveys which we have done — at the moment we are just doing post-poll surveys — you can see the data that we have got in all the pre-poll surveys. You will not find a single poll where you find the actual vote share in that election — if Congress got 40%, then we have got 30% — it was never so wide. Our vote share would come to around 38%, or maybe 1% more. It has always been in the range of 2%. Plus or minus 2%. In fact, the vote share has never swung more than that.

SS: Is this for post[-polls] or pre[-polls]?

PR: Pre-polls. Just before the elections. In fact, we have got it within the range of 1-2%. So you can just imagine that we have got the vote share 95% of times right. So that means that the sampling method which we are doing, the field investigation we are doing, our systems are perfect in the sense that we get it right.

But even with those right vote shares, when you do a seat prediction — as I told you, we got six or seven right out of nine. One or two we got it right by default. By default, one error cancelling the other error, as I told you. That means that that system is still not completely developed. Either you do a lot of research and development, when you can put in all these variables, like concentration of votes and all these kinds of things and different vagaries different coalitions. Either statistically over a period of time, and if somebody does that, but at the moment I don’t see anyone.

Because even if you see a new polling agency which just came up, its name is Chanakya, Chanakya got it right in 2014. You will not believe. They got the seats right for BJP, okay, and all the polling agencies missed it. Everybody missed it. And I think everybody laughed also again. BJP will get a majority. Everybody was laughing. “Who is this Chanakya?” and all that. And then Chanakya got such a big name. And then again have started floundering. Delhi, they have got it completely– Bihar, they have got it completely opposite. Now people are laughing. They are saying that Chanakya by mistake, instead of putting UPA, they are putting NDA with some grammatical mistakes [editor’s note: Today’s Chanakya exit polls showed an NDA landslide. Following the announcement of election results, they said their prediction was off because a computer error had mixed up the names of the alliances]. (laughs)

SS: Yeah. I saw that. Interesting excuse, yeah.

PR: Prannoy Roy is one of the leading psephologists in this country. He was the one who started doing opinion polls through his channel, and all the swing votes, and all that, he made it so popular. So I think Yogendra Yadav and Prannoy Roy, not only are they the best political analysts but they are the leading psephologists. And the way Prannoy Roy got it wrong this time, he had to render an apology. So now this really calls for a question. In fact, people should ask. The media should go and ask him, “What kind of a survey [did] you [do]? Your survey was wrong.”

Because there are a lot of stories going around, and they were trying to show this, push this, that should [the] BJP comes in power, they have all these taxes left and these debts will be paid. They are getting pressures from above.

SS: It sounds like CSDS has a pretty solid track record. You’ve gotten a few seat projections wrong. And maybe that’s because of the methodology, because of the focus on post-polls rather than pre-polls. It seems like when you do pre-polls, you try to do them as close as possible to the election as you can.

PR: The whole purpose of doing an opinion poll for us is it’s a study. In fact, we call it election studies. Our purpose is that. The reason why we started going to the media is that there is a huge cost involved in doing this. Until the 80s, the government agencies provided funds, but after [the] 80s, there was no funds available for conducting election studies. It is still not a discipline in India. It is still a sub-discipline.

So when we had the tie-ups with the media, that helped us in doing our surveys and giving them the required data like they are just interested in the vote share, the popularity ratings of the leaders, those kinds of questions. But apart from that, using those funds, we have been doing a huge, a very, very in-depth study of that election. So the data is with us. So our purpose was that. And when we got into seat prediction, at that time [the] media said, “Since you all are experts in election studies, why don’t you get into this?” And we also went into that to see how can we refine, and the model which Rajeeva Karandikar, I think, is still using is the one that was developed at that time. Then we started doing the seat predictions. But once they started going wrong, we said we don’t want to do it. Because our purpose is not that. Our purpose is to have a set of data on each and every election that happens in the country.

SS: And to understand why the results are happening and not necessarily just predict them beforehand.

PR: Yes.

SS: Thank you very much.

Yashwant Deshmukh on the challenges of polling in India and the forthcoming Indian Polling Council

Yashwant Deshmukh is the Founder Editor of CVoter, and a Communications Professional with the working experience of Journalist, Pollster, Evaluation Expert, International Observer and TV News Anchor rolled into one. He founded Team CVoter in 1993, when he was still studying in IIMC. After receiving the UNI award for best research dissertation and for topping the 1993 batch across all streams, his company CVoter was hired by the premier news agency UNI to take care of on-line real time election analysis. Team CVoter continued to grow, and is now one of the largest media and stakeholder research agencies in Asia with expertise in Public Opinion Research & Election Studies. Today more than 120 team members work for CVoter across their 24×7 offices in Washington DC, Dubai and New Delhi.

CVoter is also one of my research affiliates for this project. I have been regularly visiting their Delhi office (which is technically in Noida) to learn about how they do their research. I visited Patna to observe their exit polls during the third phase of the Bihar elections.

Now based in Dubai, Yashwant was in Delhi last week for the Bihar election results. Last Friday, I met with him at Janpath Hotel near Connaught Place to talk about his experience doing public opinion research in India, the challenges he has encountered, and a forthcoming association for Indian pollsters called the Indian Polling Council. The transcript of the conversation below has been lightly edited for length and clarity.

On polling in India

Sam Solomon: Please tell me about your background conducting public opinion research in India. How did you get into this field?

Yashwant Deshmukh: I am a major in journalism, actually. I did my post-graduation in journalism from IIMC. I am a journalist. That is my primary identity and my training.

But yes, I have always been interested in elections, ever since I was a kid. Had a personal history of roughing up with [the] Emergency. I wrote a piece on Huffington Post about my experiences during the Emergency. You might like to pull it out and read it once so that you can get the context of how as a kid I observed the Emergency and I got into the ‘77 elections which was a watershed election in Indian electoral history. Elections were always like something that were good for the people, good for the country, something which und[id] the wrong and [did] the right thing. So that’s the kind of personal background that I was raised on.

And then when I became a journalist, I was — I did my research dissertation on elections only, in IIMC — and then I wanted to join some polling organization as a political analyst. So I didn’t get a job there. I got a job in the campus placement in the newspapers and the magazines. But I didn’t get a job with the polling companies. So when I didn’t get a job with the polling companies, I had to start on my own. (laughs)

SS: What polling companies were around at that time?

YD: At that point in time, there were no multinationals around. There were no multinationals around.

The biggest company at that point in time was ORG and Marg — they later got merged as ORG-Marg, later years. But yes, they were the people who did the fieldwork for the initial polls of Prannoy Roy, which were the path-breaking polls in India, in ‘80 and ‘84 elections, to be precise. ‘84 was the time when psephology got a wider acceptance of political polling, in a way, when Prannoy said that Congress was going to end up with more than 400 seats. He got it right. So they were the people who did the fieldwork. Prannoy and Ashok Lahiri, they analyzed those elections. That was their job. They designed the poll and they analyzed the poll. Prior to that, in India the polling was only limited to what Eric de Costa was doing in IIPO [Indian Institute of Public Opinion], and there was no history to political polling as such.

I think that when I was graduating in the early 90s, that was a very critical and interesting time in Indian political life otherwise. It was the peak of what we call the Ayodhya controversy or movement. The BJP was emerging. The Congress was having problems. In [the] ‘91 elections, unfortunately, Rajiv Gandhi got assassinated. It was a very transitional time when PV Narasimha Rao became the Prime Minister. It was the time when [the] economy was opened up. So altogether [the] early 90s were politically speaking very charged, after [the] ‘89 elections, you know. V.P. Singh, Mandal Commission, Ayodhya, opening up the economy. I think there was too much to consume. It was a thorough transition and transformation that was going on.

And it was a time when I was graduating as well and being a journalist, it was interesting for me to see and map the public opinion on different things. So that’s where I started. I floated CVoter when I was studying at IMC in 1993. I got lucky because I topped the ‘93 batch at IMC, and I was awarded a UNI award for my research presentation on elections. There are two primary news agencies in India, PTI and UNI. And UNI the next year actually offered me to analyze the elections for them as the live counting was done. So that was our first major break. Probably my research dissertation and getting an award on that luckily helped me to get into our first assignment.

And after that, because of UNI, the fIeld was going to more than seven hundred newspapers across the world. Slowly and slowly, we started working for other media houses. The Week was the first one to give us a commission on an opinion poll.

SS: When you say “we,” you’re talking about CVoter, right?

YD: Yeah. So that’s where it started. And one after the other, it kept on happening and it was happening at a very frantic pace because we had ‘96 parliamentary elections, then ‘98 parliamentary elections, then ‘99. So within the gap of three years we had three parliamentary elections in India.

And it was also the same time when satellite TV opened up in India. The first 24/7 news channel opened up in India; that was Zee News. Prior to that, the news operations were limited only to another satellite network called Jain News. Incidentally, Professor Yogendra Yadav also did his first election on TV with Jain News. That was the only private satellite network which was doing news in India at that point in time. We are talking about 1996. Then, Zee was the first 24/7 news network. Zee hired me to do the polls. After that, Aaj Tak came in, then Star, and so on.

Basically, being a journalist, being a broadcast analyst, being a TV guy and a media guy, it kind of helped me to be on the screen because I could not just analyze but also interpret the data. It kind of helped me stabilize CVoter in terms of the operations and other things.

This had been going on, then after that we went beyond political research to socioeconomic research. We entered into disaster mitigation research. Conflict resolution research. That’s where the majority of our work right now is going on. Into the international work as well.

I was always grateful to Robert Worcestor, the founder of MORI, because he allowed me way back in the ‘90s to observe the MORI exit polls in the UK. So I learned that part from there and I applied that as much as possible in India. If I have to pick my godfather– I guess Bob has been to many other pollsters across the globe. I’m happy that I learned from him.

It was also the time when the multinationals started coming into India. AC-Nielsen came in. TNS came in. Ipsos came in. Gallup came in. They were following their clients in India, because the economy was opening up. And it was also a transitional time when the bigger Indian companies were being taken over by the major groups. So all of a sudden we realized that now we are the only Indian company working as such in the private sector with that kind of footprint and operation, volume and scale.

We entered the US five years back. We did the last presidential [election] in the US. Rated very well in our very first operation over there. We did South African elections, bigtime. We did the backward integration for the UK elections as well but we did not take the CVoter brand over there. We were working for some other group over there. Hopefully in the next election, we will enter the UK market as well.

SS: During all this time that you’ve working in polling, how has the polling industry in India changed?

YD: Difficult question. So much has changed and still it feels like nothing has changed.

What has changed is the technology of it. Technology has changed a lot.

We have moved the majority of our operation from face-to-face to CATI [computer-assisted telephone interviewing] now because of the simple fact that India is right now the biggest mobile market in the world. Even though 85% mobile penetration is there now. Actually, 85% is of the total population. If you talk about the adult population, it is at saturation, more than 100%. Statistically speaking. We know that about 20% of Indians don’t really have mobile phone. But yes. It’s huge. And because it’s calling party pays, we started CATI operations seven years back. And now it’s getting standardized. I believe that’s the future. That’s why we invested in that.

Online research in India is still far out because only 15% of Indians technically speaking have the Internet facility, but only 5% of them are actually [the Internet] for social media or other purposes like that. It is just that the volumes are too big. It’s not representative as such. It will still take a lot of time to come. CATI is getting there. So technologically speaking, there ha[s] been a lot of improvement.

From the quality [perspective], unfortunately I actually see a downswing because this place was earlier being observed by a handful of key players. They were open to skepticism and criticism. But all of a sudden with the new electronic media and everything, we see many players who are opaque and the methodologies are not being discussed [in the] way they should have been discussed.

The polling industry as such is getting to a stage where less interaction and less education of the media and the media consumers at large is proving to be detrimental for the health of the research industry as such. The concept of polling has arrived. Many people work very hard for that: Eric de Costa in the initial years, Prannoy [Roy], Dhorab [Sopariwala]. They all worked really hard. Professor Yogendra Yadav. He did his part very well. But it’s trivialized now in the media circuit. The media and journalists at large don’t understand. They don’t understand what to expect from the polls, what are the limitations of the polls. And that is why their expectations from the polls is wrong. And that is where it gets trivialized. And when it gets trivialized at the editorial level, it gets automatically trivialized at the readership level. So that is a big challenge that needs to be [addressed].

But technologically speaking, a lot of improvement has happened. And from that perspective, it is getting better, I would say. It is getting better with time. It is getting better with each and every assignment.

But the expectation from the research industry is wrong. The limitations, the plus points, they need to be more discussed and understood more properly.

SS: You said that it feels like everything has changed and yet not much has change. What do you feel hasn’t changed about the polling industry since you got into it?

YD: What hasn’t changed is the pathetic understanding of the polls by the media. What hasn’t changed is the absolute lack of data awareness among the journalists in this country. What hasn’t changed is the knack of sensationalism in the media scene while they are reporting the polls.

What hasn’t changed is the lack of continuity of the data gathering and analytics, the serious component of the research of the changing trends on on political sectors and indicators. What hasn’t changed is [the] absolute absence of philanthropic funds which should go into the research industry. In the West, right from the start and even today, almost 90% of the socioeconomic research is being funded by philanthropic funds. In India, it is a big zero. So the only funding that the polling actually gets in India is either government or the media. And they have their own limitations on doing trackers and serious socioeconomic research. So that hasn’t changed. There was no funding from [philanthropic funds] back then, there is no funding in it today.

So what hasn’t changed is the lack of serious polling on socioeconomic issues. What hasn’t changed is the lack of continuous polling on serious socioeconomic issues. These are the things which have not changed.

SS: When you talk about the lack of quality coverage and awareness of how to analyze data at the media level, could you talk about the specifically with regards to Bihar? We were talking earlier about the headline today for the exit polls, how it just shows the midpoint, it doesn’t show the ranges [of seat projections]. How do you think the media coverage for Bihar has been as compared with other elections?

YD: Basically, it underline[s] all the things which I just mentioned. If the understanding of the polls and the understanding of the research would have been better, it would have been easier for the media to say that these elections are close to call. If something is close to call, why [does] it ha[ve] to be a matter of ego than one needs to take a call? When you take a call that it is close to call, that itself is a call that it is close to call.

The big problem in these Bihar elections has been as usual the media’s inability to accept that this is close to call. And them forcing the pollsters to take a call. Because that’s what we are getting paid for. All the pollsters are getting paid to get a call. But I think the change is that now we are also forcing in the ranges which are overlapping. We are forcing them to say personally, even if it has to be on a personal level like me standing in front of the camera, saying, “Listen. This is close.” Statistically speaking, it’s difficult to call. My ego is not getting bruised or hurt by saying that I am unable to call this. No big deal.

In fact, more than Bihar, I would like you to read the one-year report card of the Modi government programme. It was a brilliant programme on India TV and Times Now where it was a Mood of the Nation poll. Probably the first time where I requested [of] them, “Please don’t force me to give you the seat share and vote share.” Because that’s not really the thing you should be analyzing in a weekly report card.

You are aware that we do the weekly tracker, which is the only such vehicle in India. Nobody does that. It was nice to see week-on-week Modi’s popularity going up or going down and analyzing that. It was fascinating to see that right after than 9 lakh suit controversy Modi’s popularity went down. As they say, pollsters are the chroniclers of history. It was important to see the last fifty-two weeks on different scales of popularity, the satisfaction, the issues, which way the country is going. If I would have given the vote and seat share projection, then nobody would have talked about those issues. “Ah! If the election are held today, Modi would have faced problems or Modi would be flipping more [seats]…” That’s trivializing.

So the big change is that I requested them and somehow I got lucky. Both my editors, Mr. Sharma and Arnab, they actually saw the merit in that thing. And they did a brilliant program which was [a] first in Indian television, that you are having a political program on a one-year report card without showing the vote and seat projection. You never have any state of the nation, mood of the nation [program] without vote and seat [projections] in this country. So that is the change which I am trying to push. I am lucky that I got a couple of good friends in media who are trying to see the merit in it. I hope that it continues.

But somewhere we have to put the foot down, that this is what the limitation is. 90% of the research material that goes into serious socioeconomic research and results go unreported in the media. The only 10% of the political aspect of it gets 100% out-of-proportion coverage. That needs to be changed.

And that is what we need to learn from the West. There are polls year round happening day in and day out on thousands of issues which are mapping the health of the society, the socioeconomic patterns, the issues, the trends, which way the society is heading. Not which way the politics is heading. We need to change that.

To change that, a lot of such surveys [need] to be done on a regular and continuous basis. To do that, a lot of funding is required. Not just from the media, but also from the corporate sector. To map those sentiments, to map the socioeconomic trends of the society, that needs to creep in. Somewhere it has to creep in.

On the challenges of polling in India

SS: What are some of the challenges that you’ve encountered during your career in terms of accurately measuring public opinion in India?

YD: That has to do a lot with two components. One is the training of the researchers. Two is the response rate in different sections of the societies.

Training of the researchers, that’s something which is directly related with the first part of my answer. In the absence of regular polls, even if you train a few researchers here and there, you are likely to lose them. Because if I am going back to Bihar after five years, I am unlikely to get back my trained researchers which I trained five years back. And I am unlikely to get the researcher which I just trained now five years from now unless I have a continuous assignment in Bihar and keep on hiring those people and keep on honing their expertise of interviewing and constant improvement. The researchers is a big issue and that primarily because of the lack of regular research in the field.

The second part would be the response rate of the different categories. In India, you know it’s a very heterogeneous society. Largely speaking, all of us have been getting lesser response [rates] from the minorities, from the Dalit communities, from the females. That’s a historic thing.

That is something which cannot change overnight. We tried to come across these things by say, for example, for minority respondents, trying to field same-faith researchers into those localities to conduct the interviews. That helps [in] getting the response rate better. Fielding more female researchers to talk to more females. That helps. But then, fielding the female researchers increases the cost of the operations. It’s a tricky thing to understand that–

SS: How does it increase the cost?

YD: It increases because of the local sensibilities. Say, for example, if I have to conduct research 100 miles from here, or 200 miles from here, or 500 miles from here into some interior village, and I have to send a team of four researchers over there, ideally I should be sending two females and two males who can move together. If the selected respondent is a female, let the female interviewer take the interview. If the selected respondent is a male, let the male interviewer [take the interview]. Because in the rural areas, it is absolutely improbable for an unknown male to conduct an interview of the female of the household. It’s not part of the culture. It’s not done. Anybody who says that it is done does not understand India or South Asia, to be precise. It’s difficult. It gets easier to get the female response if the female interviewers are interviewing them.

But then if you have to send two researchers in the field, the cost goes up. If you are staying there overnight, if there are two males, you can still pick up one room for them to share. But if there is one male and one female, you have to pick up two rooms. If there is no boarding facility in that area, then you have to ensure that they come back to the district headquarters or their main place before it gets too dark. So your fielding operation times get lower because you field for the security of your female colleagues. And for obvious reasons.

All in all, this is a practical problem for all the polling companies. Barring the metros, barring the areas which are urban, barring the areas where they can hire female researchers who can conduct the interviews and come back on their own safe and sound, it is unlikely to send female researchers to the unknown interior areas without knowing the safety and security [is] in place. That’s the cultural context of it.

And when the clients are not willing to pay extra for covering up those additional costs, then the chances are the [researchers] are, to work within the given budget, more likely to send a male-only team into the field. And those male-only teams are likely to return with lesser female sample because of the non-response.

So it’s a vicious circle. One thing adds to the other. But you can always weight [after the fact]. But in the field operations, these are the things which one has to deal with.

SS: What are some of the challenges you’ve faced in terms of effectively communicating poll results to a client, to a media outlet, or to the public?

YD: To the media outlets, as I said, they always need a number, especially the seat share in an election poll or a political poll. Not everybody understands very kindly that the science of surveys stops at the vote share calculation and conversion of votes into seats is not really part of surveys.

Everybody has got their own algorithm or mathematical formula to do that conversion. But that’s not foolproof. That’s not proven. That’s a work in progress always. Even in the oldest of the First World democracies. What we have seen in the UK elections. Even they could not convert it properly. They could not sense that the Tories would be getting a complete majority in the UK. And they have been doing the polls for — I don’t know — seventy or eighty years now.

The media doesn’t understand. We have been saying this. Everybody. Yogendra has been saying this, I have been saying this, everybody who has been appearing on TV has been saying this for so long. But still, the fascination of media with the seat share is amazing. They just refuse to understand. And the most classical [example] is that some of the media guys are so ignorant that when you talk about the margin of error, they put the margin of error on the seat share calculation. So that is the level of understanding the data. How many media companies have a research editor? Barring one or two, I don’t know if the newspapers or TV stations have research editors in them who understand data. They don’t have [them]. That’s a problem. It’s very difficult.

Yes, classically speaking, the normal viewers, the people actually understand it better. It’s very funny. When I interact within the social media with the people, when I tweet, they understand it better than most of the journalists around. So it’s difficult to make them understand.

Somewhere probably we have to draw the line, that okay, you see we can at worst try to come up with the vote share calculation. This is the seat share calculation. But we can’t hone it. It’s not scientifically proven. It’s only an assertion. It’s only a calculation. It’s only our best idea. But it is not the thing. It’s not the thing.

SS: CVoter is an international firm. You mentioned earlier that you’ve done polling in the United States, South Africa, and the United Kingdom. In a previous conversation that you and I had, you said that nowhere is more difficult to survey than here in India. Why is that?

YD: The sheer heterogeneity of this country. If you look into our operation that you have seen already, look into the kind of languages that we cover, the kind of demographics that we cover, you will understand that polling in India week after week is kind of [like] running a Eurobarometer week after week. It’s as simple as that. If you can understand the difficulties in running the Eurobarometer, you can understand the difficulty of running the Indian operations as well.

And then the sheer length and width of this country. The geography. The topography. How difficult it is to reach the interiors. That is why we took a conscious decision to going to CATI on the mobile, because that is something that shows us that we are reaching the remotest of the places in random probability sample. And we are speaking twelve different languages. And we are doing [the] audio recording of each and every interview. And doing the quality auditing of each and every interview. That kind of quality is possible only in CATI. That is why we took a conscious decision that [the] future is that. We cannot leave the field operations to happen in that way, doing only ten or twenty percent of back-checks in the face-to-face surveys. Everybody understands the handicap of doing that. Everybody understands that the where and why it can go wrong.

For example, in the West, the idea of human labor to be calculated in the per-hour approach. It’s a very objective thing. That you work for eight hours and you will be paid on a per-hour [basis], or how many samples you cover. So the per-sample based approach to the researchers, it is so West[ern].

In India, it doesn’t work like that. Or this part of the world, or even in Africa, it doesn’t work for a simple reason. I send you to a certain village. Thou shall go there and thou shall interview people and come back, and per-interview I am going to pay you this much. Now, you in the best possible way, picking up the autorickshaw, going to the bus station, taking the bus to that village, reaching that village, figuring out the random probability sample, and getting to that person, and the person refuses. And then you have to also worry about, “OK, at four o’clock is the last bus. I have to come back.” So what is happening is that you have spent an entire day doing the best possible work and still you might end up [with] two or three interviews. So if it is not economically viable for the researcher, what do you expect the researchers to do? To give you a complete random probability sample? No. Hell no. They are likely to do a cluster sample and come back and report to you as a random probability sample. Because they are getting paid on a per-sample basis.

This is why about twelve years back, we changed — in our organization, in our system — the per-sample system is wrong. Because the researcher is going there, doing the job. He or she may not get a single interview the entire day. And that’s not his or her fault. So the payment has to be on the complete day basis. That’s where we changed it in our organization. So today in CVoter, it’s been almost for the last twelve odd years, we tell the researchers, “You are being sent to this place. Go and do it. At the end of the day, if you do one interview or zero interviews or ten, you are going to get this much of payment which you deserve [for] doing the hard work. Plus, please make sure, even if you are doing one interview, that one interview should be scientifically, properly done and reported.”

But then this methodology or this way of functioning increases the cost of the operation. And when the quality goes up, the cost goes up. And if the clients are unwilling to pay for that cost, that is where the quality degradation happens. Now, understanding this thing is something which is important. The client’s understanding of this is more important for them to know that this is difficult, this is how it has to be done. So all these things, they need to be taken care of. New, better methodologies and technologies are to be adopted to improvise this further. And the clients need to understand this as well: what are the limitations of it, how it is to be done, and how the best could be achieved from this.

SS: What are some of the tradeoffs that you have to consider when you’re deciding whether to field a survey in person, over the phone, or over the Internet?

YD: Well, Internet is a no-go. First of all, it’s not really representative. For a simple reason that only 5% of the people are actually active on the Internet. The 15% that is floating around is for the people who have the data pack or the data plan, but not everybody who has the data plan is active on the Internet. So first of all, it’s a skewed number. It’s not representative.

The second thing is that even in the West the online opt-in is scientifically not a representative sample. That’s a fact. The so-called river sample is also again not a random probability sample. I mean, it’s a random probability sample among those who are visiting that part and willing to comment on it. So even a river sample is not exactly a simple random probability sample. In India, you cannot even think of doing that simply because number one, it’s not representative, and number two, the incentives with opt-in panels are further going to degenerate in that direction. So as of the current moment, the online thing is not done.

We do a random online thing, when we are doing our CATI every week, we also keep on asking in the final question, “Do you use Internet?” and “Would you like to be part of our panel?” So that is something which is [a] much more randomly recruited panel. But even in that random[ly] recruited panel when we have tried to do the surveys, one reminder, two reminder, three reminders, people are not very keen to click and answer. And probably the third round when we call them from the CATI center, the more likely answer is, “Why are you pushing me to click and answer? Why don’t you simply ask me? I will answer you right now.” So it becomes more like — instead of online — it becomes WAPI, web-assisted personal interview. That means you are talking to them and you are punching the data on their behalf. The only good thing is that some clients understand that and they agree to it whenever they commission it, because we have the audio recordings as the proof of the interview. But still a long way to go on the Internet.

CATI is the future right now because Indians may not read and write but they can certainly talk. This is not my saying. This was something which was said by Dhirubhai Ambani when he was launching his mobile services in India. [In] India, a big number of people are not very educated. A big number are illiterates, to be precise. But they will certainly talk on the phone. Everybody has a mobile these days. So that helps. So CATI is the future, as far as I’m concerned.

SS: Have you found that certain populations — because of cultural factors, because of socioeconomic factors — are more or less likely to be selected and participate in a survey? You discussed this a little earlier.

YD: Yes. The more educated, the more well-to-do, urban, affluent, upper caste, males are more likely to respond. The less educated, females, minorities, lower castes–

SS: Religious minorities?

YD: When I say minorities, I am talking about Muslims especially.

SS: What about Sikhs or–

YD: Sikhs, I never had [problems]. Because Sikhs are, from the class distribution perspective, Sikhs are one of the wealthiest communities in India. Jains are one of the wealthiest communities in India.

So more educated, well off, urban. The response rate of Sikhs is fabulous. [The response rate of] Jains is fabulous. The response rate of Christians is perfectly fine. The response rate of Christians even in the tribals is perfectly fine because in all likelihood they get better education because of the Christian missionaries running the schools. They get better education, so their response rate is better in that way.

But the females at large — regardless of caste, creed, community, or whatever it is, religion — females at large the response rate is lower. The Muslims, the response rate is lower. Unless you are sending the same-faith research into their community to ask the questions.

In certain areas, the Dalit response rate is lower. When I say certain areas, it is directly proportional to how politically empowered the Dalits are in those areas. Before Mayawati happened, the Dalit response rate in UP was very low. But not anymore, because now Dalits in UP are now wearing their identities on their shoulders. So they are not really shy to say. Before Lalu happened in Bihar, the Dalit response rate was low. After Lalu happened in Bihar — one may call his rule as a misrule, or whatever — I have no problem in saying that Lalu Yadav was a game changer in Bihar for that downtrodden, that oppressed Dalit and other communities in Bihar to come up in their social aspirations and their sense of being powerful. So I don’t get a poor response from the Dalits in Bihar. It doesn’t happen anymore.

Yes, in Rajasthan, I still get poor response [rates from Dalits]. In Punjab, I get poor response rates from the Dalits, including the Dalit Sikhs. In Madhya Pradesh, I get poor response from Dalits. So the states where the Dalit political identity is still weak, the political identity being weak is directly proportional to their response rate.

I don’t know how to phrase it correctly. I may sound very unscientific, but I am talking only from my personal experience. The stronger the political identity, the better the response rate of the community is supposed to be.

SS: Do you see this with Muslims as well across states?

YD: Yes. It is there. But with the Muslims, it’s not that they don’t wish to talk. It’s not that. Whenever we send the same-faith researchers, the response rate dramatically increases. They don’t say much, their response rate is down, when there is a sense of polarization. Then they go quiet. But in a normal situation, they speak well. They speak as good as Hindus actually when there is no fear of polarization around.

But yes, it is next to difficult to interview the Muslim females. It is almost impossible because of the cultural context. We must respect and understand that. And it’s not just related to India. I have worked in Indonesia. I have worked in other South Asian countries. I am right now working in the Middle East, bigtime. And I know it’s impossible for a Muslim female to talk to a male stranger. It’s the cultural context of the situation. The same is applicable. And then what happens is that in the puritan way, as many of the methodologists call it, even if she agrees to be interviewed, the chances are that she has to be interviewed in the presence of male members of the family. And when you are asking about the questions that are uncomfortable, can you imagine a Muslim female being interviewed on the uses of contraceptives by a male stranger in the presence of her husband? Can you imagine that? And can you imagine that the answer flowed freely without interruptions or without any things from her husband?

And that applies equally well on the Hindu females. Can you imagine a Hindu stranger talking to a Hindu female in a household about the contraceptives she is using in the presence of her husband or father? I’m sorry, if somebody says yes, he is lying. I’m sorry to be so harsh about using the word “lying” because it’s impossible in this cultural context. There are cultural sensibilities which are to be taken care of, which are to be honored.

And when you honor those cultural sensibilities, that doesn’t mean you are compromising the quality of the data. It only means you are trying to figure a way out on how to get the best possible data. So the answer is that if you have something sensitive like uses of contraceptives, it is always better to send the same-faith female researcher who can interview alone and get the data.

And the same thing happens with the hard-to-reach populations. We did a fabulous survey when we were doing the census for sex workers in one of the districts in Bihar. It was wonderful research. How do you expect a sex worker, a victim of human trafficking, to reveal their socioeconomic problems to a complete stranger? What are the chances like? We know how difficult it is to interview the hard-to-reach populations in stressed conditions. That’s why we did a project over there where we actually trained a big number of children of sex workers who are minimum high school graduates. And we trained them as a researcher to go and conduct the interviews in their localities. Now they knew the localities. They were talking to their moms, their sisters, their neighbors. And they were more likely to get the data correctly. So we named that project as Project VASE, Victims As Social Evaluators. And it worked fabulously.

Even when we were doing our post-tsunami research in Aceh. Because in Aceh, there were conflict victims as well as tsunami victims. It was almost impossible for the normal researchers to go into the interiors, to visit those areas, and talk to the female multi-victims, as we say, those who were the tsunami victims as well as the conflict victims. There we trained a special team of ex-guerrila fighters of GAM [Free Aceh Movement] who had laid down their arms and surrendered and were looking for better jobs. We trained an entire team of them to conduct research on sexual harassment and other cases and other difficulties. Fabulous set of data. Because first of all, they were themselves the victims. They understood the sensitivity of the subject. And they were going to interview other victims. And they were coming up with data which was kind of unheard of. When you get an interview from somewhere in the interior of Aceh, where in the end of the form, the researcher has written that this lady was raped three times by the security forces. Now that’s an insight which no one can ever provide you. That’s an improvisation.

So Project VASE has been something which is very close to my heart. I have been working on that. I used that in Bihar for the sex workers survey. I used that methodology in Aceh. In Sri Lanka, in the northeast of Sri Lanka. It’s amazing. It’s amazing. That is something which I am going to focus more and more on, on how to get better responses from hard-to-reach populations, in special conditions. These are special populations.

But in routine conditions, yes, my solution in India is very simple and straightforward. Sending more female researchers gives you a better female response. Sending more same-faith researchers gives you a better response from Muslims. Sending more same-community researchers gives you a better response from the Dalits.

SS: Going back a bit to your discussion of Muslims, you said that at times when there’s a sense of polarization, the response rate decreases among Muslim respondents. Could you provide some examples of that?

YD: In Gujarat, we have always been getting lesser response post-Godhra [riots in 2002]. In fact, every time there is a Gujarat election we do a special sample in Godhra. That is something which is part of our study also.

Whenever there is a communal riot or something like that, [in the] post-riot scenario, the response rate of minorities actually goes down. Especially in the areas where they are even less spread, but really lesser in number. Probably the fear cycle just takes over. They don’t really wish to talk and be seen as something different. They want to assimilate and be safe. Their response rate goes better only when you are entering a completely ghettoized area where they are kind of feeling safer and the same-faith researchers are going. Then it goes up.

In a normal scenario, they respond as normally as the Hindu or other communities. This happens only when the political surveys are happening and if the atmosphere is charged on communal lines. Then only this thing happens. Otherwise, normally we get the normal response rates.

SS: Did you see that in UP after the Muzaffarnagar riots?

YD: Yes, that happened. That happened in UP after Muzaffarnagar.

SS: The last group I want to check with you is Scheduled Tribes. Do you see a different response rate among Scheduled Tribes?

YD: Yes, yes, yes. It is lesser among the Scheduled Tribes as well. But that lesser response rate is not because of something like they don’t wish to talk. Scheduled Tribes, there is no fear. They wish to talk. They can talk to you easily.

It depends on what exactly are you asking them. So if you are asking about something like– and actually, funnily enough, we do questionnaires like that from clients. “What is your view on Russian President Vladimir Putin? Are you favorable or unfavorable?” Try talking that to a tribal  in Gumla district of Jharkhand. I mean, come on! Come on. I sometimes really wish to ask those who frame those questions sitting in some plush university campus in America or Europe, “Man, go out of your office. Try interviewing ten people you come across outside your office and ask them how many of them know the name of Mr. Putin over there. And you are expecting that a person in a remote Gumla tribal village is going to answer that?” I mean, it is more contextual, more on the subject. So the issue of response among the tribals is not because they don’t answer. You talk about their problems, their day-to-day lives, the issues around them, they are more than happy to answer. It’s just that they are more likely to say in that kind of questionnaire, 99% of the questions will be, “Don’t Know / Can’t Say.” And when you say, “Don’t Know / Can’t Say”, how can we accept this interview where everything is “Don’t Know / Can’t Say?” “Don’t Know / Can’t Say” is information as well. They just don’t know Putin! What is the big deal? Hell. They might even know about Putin but they certainly won’t know anything about the head of the state of Brazil for sure.

How many in this room…? I mean, we are sitting in New Delhi. You go and talk to that table, “What do you think, favorable or unfavorable, of Dilma [Brazilian President Dilma Rousseff]?” The next thing they are going to say, “Ah, Dilmah. They make good tea in Sri Lanka.” (laughs) What are we talking about?

So the issue of non-response also kind of has to be seen in the right perspective. The issue of non-response or lesser response has directly to do with what exactly you are trying to ask.

SS: Do you see big differences in response rates across states?

YD: Yes. There is. There is. The more politically charged-up states actually get a better response rate. The more educated states also give better response rates. The response rate goes down typically in urban, affluent locations where the people just don’t– They are educated. They are upwardly mobile. They are wealthy. They know. But the response rate is going down. They are least interested in talking to you. So that’s a different kind of problem of response. That’s the other end of the socioeconomic spectrum. The affluent class, the upper middle classes, their response rate is going down because they don’t wish to answer. They don’t have the time. They are not interested. They are just too happy counting their money and living their life.

SS: When you said the more politically charged states, which states do you mean?

YD: Bihar, Tamil Nadu, UP, parts of Maharashtra. We are likely to get a good response rate.

People are very happy to talk. Indians are very talkative. They love to talk to you on any issue under the sun. And it’s not that they don’t wish to answer. The problem is that on CATI, for example, for us a ten-minute interview on CATI usually ends up being fifteen or sixteen minutes because people keep on talking. We don’t know how to cut them [off] because it is rude to cut them [off]. We cannot cut them [off]. “Sir, please don’t explain your answer. Just tell it to me yes or not.” No. They don’t tell you as yes or no every time. They give the context and they talk.

Probably they talk to us more because of our media identity. That is something I am agreeing to concede. Our branding is primarily of a media entity. People have seen so much of our name and face on TV and the newspapers that when we say we are calling from CVoter, we are doing a survey for India TV or Times Now, probably coming from the media background opens them up immediately. We also designed the questionnaires in a way which starts with the problems of their life, to open it up. The moment you talk to them about their problems, their lives, they are more likely to open up and talk to you. And then they talk like hell. They keep on talking.

SS: Have you found for election surveys that some states are easier to poll? That you have a better track record with some states than others?

YD: No, nothing of that sort. I believe that from the election polling perspective, the only easier polls are those when the anti-incumbency issue is huge and the feedback from the people are so crisp and clear that there is no confusion whatsoever in your mind that, okay, this government is going to lose. They are the easiest ones. Otherwise, I would not say any particular state is difficult or easier to poll. Every state is similarly as difficult and as easy to poll, but depending on that particular election, whether or not there is a clear cut winner or clear cut loser.

On transparency of pollsters, disclosure of methodology, and the forthcoming Indian Polling Council

SS: A number of the people I’ve spoken to, including researchers — and this is again something you were saying earlier — have critiqued the lack of transparency among pollsters in India when it comes to reporting methodology. What do you think can be done to make pollsters more transparent about how they’re doing their surveys?

YD: We have already taken the first step. You might be aware of this. This we have been trying for a really long time, and now there is a group formally in place called the Indian Polling Council, to start with the colleagues from [names of polling companies have been removed]. They have pitched in together. The idea is basically to make everyone accountable. The idea is to make this entire process more open.

We have been trying. Our office and our working posture has been always open. You have visited yourself. You can talk to any colleague in the industry that those who are confident in their work are always open. When I say open, that means even physically opening up the facilities, to visit the office, see the data, see the process, how it is being done. It is as open as that. So those who are trying to do that in media, especially the ones who have been doing it for a longer time, they really do feel the importance of making it more accessible, more transparent so that the public at large, the media at large gets educated and they can differentiate a good apple from a bad apple. Purely on the merit of the data. Institutionally speaking, it is an important thing to do. So that’s why we have formed that.

The biggest differentiator of those who would be in this organization or this setup and those who would be outside this gambit is the question of accessibility. Anybody who is in this group is committing and opening up for peer review of their data if there is any controversy around. If I am opening up, “Okay, you are happy. Please come and visit my data. See the data. Analyze it. Figure it out. If I got it right, why? If I got it wrong, why?” Whatever it is. If I am opening it up, I am pledging myself to do that, then I am transparent enough for my industry and my peer review. If somebody is not agreeing [to] that and remaining opaque, their problem. But at least people would know who is part of the Indian Polling Council and who is not. Who is opaque and who is transparent.

Somewhere we have to draw the line. And I believe the responsible ones have started realizing that. It took a little bit long. I believe that this should have been done five years back. But better [to] be late than never. We are starting it now, and we hope that this will help us in the long run.

SS: Would the Indian Polling Council determine– members would have to disclose certain things about their methodology, their sampling, their margin of error? I haven’t seen margin of error in almost any of the polls I’ve seen reported here.

YD: Yes. Pretty much. We are going to make a format which will be required to be filled up by all the pollsters every time they release a poll. That standard form would be uploaded on the website. So for any research the disclosures are complete and full. That standard format is applicable to anybody who is willing to offer. And this has been asked by the Press Council of India, that it should be done. This has been advised by the Indian Law Commission, this should be done. And they actually advised that this should be done in consultation, something like the British Polling Council.

I wrote up to Nick Moon, who is the Secretary of the British Polling Council. Nick Moon was running NOP. They immediately agreed to help in every possible way, that we can pick up all those standard protocols, SOPs, from their things and apply. So I wrote to Kathy Frankovich and she happily said that she would be helping in every possible way. I am also a member of WAPOR. So what I am trying to say is that those things which have been done and tested over there, we just have to try and apply everything.

If anybody has a problem with that kind of transparency, it’s for that organization or that person to explain why they are uncomfortable. But if you are to be part of this particular setup, you need to be transparent. You need to be open for a peer review. That’s the bottom line. That if there is a question mark, then you shall be sending the entire dataset for the peer to see through and understand what the problem’s like and help you with the answers. It’s about the intent. Transparency is basically a question of intent. Whether you are opened up or not, whether you have the intent to let the peers review, that’s transparency. Those who are not, it will automatically come out that they are not a part of this group. Because they refuse to follow these procedures. They refuse to follow these standard operating protocols. They refuse to disclose this part of the methodology. They refuse to disclose the minimum amount of information that should be released with every poll. So we are forcing now these guidelines.

This should have been done many years back. I understand that. But, I don’t know why, the seniors at that point of time did not do it. I don’t know why and I am not questioning them. Maybe these were not the important things at that point in time because there were only a handful of players and even then we were in good communication with each other, and offering help and solutions to each other. It was a good, cozy club. We used to talk to each other and give advice to each other. If I had a problem, I could have picked up the phone and [talked to] Mr. Yogendra Yadav and he had been kind enough to suggest [to] me the solution at that point in time.

But now we have moved because all of a sudden we are looking at the industry growing. It’s good that the industry is growing. If it grows, it’s good. But then the growth has to come with the discipline of the quality consciousness and transparency.

SS: How soon will the Indian Polling Council be a more formal organization? Will we see this in place by the time of next May’s elections?

YD: Right before that.

It’s already in place, first of all. We are in regular communication as a group. It’s already in place. We are talking to each other. We are helping each other. It’s just probably I believe that we would like to do it this calendar year, that everything [is] formally in place. Most probably, if all goes well, we would like the general secretariat to be at CSDS for a simple reason that number one, they are the oldest of the bodies who have been doing that, and number two, they are [a] noncommercial venture. They are academic in nature. So they are a nice [place] to be stationed. I hope that Sanjay [Kumar, Director of CSDS] agrees to that. We are likely to have our very first meeting very soon.

SS: Thank you very much.

Rajeeva Karandikar explains how he converts votes into seats

Rajeeva_Laxman_Karandikar

One of the greatest challenges in Indian psephology is the projection of seat counts for Vidhan Sabha and Lok Sabha elections. While survey data can reveal how much vote share each party will win with a high degree of accuracy, they do not show how that vote is distributed across a state. Projecting that a party is to win a certain percent of votes does not say how many seats it is expected to win in an assembly, and therefore whether that party will perform strongly enough to form a government. In cases where the vote share of one party or alliance is much larger than all others, predicting the winner is fairly simple. In cases where the vote share of different parties or alliances is quite close — as is anticipated by nearly all polls for the Bihar elections — the vote-to-seat conversion model is key to predicting the winner.

Because this is such a crucial and challenging step to the projection of election results, pollsters are especially reluctant to share how they do their vote-to-seat conversion. The exception is Professor Rajeeva Karandikar, who for a long time did the seat share projections for the Lokniti unit of the Centre for the Study of Developing Societies (CSDS). Professor Karandikar is very open about the principles undergirding his vote-to-seat conversion models. Last year, he wrote a piece for the Hindu Centre for Politics and Public Policy outlining CSDS’ sampling methodology and detailing the principles of the model he uses to convert votes into seats.

Rajeeva Karandikar is the Director of Chennai Mathematical Institute in Chennai. He obtained his PhD from the Indian Statistical Institute (ISI) in 1981, and was on the faculty at ISI till 2006. He had a brief foray into the commercial world at Cranes Software as Executive Vice President from 2006-2010. Karandikar has made significant contributions to various areas including stochastic calculus, filtering theory, limit theorems, Monte Carlo techniques, and the theory of option pricing. He is a fellow of the Indian National Science Academy and Indian Academy of Sciences, and was awarded the S S Bhatnagar prize by the Council of Industrial and Scientific Research in 1999. Karandikar has been involved with opinion polls and exit polls in India since 1998. He worked with India Today, Doordarshan, and TV Today (Aaj Tak) from 1998-2005, and with CNN-IBN from 2005-2014.

On Tuesday morning, I met with Professor Karandikar at the Indian National Science Academy in Delhi, where he was visiting for the day. Sitting in his hotel room, he explained to me how he found himself working in psephology, painstakingly went through each step of his vote-to-seat conversion model, and considered what might be required to ensure higher standards of methodology disclosure. The transcript of the conversation below has been lightly edited for length and clarity.

On his involvement in public opinion research

Sam Solomon: You are a mathematician by training. Could you please tell me about your background related to public opinion research and how you got into psephology?

Rajeeva Karandikar: I’m a mathematician/statistician and keen observer of politics, though I have nothing to do with it. Fifteen years after my PhD, I was on the faculty here in Delhi at Indian Statistical Institute, and I used to read whatever little was written in the media about the methodology of opinion polls.That may have not been whole truth but whatever little blurbs were coming in, I used to be fairly critical of that. Especially about sampling methodology.

We had colleagues who were economists. One of my colleagues, Bhaskar Dutta, had some contacts at CSDS. He had come to CSDS for some consultation. And that’s where Yogendra [Yadav] asked him, “We are now going to do a big national poll and we want a proper statistician to advise us on statistics.”

SS: This is when he was launching Lokniti?

RK: Yes, in ‘97. So Dutta came and asked me, “Would [you] be willing?” And I said, “Definitely.” That’s how he got me in touch with Yogendra.

At that point, polls were not [that common].

SS: They did them in the 60s and the 70s, right?

RK: Yeah. In the 60s and the 70s, they did them on a small scale because of money. They developed a robust methodology, questionnaire design, all these things they developed. But ‘97, they were going national. Or maybe they had done it once national, but now it was going to be sustained national polls coming in.

You know, questions like, do we go to the same people who we interviewed last time or do we pick fresh sample? What are the pros and cons? So Yogendra had such questions. And he is a fantastic applied statistician. Apart from being a political scientist, he has a great sense of applied statistics. But he wanted to know theoretically how sound it is and so on. So that’s how I got roped in.

And then within a month of our meeting the parliament got dissolved and the polls were announced for three months later. So we got frantically working for a national poll. That was ‘98. That is where it began.

And in that I got involved so intricately that from then on, it’s almost become my second profession. Almost.

SS: So when you described this in the Hindu Centre article, you had these extensive conversations with Yogendra Yadav but also with a British professor, Clive Payne, about how to do this research and how to convert vote shares into seat shares. You say in the article that you thought UK-style models would not work with the Indian political system. I’d like to know–

RK: A little more about that.

SS: Yes. The unique features of the Indian political system.

RK: In any such applied work, whenever it’s involving sampling, let’s say, the first and foremost important thing is what kind of data is available. By that, I mean the sampling frame, that is, the list of potential respondents. The population. How is it listed? That is number one.

Number two, what characteristics of these respondents are available to you? For example, do you know their age? Do you know their sex? Do you know their economic condition? Do you know their education? Do you know their other social background? In a list. Or, if not at the individual level, at what level are these characteristics available?

This is one of the big differences between the UK and India. In the UK, [for] precincts, which are roughly like polling booths or clusters of polling booths, the socioeconomic profile is available. Not for individual persons, but at least at the precinct level.

SS: By socioeconomic profile, what specifically do you mean?

RK: Education, economic condition. In the Indian context, it could be religious breakup. It could be caste breakup. It’s not all, but at least some of the socioeconomic variables — could be education level, could be income level — are available at the precinct level. That information could be used in modeling the final outcome, and when you can do that then that can also decide your sampling. So this whole thing has to be done together.

In India, that is not available. The census data is there, but census data is aggregated and published at the level of a district. So a lone booth or cluster of booths, even a constituency’s socioeconomic profile is not available because there is no unique identifier between districts and constituencies. A constituency can have parts of different districts and a district can be distributed across several constituencies. This is a reality one has to take into account for modeling purposes. So this is one thing which they use strongly in UK models that Clive Payne uses which we do not have here.

The other important difference was what I call volatility of opinion: what percentage of people change their opinion from one election to the next, say Lok Sabha to Lok Sabha, Vidhan Sabha to Vidhan Sabha. In the UK, talking to Clive Payne, at least at that point in time — it may have changed now [since] this conversation was seventeen years [ago] — the perception was that in the UK the volatility is rather low. Across large sections, all their lives they continue voting Labour or Conservative. Whereas in India, it’s highly volatile. Extremely volatile. As you can see from successive percentages, if you try to look at any kind of data historically, you can see wild fluctuations at the aggregate levels. So now you can imagine [the fluctuation] at the individual level.

At one point, that is ‘98, with that Lok Sabha poll, we had the luxury of doing the following: We did a pre-poll for the whole country just before the first phase, published the outcome. Then CSDS had something for a television program, a counting day program, where Yogendra’s idea was, and we did it rather successfully–we researched, we worked out the model, and the idea was his, but we worked it out: In those days, the counting used to take three days or two days. And the idea was that at any given time, looking at the counting data at that point, [to] make a prediction of the final parliament. So you take the opinion poll and you take the counting data. So for this purpose, Yogendra got money to do a post-poll. Though we were not going to go on air with our findings of the post-poll, we got money to do the post-poll, which was going to be used for this part. This part worked out beautifully. Anyway, that has lost meaning with the electronic voting counting and [after] four hours everything is over. So our ESP, as we called it, Early Seat Prediction or ESP, that has lost its sheen. But at [that] point we did have it.

Anyway, the point is we got the money to do the pre-poll and the post-poll and we went to the same respondents. What we found was that 30 percent of people had changed their vote. And the timeline was, one third of the country eight days, another one third sixteen days, another one third maybe twenty-six days. So on average a little over two weeks and 30 percent of people have changed their minds. What that shows is actually opinions get formed at the fag end just as voting day comes near.

So I have grave question marks on the predictive power of any opinion poll done by whatever, [even] the best of methodolog[ies]. In fact, for several years CSDS and me, that is, we were not doing any pre-election poll.

SS: This was the first year, with Bihar, that they got back into pre-polls.

RK: In the assembly. For the Lok Sabha, we were resisting, but we were told that if you don’t do it, we have to go to someone else. The channel cannot just stay out of that race. So we did that.

Predictive power is rather poor because the volatility is too huge. Some small thing, some statement by someone changes public opinion and everybody turns around and changes their mind. In fact, at a few points, in various times in CSDS polls, we started putting questions like, “Have you decided whom you are going to vote for?” Then, even if they said no, then our follow-up question is, “If the polls were tomorrow, whom are you going to vote for?” Again, 25-35% of people say they have not made up their mind in the pre-poll data.

SS: Did you find that the people who made up their minds later fit a certain demographic profile?

RK: We tried this, but not really. India is not really one homogeneous country. But as soon as you drill down to a state or one section then your sample size is too little to make any such inferential correlations. And it changes from one election to the next. Perhaps the more educated ones are less likely to change.

SS: Is that your own personal experience? Your friends, your family, do they switch their votes pretty regularly from one party to another?

RK: Not as much as we found in the surveys. My friends, my family–

SS: And yourself?

RK: Myself. We make up our minds fairly early. But this fraction is low.

With Bihar — now I have not looked at any post-poll data, I don’t know if you have — but without the post-poll data, I’m not putting too much weight on the pre-poll findings. At best, my statement is that while going into the polls they were neck-to-neck, both sides. And later on we are hearing all kinds of buzz by the media, most of it the English-language media, who are painting one kind of picture. Maybe that is the correct one. Maybe they have feedback from the post-polls or exit polls. I don’t know.

SS: I guess we’ll see.

RK: We’ll see.

On vote-to-seat conversion

SS: Based on your article, a key assumption of your model is that the change in percentage of votes for a given party from the previous election to the present is constant across a given state.

RK: Or some region in the state.

SS: Or some region in the state. This is actually different from what I’ve heard from some pollsters about elections in India and they have said that there isn’t really such a thing as uniform swing.

RK: It is true that any kind of statistical analysis post facto, or back testing as it is called, if we do this model it fails miserably. But this model is the basis which allows me to mathematically come up with a methodology, as opposed to just ad hoc. And this has proved to be reasonably good and no one has been able to show me a methodology which is better, other than their gut feeling.

SS: And a big part of it is that you’re predicting the aggregate number of seats, not particularly for any individual constituency.

RK: Absolutely not. Absolutely not for the constituency. In fact, when we do parliamentary polls I used to be extremely reluctant to even give [projections] at the state level. I would say that at best we will give the four regions: north, south, east, west. Not even at the state level.

SS: So you notice state swings but also regional swings within states. How do you group the regions of a state then?

RK: Grouping of regions is done politically by CSDS. So they’ll say, “Socioeconomically, this is [region] 1.” They used to do the splitting of states. It was generally geographic, but still how would they draw the boundaries? This is where their understanding of the socio-economic conditions across the country played a crucial role.

Let’s take Maharashtra. Four regions. Greater Bombay is one region. Now suppose we measure that the overall swing for BJP statewide is 5%, and in Bombay region it is 3%. We’ll assign some way, either half-half, or one thirds-two thirds, so I’ll take half of the state swing plus half of the region swing; that would be my swing for every seat in Bombay region.

Now these regions can also be extended to what we may call phases. Now that elections are going on for a month and what happened first phase and what happened in the fifth phase, lots of things have changed. So I’ll take one third of the state swing, one third of the phase swing, and one third of the region swing, let’s say.

SS: The phase swing, what would that come from?

RK: The phase swing would come from last election in this phase what was the percentage of votes, and in my opinion poll [for this election] what was the percentage of votes. So the difference will give you the phase swing.

SS: And you look at the phase across the entire state?

RK: Across the entire state.

SS: And so you take the last election, 2009, or 2014 and you’d say–

RK: No, no, no. So in Bihar, there are lots of interesting points which will emerge. Primarily, we take not 2014 but the 2010 Vidhan Sabha poll.

SS: For Bihar 2015, you take the 2010 Vidhan Sabha data.

RK: However, from 2010 to now, lots of things have happened. The alliance structure has changed dramatically. Somehow we have to take that into account and come up with what we call a simulated initial file. If nothing had changed, the 2010 [file] would have been the initial file on which we look at the swings. But now what we do is — this is ad hoc — but firstly, wherever BJP contested on behalf of the [former JD(U)-BJP alliance that governed Bihar from 2005 to 2013], I would split it, let’s say, 60/40. And if it is a JD(U) candidate, I will give it 60/40 [favoring] the JD(U).

SS: And how do you come up with those figures? You just have to make some sort of subjective judgment.

RK: Yeah. Some sort of subjective judgment. And it used to be easier when Yogendra was a partner because then he would have a very good sense of these things.

SS: Just from his own experience.

RK: Yeah. And over the years I think I have developed a keen [sense]. So now I don’t fret over it. I do it and proceed.

The other thing is it is very robust because it is only used to do the initial file. 60/40 or 55/45 or even 65/35 is not going to make a huge change in the final analysis for the total prediction at the state level. I’ll experiment with this. Of course if I do 10/90 or 90/10 it will not be [similar], but in a certain range it really doesn’t change dramatically.

SS: It doesn’t make a huge change at the aggregate level. Maybe at the individual constituency [level].

RK: Aggregate level.

So likewise, when two groups align, then we don’t simply add their votes, but there is going to be some friction. So we might take 85% or 90% or something like that.

Or if it were UP [Uttar Pradesh], let’s say, and in the previous election Mayawati and Congress fought separately, and let’s say in this election they are fighting together, Yogendra would say that, “Whatever were Mayawati’s votes, even if there was a Congress candidate, it will transfer 100%. But whatever Congress votes, if there was a Mayawati candidate, it will only transfer 50%.”

SS: And that’s just knowledge he has from being in Indian politics for a very long time.

RK: Yes. So we take into account whatever we can to bring in a subjective judgment about, “If this were the alliance picture [in the previous election], what would the vote have been the last time?”

Now, in Bihar, there is another challenge. We see that there was a Lok Sabha election which changed the realities tremendously. So for Bihar what I had been doing was, I took the file from 2010, applied the 60/40 [vote split between the JD(U) and BJP], and then I took the [2014] Lok Sabha [file], and I took an average of these two.

SS: You took an average of the [re-weighted] 2010 Vidhan Sabha file and the 2014 Lok Sabha file as your base [file].

RK: Under my model, the level of support is determined by the opinion poll. I am not applying any subjectivity there. The whole thing is how is that vote distributed across the state? If you say the BJP is getting 41% in Bihar, how was it distributed across the constituencies? So my base file’s only role is to make an assessment about how it is distributed.

SS: How it’s distributed within each constituency.

RK: Within 243 constituencies [for Bihar]. Where is it that they are polling more than the average? Where are they lower than the average? Things like that.

SS: And then you’re going to take the uniform swing — or the uniform swing plus the regional swing plus the phase swing — to calculate what you think would happen in each constituency?

RK: Once again, the base file and the model together tell me how this 41% of votes is distributed across the constituencies. That model, I am not using it to say it is not 41%, it is 44%, no. The level is pegged by the observed data. The model and the base file altogether only help me in attributing that across constituencies. For the BJP as well as for others.

SS: At the aggregate level and not in individual constituencies.

RK: No, no. I do it at each constituency, I aggregate assigned votes. Then I would say who is winning where and I will add it up to get my final number.

SS: But the actual constituency level projections are not– the model doesn’t do well at the micro-level.

RK: No, no, absolutely not. Absolutely not.

SS: [So] in the aggregate the errors cancel each other out. Or at least, that’s the theory.

RK: That’s the theory. And at least I have observed [it] in practice reasonably well. If our estimates are good, this is good.

SS: I was hoping that we could — we kind of did it already — but we could walk through each stage of the process, slowly and clearly, just to explain the vote share to seat share conversion. So I have it all on [the] recording.

RK: Yeah.

SS: So first, you do the survey to determine the uniform swing across the state. You also have a  regional swing and a phase-based swing. And then you’re using that to calculate projected vote share in each constituency.

RK: For each party. For each of the major parties.

SS: So, for example, if the BJP is projected to win 5% more votes statewide in Bihar, and they won 36% of Constituency X in 2010. Then it becomes 41% in 2015. You just use Vidhan Sabha figures to build the model, but you just said that–

RK: In special cases like Bihar this time.

SS: And the reason for that is because the 2014 Lok Sabha was such a huge–

RK: Tectonic change. It’s a big change.

So that is why while there is one broad model, there is no relevant software where you just plug in data. Each time we may have to come up with some tweak, like this one in Bihar.

SS: So once you’ve got the projected vote shares in each constituency, you calculate the probability of victory for each of the parties for each constituency. Do you do this by doing a multiple proportion z-test for each constituency?

RK: Something like that, something like that. And there is again a subjective parameter, the sigma, or the standard error for the vote estimate.

SS: That’s subjective. You’re using the standard error for the entire sample but you’re using it for just a constituency.

RK: So therefore we put a factor of safety. So whatever is the sigma, I inflate it by 2 or 3 or something like that.

SS: You’ll inflate the standard error to allow for more uncertainty in the model. Okay.

But you only do this for the top three parties. Why do you stop at three? It seems like there might be some cases in India where more than three could be competitive in a constituency.

RK: The first time we did it, we did it with only two [parties]. And then we found that there were [many] places where the party [that was in third place] ended up winning. So we extended it to three. Now, rarely is someone who is fourth in my estimate actually winning. It’s very rare.

It can be done. It will make matters a lot more complex.

SS: And it doesn’t add a lot to the model.

RK: It doesn’t add a lot to the model. That’s the point.

To come up with these conversion tables and so on, all we had to was do it with the bivariate normal distribution. If I have to go to four, I will have to go to three-dimensional Gaussian, and that can be done. If it was necessary, it can be done. But it simply was not worth the effort.

SS: It sounds complicated.

And then you say it’s as simple as summing the probabilities of each constituency across all the constituencies. But wouldn’t you want to run the model many times and see what kinds of distributions you get?

RK: Of course. Of course. That I do.

So, for example, I can play with the different weights for state/region/phase [swings] and so on and so forth. I can play with different weights there. Then I can vary the sigma parameter, the inflating factor. I’ll run it several times and then get a sense and try to position myself somewhere in the center, normally.

Whatever are my subjective parameters, I vary those. I don’t vary the vote percentage. I vary the subjective parameters.

SS: Does that include the base file? For example, if there’s an alliance–

RK: To begin with, I might have made two or three base files. I don’t vary them tremendously, but I might have made two or three base files.

SS: If there is some unusual alliance, as there is in Bihar.

RK: For Bihar, what I have is one pure 2010 base file, one pure [2014] Lok Sabha base file. That I will not use, but I will [weight them against each other by] one third-two third, half-half. So this is the variation I might try.

SS: With each of those, you do different projections based on changing state swing, or regional swing?

RK: No, weight for the regional swing.

SS: Ah, how much you balance the regional swing and the state swing.

RK: The main point is none of my things make a modification to the opinion poll data. It is just interpreting that vis-a-vis seats which is where I play with my subjective model and parameters.

SS: And again those would include how you’re weighting Vidhan Sabha versus Lok Sabha data for the base. It would include the different vote shares that the alliances are picking up. And then also you said the sigma, so how much uncertainty you add to the model for each constitutency. Any other subjective parameters that I’m missing there?

RK: Not really [editor’s note: we both forgot to mention here how the state, regional, and phase swing are weighted against each other].

SS: How does this work for Lok Sabha elections? You say in the article that there’s uniform swing within states, within regions of seats [editor’s note: this is incorrect. He uses uniform swing and regional swing as the basis of making projections but does not describe the seats as uniformly swinging one way or another], but not across the country. Do you treat each state as its own election?

RK: Absolutely. Absolutely. Because that’s how India is, at least electorally. [There are] districts across Tamil Nadu and Karnataka boundaries, or districts across UP and Bihar boundaries. Socioeconomically they may be very similar but political behavior is very different. That’s why there is no nationwide swing or nationwide effect. So I treat each state as separate.

SS: Those subjective factors that you listed in the model, [those] can account for a lot of [variance] in seat projections, right? For a lot of the polls for Bihar, a lot of the vote share projections have been pretty similar but the seat share projections can be pretty different.

RK: See, but nobody else even says how they are doing the seat projections.

SS: So you have no idea how the other–

RK: No, because they don’t say. I am the only one who says, and I go and give lectures, and my methodology is public [and] open.

SS: But no one else does this because it’s their “secret sauce.”

RK: I believe there is no “secret” and there is no “sauce.” They just do it ad hoc, is my guess.

SS: Do you think so?

RK: Yes. They will have some rule, “Oh, if it is two parties and only 5% swing then maybe…” Something like that. Some rules.

SS: But not transparent.

RK: No, certainly not transparent.

SS: And not methodical in the way that you’re describing here.

RK: At least, they have not disclosed it.

I would have no issue in debating with anyone my methodology versus theirs, but no one else comes forward.

On transparency of pollsters and disclosure of methodology

SS: You spoke at the beginning about transparency in disclosure of methodology, and how you were looking at this in the ‘90s. Has it changed a whole lot from the ‘90s to now?

RK: Not really. Not really. We zeroed onto a fairly good methodology and it’s these tweaks, little tweaks, taking into account the ground realities which we may be doing, but other than that the primary methodology has remained the same.

SS: In terms of the reporting, the disclosure of how these surveys are done. I’ve talked to people in media but also researchers themselves who say, “We need to get better about disclosing our methodologies.”

RK: Absolutely. The only thing is it cannot be mandated by law, but it should be voluntary by the media.

SS: Why can’t it be mandated by law?

RK: There is a very tricky issue. The regulation of media is a touchy issue. What can be regulated and what cannot be regulated is also debatable.

For television stations, the government can come out with a law because these are of recent origin and when they got licensed to uplink their signal from the land, they signed on [to] all kinds of norms, and opened themselves for regulation by the government. So they can be regulated. Otherwise, their license can be cancelled.

However, when it comes to print media — because print media predates independence by a century — print media cannot be regulated. In fact, Mr. N. Ram [the publisher] of The Hindu [has given] lectures in which he has categorically said that, “We obey your 48-hour [embargo on releasing polls before Election Day] of the Election Commission, because we think it’s a good idea. Not because we are bound to it.”

SS: So there has to be some kind of internal self-regulation.

RK: There is something called the Press Council in India. There is some other body for the television broadcasters association. These bodies can together come up — Yogendra had suggested this, and I strongly support that — at least, when was the survey done, what was the sample size, who led the survey, what [were] the names of the team members who did that. And a little more about the vote percentages / seat percentages methodology.

After Yogendra became a member of AAP, he in fact said that the entire survey data should be made public. That maybe is going a bit too far. But it’s like an audit. No company accounts are made public. However, it is subject to an audit by a professional. Likewise, we can be required to maintain records in a certain fashion that can be subject to audit by other professionals.

SS: Why do you think this hasn’t happened yet, if Yogendra has talked about it, if so many researchers have talked about it, [and] people in the media are talking about it?

RK: I’m not sure how many actually want it. Yogendra definitely wants it. I want it. But for both of us, I have to say, that for both of us it was easy to say and even practice because this was not our primary profession. We both have a different profession. We have our livelihood secure. Whether this opinion poll happens or doesn’t happen, whether we are engaged or not engaged, we don’t care.

With Rajdeep Sardesai and me and Yogendra, we had a very clear understanding that Rajdeep had no say in what we were going to say. He could say, “Hey, what you are saying doesn’t seem to be happening here. It’s not going to happen.” We would say, “Okay, we’ll examine our analysis once more.” But he could not tell us to reduce it or increase it, no.

SS: You had complete independence from media sponsors.

RK: Yes, absolutely.

On experience conducting surveys over time

SS: You got experience not just doing the vote-to-seat projection but also working on the surveys themselves. The specific focus of my research is systematic sampling error in the polls, whether there are certain populations, demographic groups — gender, religion, caste-based — that are more or less likely to be represented in these surveys. Did you find consistent patterns related to that?

RK: With CSDS data, I don’t think there was a consistent pattern [other than] underprivileged groups being slightly underrepresented.

SS: Which types of groups specifically?

RK: Rural areas, SC [Scheduled Castes], ST [Scheduled Tribes]. But this, Sanjay [Kumar, Director of CSDS] would have a much better sense than me.

But with other surveys, there are grave doubts about what happens, when they do convenience sampling. You know most others do convenience sampling, quota sampling.

I have [put] on record [several times] that whatever [are] my successes or limited successes is largely due to great quality data by CSDS. If the data is no good, my analysis cannot be good to begin with. And CSDS has done a phenomenal job.

SS: Have you found that certain states are harder to project than others?

RK: Not really, though our track record said that we did very well in east India and better in south India. Not so well in west India. (laughs) But it’s by chance. I am not sure why it happened.

In the east, [Lokniti and I] have a phenomenal record.

SS: The northeast?

RK: East, not so much northeast. [In the] northeast, only [in] Assam have we done surveys. Assam, we have done very well. Bengal and Bihar, we have got bang on when others did not expect it. Both victories of Nitish. the first time in 2005 that he gets a majority, and then in 2010 [when] they crossed 200 [Vidhan Sabha seats], nobody had dreamt. Nitish himself had not dreamt. And we got it.

SS: And you did very well for southern states as well?

RK: Reasonably well for southern [states]. Not so well for [the] west. I am from [the] west. (laughs)

SS: Which southern states and western states do you mean?

RK: Gujarat in particular, we had a checkered record. Not so good. Maharashtra, so-so. But [the] south: Tamil Nadu we have done well, Karnataka we have done well. AP [Andhra Pradesh] okay.

SS: Thank you very much.