Sanjay Kumar on the challenges of survey research and CSDS’ role in Indian society


Sanjay Kumar is the Director at the Centre for the Study of Developing Societies (CSDS). He is also the Co-Director of Lokniti, a Research Programme of the CSDS. His published books and edited volumes include Indian Youth and Electoral Politics: An Emerging Engagement, Changing Electoral Politics in Delhi: From Caste to ClassMeasuring Voting Behaviour in India (with Praveen Rai),  Rise of the Plebeians? The Changing Face of Indian Legislative Assemblies (with Christophe Jaffrelot)and Indian Youth in a Transforming World: Attitudes and Perceptions (with Peter R de Souza and Sandeep Shastri). He has contributed chapters for several edited volumes, written various research reports, published articles in both international and national research journals and writes regularly for popular newspapers.

As an expert in survey research, Kumar has directed various national and state level studies conducted by the CSDS, which include the National Election Studies (NES) 1998, 1999, 2004, 2009 and 2014, various round of bi-annual ‘State of the Nation Surveys’ and various state level studies. While his core area of research is electoral politics, he has also been engaged in survey based studies on Indian Youth, State of Democracy in South Asia, State of Slums of Delhi, State of Indian Farmers and Issues of Electoral Violence.

Sanjay Kumar is also the supervisor for my research project. Last Tuesday, we sat down in his office at CSDS to discuss his background in survey research, the challenges of survey research in India, and how he defined CSDS’ goals and mission. The transcript of the conversation below has been lightly edited for length and clarity.

On his background in survey research

Sam Solomon: Please tell me how you got into the field of survey research.

Sanjay Kumar: I won’t say it was very pre-planned. It was kind of an accident.

I had enrolled myself as a PhD student in the Department of Political Science at Delhi University. But I also wanted to get hands-on experience of how you collect data, how you analyze data. So I came to this place [CSDS] to work along with the team that was headed by Yogendra Yadav and VB Singh. This was 1993, October. They were initiating a project to study assembly elections which was being held in five states: Delhi, Madhya Pradesh, Rajasthan, Chhattisgarh, and some other state. I came to know about the project. So I came and [told] them that I would be happy to work with them on the project. They said yes, this is a small project, we would be happy to associate [with] you for a few months. Maybe two, three, four months.

So I started working on the project. Got more and more interested in the kind of analysis which they were trying to do with the data, the fieldwork, etc. With my initial… not contract but kind of informal understanding that I would be working for three, four months, I also got a chance to the go to the field. I went to Madhya Pradesh to do a study of one constituency. I extended my research to making a comparison of four or five constituencies, looking at the Muslim voting behavior, which is my first article in EPW [Economic and Political Weekly], “Myth of Muslim Vote Bank.”

So I continued. I worked for three, four months. Then the project was getting winded up. But still VB Singh and Yogendra Yadav were looking forward to bigger studies in Bihar and in Lok Sabha elections, 1996. So they checked with me if I would be willing to be part of the project for the next six months or one year’s time. Since my initial experience of working three months, I got very excited about the kind of work which they were trying to do. So I said yes, I would be willing to work for the next six months or so, or one year. Then I think we moved into some kind of formal contract.

And then time passed. After that, we had the assembly elections in Bihar. I was deeply involved in that because I belong to Bihar. So my home in Patna was kind of a central place from where we were running the project. I went around to several constituencies to do the study. I especially studied one constituency, Mokama. But along with that, I was also coordinating the entire study in Bihar.

And by that time, I had almost decided that now, it’s enough. I should actually go back, do my PhD, etc. So that’s the time I gave a hint to Yogendra Yadav, with whom I was closely working, to say, “I think it’s enough. I should go back to my PhD, etc.” And then, I think, this was a conversation I think we were having at the railway station when I came to see him off. He said, “Let’s discuss this when you’re back in CSDS. Why do you want to take a decision immediately?”

And when I came back, we had [a] long chat. He said, “If you look at the history of CSDS, people who have worked here, their careers have gone up. Their careers haven’t gone down. So why do you want to leave? You can continue your PhD. You can continue working with the projects if you want. We have a space for you.” Space, in a sense, space in a place to work with me. I thought several times, and then decided okay. Then he said, “Now we’ll work out another arrangement. In a sense, it would be kind of a lecturer-level arrangement now. You have been working with us the last two years as a research officer.” The initial six months was a research assistant, then it was kind of a research officer. After the Bihar assembly elections, they said, “Now we are heading to the big Lok Sabha elections. Now you have gained experience. We value your association. And you would gain by working with us, and I don’t think your career is going to be on a downward slide. And if you agree to work, then we can work another arrangement. Give you a lecturer’s level position. So you will be a research associate now.”

I decided to continue and then finally after the Lok Sabha elections of 1996, we had a detailed discussion about how I [saw] my career moving. And they said, “Now we want to take this election study forward. We don’t want a break in this. And we would like a person like you in a permanent position. Are you interested in that kind of thing?” The institution advertised for that position. I applied along with several other applicants, and I was selected by the selection committee. Formally, I would say I joined CSDS faculty in April 1998.

[For] my entry into survey research, this is the kind of journey I took. But election as a theme has always interested me. It’s not that it was something not interesting for me. Even when I was in my school days — I remember class 12 — I was so fascinated about elections that I used to do, if I recollect, my kind of psephology. I used to read newspapers and prepare charts on my own. Who is likely to win from which place? Though I miserably failed because this was 1984 elections, and ‘84 election was a wave election for Congress. That was the time, if I remember correctly — and I do remember correctly — that I was kind of anti-Congress. Really did not like Congress. So when I was doing my own kind of psephology, I think I used to be a little biased. When I was making a judgment about who was going to win, I would actually defeat most of the Congress candidates on paper. “Oh, he will get defeated…” I remember defeating on paper [Bollywood superstar] Amitabh Bachchan because he was a Congress candidate contesting against H.N. Bahaguna, who a very tall leader from another party. That’s my entry into survey research.

SS: You joined CSDS informally in 1993, so you’ve been with CSDS for twenty-three years. During that time, which aspects of survey research at CSDS have changed the most and which have remained consistent?

SK: I think the big change is the timelines. When we started working in 1996, ‘96 was not with TV. It was with newspapers. But when we are working with television as a medium, the analysis has to be very quick and very sharp. If one is writing an article in the journal, then you have all the space. You can write 5,000 words. When you’re writing in a newspaper, you have to restrict to 800 words, 1,000 words. But when you are actually providing stuff to the television, it is very little words but it has to be very, very sharp. So I think our rules of analysis, trying to do deep analysis, that’s one thing that has changed. [In] 1996, we would do broad analysis maybe. And it would take a lot of time. That has changed.

The second thing is about the sample. It is also connected with the analysis. Earlier we used to do more aggregate analysis. We would come down to a few states, bigger states where we had bigger samples. But now we know that we not only do analysis for big states, we have been able to do analysis even for small states. Even within the states we actually disaggregate the analysis. We try to go down the level of analysis to figure out what’s happening at the ground level itself. That is a big change.

The other is that, initially, ‘96, ‘98, ‘99, even up until 2004, the survey as a tool we applied only for election studies. Now we have diversified a lot and we apply the survey method to do studies on various other themes. We have done the study on farmers. We have done the study on youth–two studies on youth. We have done studies on Naxals. And various other issues: economic reforms, Muslims, women, eating habits, all kinds of issues we have touched upon. That’s a big change. The initial years, we were only looking at using surveys as a method to study elections, voting behavior. Now we have used that tool to study various other kinds of issues.

On the challenges of survey research in India

SS: What are some of the greatest challenges you’ve encountered during your time at CSDS when it comes to 1) accurately measuring public opinion and 2) effectively communicating the findings from your research to a media outlet or the public?

SK: I think the biggest challenge has been to keep up the reputation CSDS has got. Because as soon as something is put out in the public, that this is a study conducted by CSDS, moreso with regards to elections — vote percentages, which party is likely to win — a large number of people believe it is a truth. And they would say, “If CSDS is saying Party A is winning, definitely Party A would be winning, because they never go wrong.” So that’s the biggest challenge. Maintaining that credibility in the field and trying to do it as accurately as possible.

SS: How do you address that challenge?

SK: Being neutral all the time. We try not to be biased, not to be swayed [by] our likes and dislikes for leaders and political parties. And the big advantage when we are trying to do the analysis, when we are trying to assess which party is likely to win the election, we take advantage of the huge datasets we have in our hands. The historical data.

Suppose we see the Bihar elections. For Bihar, we would have at least eight or ten survey datasets. So we try and look at what has been the pattern when we did surveys in the state. In order to try to be as accurate as possible, we try to be neutral. We try to collect data in a neutral manner, do the analysis in a manner as neutral as possible, plus take advantage of the past data to figure out if at all we made any errors when collecting data in a particular state. That’s how we try to make our findings as accurate as possible.

SS: When it comes to the surveys themselves, have you found that certain types of populations — gender, caste, class — are more likely to be oversampled or undersampled? If so, which populations have those been?

SK: If you look at our sampling technique, our sampling technique is very different compared to the sampling technique of other survey agencies or organizations. We not only sample the constituencies, the polling stations, but we also sample the voters whom we want to interview. All of these are done randomly, systematic random sampling. If we have adopted a good sampling method, a scientific sampling method, and if investigators actually follow that method — when they go to field, they go with a list of people who have been sampled, and they have to interview only those people on the list — if they actually follow that, there is very little chance of oversampling or undersampling any segment of the population. Unless we are looking for a very small sample.

Normally, we take a sample of say, 3,000 or 4,000 in a state. The chances of the sample being overrepresentative [of a demographic segment], over or under, is very [small]. That has been the record of our surveys for the last twenty years. We have put all this data on our website to figure out when did we go wrong with a sample that is representative.

But still, if we look at the chances — still there are chances — normally, we might expect women to be slightly less in our sample because it also has a relationship to the investigators you have. If there are some states where we don’t manage to get an adequate number of women investigators– it becomes extremely difficult to interview Muslim women if you don’t have woman investigators.

SS: Muslim women investigators or just women investigators?

SK: No, just women investigators.

Normally, there would be a lower completion rate in the urban areas because the urban population is more mobile compared to the rural population. Sometimes we also get samples which are underrepresentative in terms of the urban population.

These are the two elements of the sample which I have noticed that sometimes we get wrong, in a sense, underrepresented. All the time they would be underrepresented. Hardly overrepresented. Urban and women.

SS: When it comes to SCs (Scheduled Castes) and Muslims, you have census data that you can consult to get the proper proportion of SCs and Muslims in the states. So it’s not as much of an issue.

SK: I don’t think that we had issues with regards to the Muslims and the dalits, SCs, when it comes to getting a sample.

Unless we are doing an exit poll. We used to do exit polls earlier but now we have stopped. Because if we are doing exit polls, there is always a chance of heavy overrepresentation of young, urban voters. And also this issue of dalits, Muslims, or tribals not being adequately represented in the sample, that’s the case when we do exit polls. That’s largely the case. And that’s why since we are not sure of getting the right representative sample or not, that’s one of the reasons we stopped doing exit polls.

That’s a big problem in exit polls, but not in the pre-poll or post-poll surveys which we conduct. Only because we actually sampled the respondents from the electoral rolls.

SS: What about OBCs (Other Backwards Classes) or intermediate castes?

SK: There are very rough estimates about the proportion of OBCs in different states. It ranges from 40% to 55%, 60%. And it is such a large group that getting them not adequately represented in the sample is rare. But since you don’t have an exact estimate, it’s not like Muslims, it’s not like dalits or tribals, [where] you have data to compare [to the sample]. OBCs are only an estimation. In Bihar, we have roughly about 52%. We don’t know whether they’re actually 45% or they’re 55% or they’re even 60%. So that has not been an issue in the past.

SS: I find the issue of caste to be really interesting, maybe because it’s a foreign phenomenon for me, but also because it’s so complicated. Last week, [the Lokniti survey research team] were trying to figure out how to code the different castes in Tamil Nadu. Everyone in the office is north Indian, and the caste system in Tamil Nadu seems really complicated. We were struggling to figure out which caste gets categorized as upper caste, and which as intermediate, and so on.

How do you do that for every state in India? Do you have the lean on the local knowledge of the Lokniti network?

SK: You will notice that we use a codebook to code several open-ended questions, caste being one of them. That codebook has not been prepared overnight. If you get sometime a chance to look at our codebook [from] ‘96, ‘98, ‘99, [you can see] how it has progressed, how it has improved, how it has become more comprehensive now compared to the past. The 1996 codebook was one sheet, one page, for all the codes: education, occupation, caste, etc. It was just one sheet. Now if you look at our codebook it is a booklet. Six or seven pages have been devoted only to caste.

This has been possible because, as you rightly mention, of inputs from our local coordinators. At one point in time, we actually took it as a project to engage our local coordinators, and requested that they engage sociologists at the local level to figure out where each caste should be placed, and what is the parallel of caste A to castes in other states. Caste coding is really complex, and we have been able to develop [it] after a long, long period of experience.

Still, we won’t say that it’s completely perfect. Because every time we do a survey, we encounter this problem in varying degrees. And we try to develop it as [neatly] as possible.

On the goals and mission of CSDS

SS: Which factors do you weigh when deciding which research projects you are going to go forward with? Naturally, there’s always going to be an audience for election surveys. People like to read about politics and elections. But how do you decide to do a survey on farmers, or youth, or Naxals?

SK: You rightly pointed out that elections keep happening every year, and CSDS is a place that is known for its research on elections. So election studies we keep continuing. On other issues, whether we want to do a survey on Topic A, B, C, or D, the big consideration is: is this a topic [about] which people want to know, people would have curiosity? Is it topical? Do people have concerns about that?

So if you look at the kind of research which we have done on youth, there’s so much focus on youth, not only in India but in South Asia and other parts of the world as well. Their anxieties, aspirations, employment, unemployment. So we have done studies in the past on youth, and also we are in the process of doing another round of the youth study to look at the change over a period of time.

Farmers — if you actually look at the debate going on in the country for the last couple of years, [there is] so much focus on the crisis of farming in India related to land acquisition bill, etc. When somebody approached us with the idea that there is a need to do a proper study of farmers in India, though there are other studies — but mainly done by economists who look at the economic condition of farmers mainly. So we pitched in. We did the study on farmers. Along with looking at the economic conditions, we focused more on anxieties, aspirations, expectations, more on opinion rather than on fact. That is how we picked up that study.

On Naxals, [there is] so much focus on this issue of Naxals. Is it an issue of economic deprivation or is it a law and order problem? That’s the big question people want to address. Some people see it as a law and order problem and they say, “The government should use hard force to just crush them.” The other side says, “It is an economic issue. These are poor people whose land has been taken away by the government. It is an issue of their survival.” So we wanted to do a proper study [to] figure out what is the situation. We are not saying whatever we would say would be the truth. But at least we would try and bring out what is their situation on the ground.

So that’s how we pick up the themes. But we have also been approached to do studies about consumers’ perceptions about the new airports which have been developed: Bangalore, Hyderabad, etc. We were quick to say, “No, we don’t want to get into this.” This is none of our business. If we want to do a study, it has to relate to [the] social sciences. We don’t want to get into airports. We were approached to do a study about something related to electricity. Supply of electricity, are people happy, should prices go up, etc. This was in Madhya Pradesh. And we said, “No, we don’t want to do this.” This is not social science.

SS: Aren’t those issues — airports, electricity — related to issues of economic development as well? I suppose that if an electricity company approaches you and says, “We want you to do this survey, so we know how to price our electricity.” That’s one thing. But how do you draw the line between what is social sciences and what is not?

SK: The prime concern is that when we do the survey, we also want to be clear about how we are going to make use of this data in the future. If it is for the client, just to do the survey and give them the data, is that the end? CSDS is not there to make money off of surveys. So we figure out if this data is going to be useful for us in our future research. Will this data be useful for other social science researchers?

When we thought of the electricity [study], we thought this would be a kind of client servicing. You take money, collect data, and give it to the client. Airports — it would make news in the newspaper for one day, but later on social science researchers won’t be able to make any use of the data. That’s the concern.

CSDS is primarily a social science research institute. This is not a survey agency. We have to be clear about that. We keep reminding ourselves that if a study has something to do with social sciences, society at large, people will make use of the data in the future, then we will get into that.

SS: How do you envision the role of CSDS vis-a-vis Indian society today? What is CSDS’ overall mission for itself, and how does Lokniti’s research fit into that mission?

SK: CSDS’ mission of research is very, very difficult to say. We have been debating for the last several years to figure out what is the core research of CSDS. We figured out that we can’t say, “No, this is the core of CSDS’ research.” We can only say the core of CSDS’ research is to do research in social sciences which would cut across several themes: media being one, urban being another, elections/democracy/politics being an important theme. Also now youth has become an important theme for the last five, ten years because as soon as anybody talks about a good study on youths in India, they would refer to CSDS. Political philosophy. Secularism. So there are six or seven themes which we can say are the themes on which CSDS does research. We won’t be able to say, “This is what CSDS research means.”

Lokniti’s position vis-a-vis CSDS — in CSDS, people not only do research on various themes, they also adopt different methods of doing research. Within that, Lokniti fits into one of the core themes of CSDS, because CSDS was established for doing research on Indian politics. Lokniti now is continuing that tradition of doing research on Indian politics. When CSDS was formed in 1963, the initial studies of politics [were] conducted through survey research: doing surveys in Kerala when the Kerala assembly elections were taking place, then immediately after that we had national elections. CSDS conducted surveys in the 1967 election [and] 1971 elections. Lokniti is trying to the take the CSDS tradition forward of doing studies on Indian politics using surveys as a method. That is why I would say Lokniti is core to CSDS’ functioning.

SS: CSDS is like a tray with different foods and the foods don’t really get mixed together. I’ve been here for more than six months now, and I’ve really gotten to know the Lokniti team well. But I don’t know anyone from Sarai. I don’t even know the other programs that are here at CSDS. Do the other teams interact with each other? Is there ever any talk of working together or does everyone just do their own initiative?

SK: People are always encouraged to do collaborative projects. But we can’t imagine a situation where twenty faculty, twenty-two faculty, will all be working together on one project. This happens only in small institutions, with less than ten faculty, then maybe a large number of faculty will be working together. But in an institution where you have a faculty of, say, twenty-five, you can’t imagine a situation where everybody would be working on the same theme.

But yes, we have forums where people are expected to interact with each other. All the programs make a presentation to the faculty about their work. Every month we have a faculty meeting, but that’s kind of administrative. We don’t discuss academic issues. Every Friday all the faculty members try and sit together to have lunch. That was designed in such a way that people can have exchanges about their work. We have also instituted a system of annual faculty seminars which a very intensive exchange of each other’s work and ideas.

So we have tried to have forums where faculty interact, know each other’s work, and also anybody is welcome to make a presentation whenever someone has conducted a study. Now we have also instituted a system of monthly faculty seminars, which began only last month. Every month one faculty member is going to present a detailed work. We have kept two hours for that. You present your work for an hour, then a one-hour discussion.

So there are platforms available, but I can’t think of a situation where everybody would be interacting on a daily basis with each other’s work. That kind of a situation is only in small institutions, because in small institutions sometimes the director assigns work. Suppose the institution gets three projects, the director assigns these three or four faculty to this project. In CSDS, we don’t have that system. We are very, very autonomous. But still you will find Lokniti working as a team, Sarai working as a team in which three or four faculty members are associated. Four faculty members are working on the Indian Languages program. Then there is a project on violence. Three colleagues are taking that project forward. And then you will find another five to seven faculty members who are not collectively working but they are working on their own projects.

SS: What do you think have been some of CSDS’ contributions towards understanding Indian society?

SK: If you look at the various kinds of studies which have been conducted by CSDS during the last fifty-three years, the kinds of issues on which we have touched upon — Rajeev Bhargava’s work on secularism; Sarai’s work on media and urban space; Hilal’s and Sanjeer’s work on marginality, Muslims, deprivation; Lokniti’s work on elections, youth, farmers — If you look at the range of issues which have been touched upon, the range of methods which have been adopted by various researchers, all this has been in the public domain.

We use four platforms. The media, I would say, print and electronic. Research journals. Books. And these days we also have three workshops which CSDS organizes every year. A workshop in which we try to train young researchers in data analysis; we have been doing that for the last ten years. Another course on researching the contemporary which is in the sixth year. And last year we started this new workshop only for students who belong to the marginal communities: SCs, STs, and Muslims. I think there is a lot of input to the social science community by way of publications and training workshops.

SS: Which topics do you think are in need of additional research here in India? Which topics do you think are underexamined?

SK: There is a need that in social sciences people should try to pick up the themes which have a direct connection to society. Not necessarily with policymaking, but directly with society. There is maybe a little more focus on the historical method of research. I would personally want people to be doing research more on current themes. There are issues [related to] youths. There is so much anxiety about farmers. It is an important issue. Minorities. Reservations is such a hot debate in this country. Secularism. And similar such issues. Issues on which the government is trying to formulate, there is a need to debate those issues. There is a need to formulate rules, or revise the already existing rules and regulations, which relate to the common man.

Also, there is a focus these days on India-China relations. But as you know, CSDS is not a place where we actually do foreign relations. Maybe some research on — not exactly on foreign relations, but something related to relationships with neighbors, which would fit into the big ambit of foreign relations.

SS: Thank you very much.


2 thoughts on “Sanjay Kumar on the challenges of survey research and CSDS’ role in Indian society

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s