I will be flying back to Chicago tonight so am frantically packing my things. But in light of the busyness of this last week, I wanted to wrap up this blog with a post that reviews some of what I found during the last few months and thanks everyone who helped along the way.
I flew to India more than ten months ago under the impression that my research would be primarily focused on sampling methods. This is in part because over the previous three years I spent a lot of time sitting in meetings and holding discussions about the challenges of acquiring representative samples, whether sample demographics matched the population’s demographics, how to improve sampling methods, etc.
What became clear to me over the first few weeks and months of my research is that this is a problem that many people have already thought extensively about. When it comes to making sure the demographics of one’s sample matches those of the population one is sampling, survey research organizations have some tried and true methods of trying to reach those harder-to-reach populations. They have good data on the demographics of the population they are working with, with the noteworthy exception of caste. When sample demographics don’t match those of the population, as will happen inevitably with some variables, they weight the data.
Furthermore, one can make generalizations about the challenges of sampling a particular population only insofar as that population has some amount of uniformity. The challenge of speaking about India is that it is a teeming multitude of populations. With 1.3 billion people spread out across 29 states and 7 union territories speaking 22 official languages (and hundreds of unofficial ones), India is more than just a country. It is a subcontinent that is home to one sixth of humanity, the most radical democratic experiment in history.
So the notion that there are certain challenges to polling India is useful only up until the point where one comes into contact with India’s variegated realities. I had the good fortune of coming into contact with just a few of these realities in my ten months here: different neighborhoods in downtown Patna, a Bodo village on the outskirts of Guwahati, a working class neighborhood in west Delhi, a mid-sized town and its surrounding villages in Tamil Nadu, a Muslim village on the outskirts of a major city in West Bengal, a comfortable suburb on the outskirts of Thiruvananthapuram, among others. These environments all had their own unique challenges for the field researchers who were sent to question designated respondents. Their knowledge of that environment and ability to navigate it proved critical in their ability to complete interviews successfully.
Here are some examples of how field researchers dealt with (or were unable to deal with) such challenges:
1. Gender
In both West Guwahati assembly constituency of Assam and Bolpur constitutency of West Bengal, female respondents were rarely interviewed in isolation. Their husbands or fathers frequently observed the interviews and would often start answering the questions for their wives/daughters. Field investigators would try to direct questions back to the respondent, but the respondent would often defer towards her husband or father, especially for questions related to abstract issues, such as ideology, or current affairs.
2. Caste
In the villages around Usilampatti, where intercaste violence between backwards castes and Dalits has been frequent, field investigators must know which villages have been flashpoints of such violence. In such villages, it is not safe for field researchers to even ask about the caste of respondents, as it may threaten their safety to raise the topic. Caste is a standard demographic variable included in all surveys, and so field investigators need to exercise a great deal of judgment about when and where to ask about it.
3. Language / ethnicity
Visiting a village on the outskirts of Guwahati that is home to Bodos and Rabhas, field investigators who spoke Assamese and Bengali were only able to interview the residents of the village who spoke Assamese. For the remainder of the village’s residents who spoke only Bodo and Rabha, the investigators were not able to complete interviews. Most of the selected respondents from the village could thus not be interviewed. Field investigators did not feel safe in the village and continued to the next polling station in the constituency.
4. Religion
In Ruppur, a village around Bolpur in West Bengal, residents in the Muslim section of the village expressed a great deal of hesitation and skepticism about participating in the survey. “Rajniti [politics],” one resident said, with disdain. Field investigators had to diligently explain that the surveys they were conducting were confidential and not for any political party or group. After the first few interviews, village residents became more comfortable with the interviews. This initial hesitation and skepticism was not encountered in the Hindu section of the village.
This experience was very different from field researchers conducting interviews in the Muslim section of Palikkal, a village about 50 km from Kerala’s capital, Thiruvananthapuram. Though many residents were not home, those who were welcomed the field investigators to the house and did not show much initial hesitance to answering questions about the upcoming elections.
5. Urban and rural settings
In neighborhoods of cities and larger towns, whether they were Delhi, Guwahati, Bolpur, or Nemom, many selected respondents were not home at the time of the field investigators’ visit because they were at their jobs in different parts of the city. If someone else was at the home, field investigators would try to set up a time to return and complete the interview. But the limited amount of time they had — somewhere between two to four days to complete all their interviews — meant that field investigators often had to make judgment calls about whether or not to return to the household and re-attempt the interview.
Furthermore, navigating urban neighborhoods to find the households of selected respondents nearly always proved to be challenging. Field investigators would ask other respondents, shopkeepers, and neighborhood residents about the addresses and names of those who were selected for interview, but people often did not know so well the names of all people in their neighborhood. This was especially the case in Bolpur and Nemom; in Usilampatti, respondents were more likely to identify names and addresses in their neighborhood. Field investigators had to make a judgment about how much effort to expend locating households in one particular neighborhood before moving onto the next neighborhood.
In small villages, such as Ruppur or the Bodo village on the outskirts of Guwahati, respondents were much more likely to know the names and locations of selected respondents. In the case of Ruppur, a group of young men led the field investigators around the village to each listed respondent. On the other hand, in Palikkal (also a small village) field investigators struggled to complete many interviews since village residents were less familiar with the names and addresses of selected respondents in the area.
Transportation to villages often proved a logistical obstacle to completing more interviews. An expensive rickshaw ride was required to go from Bolpur to Ruppur, for example (the rickshaw driver looked with puzzlement at the field investigators when they said this was where they wanted to go). Work had to be cut short earlier in the afternoon in order to catch the last bus back to Bolpur. Field investigators had to be familiar with bus schedules to ensure that they could arrive in the village and return to their home or hostel before dark.
6. Class
In one village outside of Usilampatti, many of the village’s residents were unemployed agricultural workers who earned a living from the MGNREGA scheme. Field investigators went to the field where the villagers were working and started completing interviews, but were disrupted by a panchayat (local government) clerk who told the villagers not to answer any questions. One field investigator talked to the clerk, explaining why the survey was being conducted and presenting him with the official letter from CSDS, but the clerk insisted that the field investigators leave.
Many of the selected respondents in villages around Usilampatti had left to work in larger towns and cities around Tamil Nadu. In Nemom, it was also the case that several selected respondents had left their homes and were presently working jobs in the Middle East. Field investigators were not able to interview any of these selected respondents.
7. Education
Respondents with lower levels of education often struggled to comprehend many of the questions in the questionnaire, especially those related to current events. However, some could hardly understand any of the questions beyond which party they had supported or would be supporting in the election. Some respondents in Bolpur and Ruppur who were supporters of Trinamool Congress merely answered, “Trinamool,” for all questions, even those for which “Trinamool” was not an option or for which the response “Trinamool” would convey something negative about the party. Field investigators had to make a judgment about whether respondents really meant to convey a negative impression about Trinamool or simply did not understand the question. This was a challenge encountered by field investigators working in the villages around Usilampatti as well. Many of the selected respondents simply did not understand the content of questions. One old woman asked, “Who is Modi?” for one of the questions about the central government’s welfare schemes. Field investigators therefore had to make decisions about whether it made sense to ask all questions in the questionnaire, when it was clear from the initial section of the interview that the respondent hardly understood the content of questions.
As has become clear from interviews with researchers, there are national level trends when it comes to challenges of survey research in India, and the practitioners of surveys who have been conducting them for decades (VB Singh, Yogendra Yadav, Sanjay Kumar, Yashwant Deshmukh, and Dhananjai Joshi) are well familiar with them: lower response rates among Dalits, Muslims, and tribal populations; challenges in reaching female respondents, particularly Muslim female respondents; and lower response rates in urban neighborhoods. As Yogendra Yadav succinctly explained:
Random sample with a reasonably large size gets you everything. But anything other than random sample almost invariably oversamples again the more articulate, the more well-to-do, men, urban. The slope of privileges is such that sampling tends to flow in the direction of socioeconomically more powerful groups: rich, men, urban, upper caste, media-exposed, educated. And since all of these, except the gender, has a significant correlation with each other upon another, there is a very substantial overrepresentation of one section of society.
My point in explicating the examples above is to illustrate how the challenges of conducting survey research in India are not limited to these general trends. They are the beginning rather than the end of obstacles faced by those conducting surveys in India (I did not even touch on the all-important issue of funding, nor the issue of pressures that comes from media and other clients), and the examples above hopefully illustrate how obstacles are highly localized and unique to each and every environment. In order to navigate them, field investigators must be conscious of the challenges in their assigned environments and capable of navigating them competently. Retention of experienced field staff who are passionate about the work they are doing is therefore a must.
In light of the highly localized challenges of conducting survey research in India, researchers might want to think about how the design of questionnaires can be done in a way that integrates local knowledge more extensively. I don’t mean that questionnaires should be put to a vote and of course understand the precious value of real estate on questionnaires. But perhaps the mechanisms by which local knowledge flows up to those designing the questionnaire at the highest levels can be re-examined. This might be especially helpful in the case of states that tend to be more outside the expertise of north Indian researchers. From my viewpoint in Delhi, where all of my three research affiliates were based, it was the South and Northeast where more inputs from people in the states might have expanded our ability to understand the results of the Assam, Kerala, and Tamil Nadu elections. The questions we ask are, after all, the ones that define our analysis.
This research would not have been possible without the support, encouragement, and contributions of so many people.
From my old employer D3 Systems, thanks to David Jodice, Matthew Warshaw, Sandra Newman, and David Peng for their encouragement towards pursuing this project and their recommendations. I am also grateful to my former professor and thesis supervisor, Samer Shehata, for his recommendation.
From the United States-India Educational Foundation, thanks to Adam Grotsky, Neeraj Goswami, Priyanjana Ghosh, Pavitra Soram, and Kalden Shringla for their guidance and support.
Thank you to everyone who sat down for an interview: Rajdeep Sardesai, Rukmini S, Dhananjai Joshi, Rajeeva Karandikar, Yashwant Deshmukh, Praveen Rai, Yogendra Yadav, VB Singh, Sanjay Kumar, and Mukulika Banerjee.
From Impetus Research, thanks to Ankur Aggarwal, Raghavendra Srivastava, Anil Kumar Jha, Rajni Singh, and Krishna Shahi.
From CVoter, thank you very much to Shalinder Mahjan, Shaleja, SK, Abdul Menon, Ajit Shukla, and everyone else at the Noida office. Thanks especially to Yashwant Deshmukh, who was generous with his time and allowed me to peer inside the guts of his organization in a way that commercial pollsters rarely allow.
From the CSDS-Lokniti network around the country, thanks to Dhruba Pratim Sharma, Meenakshi, Vashwati Das, Jyoti Prasad Chatterjee, Suprio Basu, Biswajit Prasad, Shaw Chandan, KM Sajad Ibrahim, Sandeep Shastri, Rajeena Aysha, Anuja, Arya, Sukanya, P. Rama Jayam, Ranith, Thanalakshmi, Pechi, and Kajinder for help with logistics and letting me follow your work.
Most of all, thank you to everyone at the Lokniti office in Delhi: Asmita Aasaavari, Dhananjay Kumar Singh, Vibha Attri, Jyoti Mishra, Shreyas Sardesai, Nitin Mehta, Arushi Gupta, Himanshu Bhattacharya, Pranav Gupta, Rahul Verma, and of course Sanjay Kumar. These people are truly at the forefront of thinking about Indian politics, and I have learned more from them in ten months than I ever could have expected.
Thank you, reader, for taking an interest in this research.
I will be back. Until next time, this is Polling One Billion, signing off.
Lokniti-CSDS
Team CVoter
Polling One Billion 