Methodological inaccuracies & biases adopted by V-Dem apart from experts enlisted to evaluate Bharat’s democratic credentials is questionable
Pummy Pandita
The encroachment of selective and biased methodologies into the arena of international rankings and surveys has raised serious questions about their integrity. Far from being an anomaly, this issue permeates a broad spectrum of indices. Among them, the Varieties of Democracy (V-Dem) Institute’s eighth annual report on democracy, entitled “Democracy Winning and Losing at the Ballot.” Purportedly known for its deep dive into state of democratic governance around the globe, this report seeks to provide an exploration into complex dynamics of democracy in various nations, with India receiving particular emphasis. India, the world’s largest democracy, stands at the forefront of discussions on democratic health and vitality.
The nation’s democratic journey is distinguished by its commitment to regular electoral engagement, a robust multiparty system and a steadfast embrace of diversity and pluralism. Yet, V-Dem’s labeling of India as “one of the worst autocratisers” casts a long shadow, provoking thorough scrutiny of the methodologies and criteria V-Dem employs. Such a characterization not only stirs discussion but demands a more nuanced understanding of the factors influencing these assessments. As observers and analysts scrutinize these rankings, the debate underscores need for transparency and rigour in evaluative methodologies employed by V-Dem and other indices around the world.
Alarmingly, implications of such rankings extend far beyond mere numbers; they significantly influence a nation’s access to resources, financial implications, opportunities, and its global reputation. Consequently, when integrity of these methodologies is called into question, it can precipitate disagreements, escalate into diplomatic strains, or necessitate shifts in policy. Varieties of Democracy Institute, anchored at University of Gothenburg in Sweden, stands as the source behind V-DEM rankings. This report includes a suite of indices such as Liberal Democracy Index, Electoral Democracy Index, Liberal Component Index, Egalitarian Component Index, Participatory Component Index, and Deliberative Component Index, each contributing to a understanding of democratic health and governance globally.
The methodology and approach adopted by V-Dem Institute have raised considerable concerns particularly in their evaluation of India’s democratic credentials. A detailed scrutiny of indices and sub-indices utilised by V-Dem reveals a mixed picture: India scores well on objective measures such as the proportion of the population with voting rights and the percentage of direct popular votes. However, a noticeable decline is observed in areas heavily reliant on ‘expert opinion.’ V-Dem’s reliance on “innovative methods for aggregating expert judgments” to derive “valid and reliable estimates” for concepts that are inherently challenging to observe is a point of contention. The field of social science research is well-acquainted with the biases and limitations inherent to such methodologies. Despite this, V-Dem’s acknowledgment of potential biases in its operations appears cursory at best, quickly passing the buck to ‘experts’ and claiming to mitigate these biases through a so-called ‘measurement model’.
This approach raises questions about the institute’s commitment to academic rigour, as it seems to prioritize self-promotion over methodological integrity. The transparency and accountability of V-Dem’s evaluation process are also under scrutiny. The institute reportedly relies on around roughly 25 “Country Experts” across five categories to assess each country, with the identities of most of these experts remaining concealed. This small group of experts is tasked with making judgments on the democratic status of nations, a stark contrast to the democratic ethos upon which countries like India are built. India, for its part, has established a democratic system that allows its citizens to shape their destiny through participatory elections, rather than deferring to the opinions of a select few.
Moreover, V-Dem’s approach to updating its methodologies and assumptions appears uninspiring. While it claims to regularly review its methods, actual adjustments are made only “occasionally.” Criticisms from countries in the Global South, which highlight the biases and ideological leanings inherent in V-Dem’s methodologies, have been persistent. These critiques often point out the alignment of such evaluations with the interests of influential figures and the lack of significant efforts by V-Dem to address or amend its flawed methods. This ongoing resistance highlights not only concerns about the transparency and accuracy of V-Dem’s methodology but also about the competence and intentions behind the reports it publishes. The portrayal of democratic performance in V-Dem’s reports also seems to echo a familiar narrative found in Western literature and analysis, where Western nations—primarily the United States, Canada, Europe, and Australia—are consistently depicted as outperforming the rest of the world. This narrative perpetuates a simplistic and often misleading view of global democratic landscapes, further complicating the trust and validity of such assessments. The persistent reluctance of V-Dem to revisit and revise its evaluative processes suggests a deeper problem than mere methodological transparency. It hints at a fundamental disconnect between the institute’s proclaimed objectives and its operational ethos. This steadfast adherence to a disputed methodology, in the face of widespread critique, underscores a concerning lack of competence or, more troublingly, suggests potential ill-intentions behind the production of these reports.
In conclusion, the approach of the Varieties of Democracy (V-Dem) Institute towards evaluating global democracies, with a notable focus on India, underscores a deeply troubling confluence of methodological shortcomings and apparent biases. While V-Dem positions itself as a standard-bearer of academic integrity, leveraging innovative methods and expert judgments, its persistent neglect in addressing and amending the inherent biases within its methodologies significantly detracts from the trustworthiness of its evaluations. This steadfast stance not only erodes the credibility of its reports but also casts doubt on the objectivity and underlying motives of the institute. Particularly glaring is V-Dem’s indifference towards engaging with the sustained critiques from countries across the Global South, which points to a broader disregard for the multifaceted expressions of democratic governance beyond the Western paradigm. The institute’s assessments of India bring these issues into sharp relief, highlighting a potential bias and even anti-India sentiment that seems to overshadow the nation’s democratic achievements and complexities. India’s democratic journey, marked by its vast electoral processes, dynamic multiparty engagements, and commitment to pluralism, stands in stark contrast to the critical lens through which V-Dem views it. The methodology’s heavy reliance on expert opinions, coupled with a lack of transparency about these experts’ selection and the institute’s resistance to methodology revision, fuels concerns over the evaluations’ impartiality and relevance. Furthermore, the narrative of Western democratic superiority that pervades V-Dem’s reports not only oversimplifies the global democratic landscape but also risks undermining the significant strides made by countries like India in navigating the challenges of democratic governance. This skewed portrayal necessitates a critical reexamination of the criteria and processes used in such democratic evaluations, advocating for a more nuanced, transparent, and inclusive approach that genuinely reflects the world’s diverse democratic practices without succumbing to narrow, biased perspectives.
(Author is head of operations at Centre for Integrated and Holistic Studies, a non-partisan think tank based in New Delhi)
CIHS - Centre for Integrated and Holistic Studies 