The Rule of Law Index: Is It Really Impartial? A Twofold Multivariate I-Distance Approach

Introduction

Over the recent years, the interest of governments, donors, businesses, and civil society around the world in the quality of governance and the enforcement of the law and the rule of law increased. Therefore, ensuring the enforcement of the rule of law has become a unifying goal of all stakeholders (WJP, 2014). In order to accurately determine the actions needed to achieve their goal, a need for a clear, easily understandable, statistically sound measure of the rule of law emerged.

As statistics and composite indices developed, they were, slowly, but surely, introduced in the sphere of social sciences, especially law. Namely, governments acknowledged the benefits of statistical data and are now turning towards and relying on them. The main purpose of such data (indicators, indices, time series,...) is to provide the needed initiative for reform (Merry, 2009). Also, composite indices of governance have the ability to captivate public interest (Michener, 2015). They have been widely accepted by the general public as they reduce a multidimensional phenomenon to a single dimension and provide an easily comparable number or rank. Moreover, they tend to provide the objective truth, they facilitate comparison between the entities and most importantly, they quantify accomplishments (Merry, 2009). Therefore, statistical data and composite indices became increasingly central to global reform and global governance.

In the recent years a daunting task has been put in front of policy makers, law experts, and statisticians: to measure the rule of law. Providing such a metric is not a straightforward task for multiple reasons (Bergman, 2012). One of the main issues is the definition of the rule of law as the concept itself which is highly complex and controversial (Skaaning, 2009). Adriaan Bedner (2010) provided a research titled “An elementary approach to the Rule of Law” in which he attempted to provide a conceptual framework for defining the rule of law. The main conclusion of his thorough review is that the rule of law has two main tasks: to protect the citizens against the power of the state and to protect citizens’ property and lives from assaults by fellow citizens. Therefore, the rule of law aims to regulate not solely how the state treats its citizens, but also how citizens treat one another. Although the main ideas behind the concept are more or less clearly defined, its functions and elements remain unstandardized, unorganized, and difficult to envelop. Consequently, the different rule of law measurements produced dissimilar results (Møller & Skaaning, 2011). Therefore, debates sparked on the topic of the validity of the used data, indicator methodology, and later (mis)use of the measurement (Kaufman et al., 2007). Another, more general issue, is the fact that law, not just the rule of law, is difficult to measure in qualitative terms (Davis & Kruse, 2007). This issue draws the question of indicator choice and its ability to measure the desired phenomenon (Davis, 2004). Regardless their limitations, composite indices that measure the level of law implementation, rule of law, or results of policy reforms have not yet received the same amount of scrutiny as composite indices of sustainability, development, or university rankings (Bersch & Botero, 2014; Gisselquist, 2014). Up to lately, policy indices have not attracted enough attention from statisticians and methodologists, mostly because a large percent of their indicators are qualitative metrics that rely on expert opinion and judgment (Decancq & Lugo, 2013).