A step toward increased accountability? | Researchers: Luís Catela Nunes, Ana Balcão Reis, Carmo Seabra
Rankings and Accountability in Secondary Education
During the last decade many countries have introduced some form of school accountability in an attempt to increase their performance. Nova SBE’s Professors Luís Catela Nunes, Ana Balcão Reis and Carmo Seabra analyzed the impacts of the public disclosure of school rankings, a low-stakes accountability policy.
Their study considered the case of Portugal, where the publication of school rankings occurred for the first time in 2001. Since then, the publication of rankings has drawn a strong interest in Portuguese society, the results appearing on the front pages of newspapers, opening headlines on the main TV channels, and triggering several debates in the media.
The researchers studied the effect of the publication of the rankings in terms of the ability of schools to attract students and of the increased probability of closing for schools that are rated poorly.
The institutional setting
To finish academic secondary school in Portugal, students must take exams at the national level. These national exams perform two roles in the educational system. Besides being a requirement for graduation from secondary school, they also determine the conditions for admission to universities. The published rankings of every high school are based on the average scores obtained by students on these national exams.
Public and private schools co-exist in the Portuguese educational system. Public schools are tightly restricted in their educational supply, and neither choose nor influence the choice of teachers. Private schools have more flexibility than public ones in that they can choose teachers and students, and set tuition fees. This asymmetry in school autonomy determines the capacity of schools to react to a changing environment.
The publication of the rankings had a clear impact
The authors’ main finding is that the publication of rankings has clear effects upon families and schools in Portugal. After publication, the number of students attending schools that are rated poorly decreases and the probability of closure of these schools increases.
Different mechanisms may lead to these developments: students will choose better ranked schools, or, if these are not available, may decide to change to vocational tracks or even drop out.
The researchers also found that the effect for private schools is stronger, as would be expected given that the freedom of choice between private schools is greater than between public ones.
What do the rankings really measure?
School quality measures such as school rankings may be very imprecise measures of school quality:
(i) rankings are often too volatile;
(ii) rankings reflect factors that are outside the control of schools, especially the socioeconomic background of students’ families;
(iii) public disclosure biases schools’ and teachers’ efforts toward an excessive investment in test-specific skills that are easily observable, at the cost of other school outputs, not so easily measurable;
(iv) accountability systems can be “cheated on” in several ways with no real content as far as effective quality evaluation is concerned.
Do we need different rankings?
The public disclosure of bad outcomes might have perverse consequences for equity due to cream-skimming of both students and teachers, or to motivational effects. The position emphasizing the possible negative effects of the publication of school rankings is shared by a number of politicians, and explains the calls for the prohibition of public disclosure of school rankings that is seen in Spain and some other European countries.
Luis Catela Nunes comments “We do not discuss the validity of the ranking as a measure of school quality. A better ranking does not necessarily mean that a school is better in the sense of having a higher value added; it may just have better students. What we show is that there are effects of the publication of school rankings regardless of their informational content. “
The results of this study reinforce the need to verify the type and quality of the information implicit in those rankings. Further, the disclosure of this information is independent of any government intervention. Therefore, the issue of the existence of potential benefits stemming from the public disclosure of complementary information gains importance.
This article is based on the paper "The Publication of School Rankings: A Step toward Increased Accountability?" authored by Luis Catela Nunes, Ana Balcão Reis and Maria do Carmo Seabra, and published in Economics of Education Review, issue 49, pages 15-23 in 2015.