John Nesbit, Phil Winne, and Carmen Gress

John Nesbit, Phil Winne, and Carmen Gress point out that school ratings such as the Fraser Institute’s inevitably have some error built into them.

The trio points out that school ratings such as the Fraser Institute’s inevitably have some error built into them. “As a result, any particular school’s rating is somewhat unreliable,” says Winne. He and his colleagues used basic psychometric methods to demonstrate that “the odds are 19 to 1 that any particular elementary school’s rating could be about 1.1 points lower or higher than what was published in the Fraser Institute’s report.”

Using straightforward statistics, Winne and his team also show that a student with an average score on grade 12 provincial exams might receive a grade from their teacher as high as A or as low as F in English 12. The Fraser Institute’s ratings of secondary schools are largely based on provincial exam scores. “Teachers can observe a student’s work over a much broader range of the curriculum than a one-shot test. It’s important not to over-interpret what a single test score can reveal,” suggests Winne.

“We aren’t claiming ratings like these don’t have uses,” Winne observes. “What we urge is not to overlook the particular limitations of any kind of school rating.”

The SFU researchers offer several practical reminders to parents: “Don’t rely on a single indicator when judging a school — a variety of factors contribute to a successful learning experience. And, bear in mind that every measure has some drawbacks.”

The data used in the analysis was obtained through a research agreement with the BC Ministry of Education.

Click here to see the complete report "Cautions about rating BC's schools" or contact: Ruby Ng, Associate Director of Communications, Faculty of Education, 604.268.7152, rubyng@sfu.ca

Related content:

More information on the school rankings issue can be found on our school rankings resource page