However, some principals and academics say that the majority of school surveys are “statistically insignificant”, pointing to small sample sizes and questionable methodology.
Berwick Lodge Primary School principal, Henry Grossek, says that as such, surveys analysing staff attitude, student voice and parents’ perceptions are, for the most part, flawed.
“There are serious questions about the nature of student attitude surveys and the questions. For example, are children being groomed in some of these surveys?” Grossek told The Educator.
“There are question marks about these surveys being self-serving or inappropriate in that the kids wouldn’t understand some of the questions.”
Grossek said many children could be responding “through a set of eyes that don’t necessarily reflect the intent of the question”.“But at the end of the day, they become part of every school’s annual reporting, so they’re given credibility that may be questionable,” he said.
Lead researcher of the annual Australian Principal Health and Well-being Survey, Professor Phil Riley, told The Educator that he and his researchers are always careful to make sure the [health and well-being] survey is “methodologically robust”.
Grossek said that the credibility of Professor Riley’s surveys comes from the fact that they use large sample sizes and are comprehensive in scope and methodology.
“The Victorian Department’s latest principal well-being strategy doesn’t take into account that a comprehensive survey to address all of the issues has already been done and we don’t need to have another survey,” he said.
“In essence, the strategy is but another starting point, dating all the way back to the DET’s own ‘The Privilege & the Price Report’ from 2004 and Professor Riley’s surveys. Yet another survey does nothing to defuse cynicism amongst principals.”
Grossek said that as for parent opinion surveys, schools might survey 120 families but get only a small handful of responses back.
“What’s the credibility of a survey like that? Virtually nil,” he said.
“State-wide samples that we’re benchmarked against are similarly small, so you’ve got high probabilities of statistical insignificance.”Grossek said one of the problems principals have with staff opinion surveys is that the Department has “narrowed the focus” in terms of teaching and learning.
“Also, to what extent is the feedback about school leadership versus Department leadership? That’s hard to untangle when we’re implementing government policy,” he said.
Blaise Joseph, an education policy analyst from the Centre for Independent Studies (CIS) told The Educator that while surveys can generate useful information, they cannot be used to generalise.
“Surveys of teachers or students do not necessarily generate standardised responses or objective evidence, and it is important that the results of surveys are qualified accordingly,” he said.
Joseph said that for surveys to be helpful in determining the general experiences of people involved in a school or program, they need to be taken from a large random sample and generate a high response rate.
“Surveys certainly do not form part of the gold standard of the evidence hierarchy,” he said.
“They are no replacement for randomised control trials, for example. In response to important findings from surveys, there should ideally be more rigorous research done on the subject, before jumping to conclusions.”
Related stories:
Private schools surveyed on students’ mental health
Govt’s principal health strategy ‘ignores key findings’