While teachers embrace creativity as an essential 21st century skill, a lack of valid and reliable creativity tests means schools struggle to assess student achievement.
The Test of Creative Thinking – Drawing Production (TCT-DP) has long been acknowledged as the premier tool to assess creativity in school aged children, but as it is expensive, slow, and labour-intensive, it’s out of reach for most schools.
However, a new and ground-breaking machine-learning model promises a timely and cost-effective solution.
A new algorithm, developed by the University of South Australia (UniSA), providing teachers with access to high-quality, fit-for-purpose creativity tests, that can score assessments in a fraction of the time and a fraction of the cost.
Applied to the current empirical creativity test – Test of Creative Thinking – Drawing Production (TCT-DP) – the new algorithm marks a test in a single millisecond, as opposed to the standard 15-minute human-marked test.
‘A game-changing innovation for schools’
Lead researcher, UniSA’s Professor David Cropley said the algorithm presents “a game changing innovation for schools” as it could save teachers thousands of hours in an already overloaded schedule.
The UniSA research team used the algorithm to analyse data TCT-DP data that co-researcher, Dr Rebecca Marrone, collected from a high school in Adelaide as part of another project.
“One of the reasons she collected the TCT-DP data was in anticipation that we would have the algorithm available,” Professor Cropley told The Educator.
“What would have cost around $22,500 and taken 12 person-weeks [450 hours] to score ended up taking around 12 hours and costing only $600.”
More recently, Professor Cropley has been working with a school in the Adelaide Hills, where teacher is using the TCT-DP test as part of a design class with groups of around 20 students. He says now there is a negligible time/cost, students can now be tested at the start of a module, carry out a project and then test again at a later point.
“Critical here is that she can give students timely feedback about their creativity,” Professor Cropley said. “Finally, I have also used the algorithm to score about 100 tests provided to me by a colleague in Holland. This took 1 hour to score, instead of about 50 hours.”
Professor Cropley said that in each case, the “radical” saving of time and cost makes it possible to use this high-quality creativity test.
“Without the algorithm, these activities simply wouldn’t be practical.”
Stand-alone desktop version coming
Professor Cropley says in the long run, it should be possible to integrate the algorithm with existing school management systems, making it more widely accessible to educators across Australia.
“Currently, the algorithm runs on a cloud platform that only we have access to. However, we are developing a stand-alone desktop version that teachers could install on their own machines, or as part of a school system,” he said.
“When that is available, teachers will be able to administer the test and score it themselves. Currently, if teachers contacted us wishing to use the algorithm to score the TCT-DP, they could send us the scanned test forms, and for a nominal cost of $2 per student [mainly to cover the cloud computer time] we can score the test and return the results within hours.”
When asked what kind of training or support will be available for teachers using the algorithm for the first time, Professor Cropley said this is “an important next step.”
“We are able to provide training workshops for teachers to show them how to interpret the results provided by the algorithm,” he said.
“This training can also be tailored to the particular needs of schools and education departments. Another pathway for schools/departments is for us to provide this kind of diagnostic feedback as a service.”
Revolutionising student assessment
Professor Cropley said he and his team of researchers at UniSA envisage that, as the importance of creativity as a 21st century skill grows, schools will build this into their assessment and tracking systems.
“Schools will likely track general student creativity at several intervals over, for example, students’ high school careers,” he said. “Additionally, schools are also likely to use this to examine the impact of more specific activities, such as a workshop or module focusing on creativity.”
Professor Cropley said there is ample evidence that the ability to think creatively plays a role in traditional academic outcomes, including things like ATAR scores and GPAs, adding that schools, increasingly will want to be able to track and report on this ability.
Another important benefit of the algorithm, says Professor Cropley, is that it scales in a very linear fashion.
“The test itself only takes students about 10-15 minutes to complete. Assuming that we were processing the tests on the cloud platform, it is feasible that an entire school of say, 2000 students, in classes of 25 could be tested in a single lesson, in their individual classes,” he said.
“If each individual teacher scanned and emailed us their class test forms, that could comfortably be done within that same first lesson.”
Professor Cropley said the UniSA researchers would then receive 80 emails (each with 25 scanned forms) which they would then be combined into a single file. The images would then be cropped and formatted as a single batch, ready for analysis – a process that takes about an hour.
“Currently, we can upload and score these at the rate of about 400 per hour. We would then compile a short report of the results [assuming we don’t do any detailed diagnosis]. Therefore, we could have the results back at the school by the close of business on the same day,” he said.
“In other words, the main effort is in the actual uploading and scoring, which scales in linear fashion.”