Assessment Design and Presidential Palaces in Kazakhstan

December 13, 2012

David Williamson Shaffer visited Kazakhstan in September to deliver the keynote address at the International Association for Educational Assessment (IAEA) 2012 conference.  His address focused on the future of testing and assessment based on games, rather than multiple-choice questions or essays, an idea that he said the crowd was “excited about overall.”

Shaffer directs the UW-Madison Epistemic Games project, which designs and builds epistemic games and innovative assessment tools. Epistemic games are designed to help players learn to think like engineers, urban planners, journalists, lawyers, and other innovative professionals, giving them the tools they need for a changing world.

The IAEA conference brings together people from educational agencies from around the world to assist in the “development and appropriate application of educational assessment techniques to improve the quality of education.”

Many of the attendees were excited about the idea of using games to test 21st century thinking, Shaffer says. But because the IAEA is composed of organizations responsible for national testing, they were interested in when they would be able to deploy such tests at scale.

This was the first time Shaffer had visited an ex-Soviet republic, so he didn't quite know what to expect. The plane arrived at about 2 in the morning, after almost 24 hours of travel, so his first impressions were mostly ‘surreal.’  Fortunately there were many other delegates to the conference arriving in the same condition, ‘so we were a friendly bunch,’ he says.

The conference was very pragmatic, he says, with a lot of focus on how to deal with cheating on computer-based tests. Shaffer was struck by the fact that, in most countries, the national tests are run by the government, whereas in the U.S. they are in the hands of private companies.

One of the hot topics was the problem of cheating on tests when they are being conducted digitally. Many attendees said (at least unofficially) that there is no way to completely protect a test from cheating once it is in electronic form. Shaffer points out that this was true with paper tests too, but now cheating is easier.

Detecting cheating has become ‘a kind of arms-race,’ much like attempts to stamp out computer viruses or spam, he says. The real solution is to move to tests that capture complex thinking, which is harder to pass to someone else than a string of multiple choice answers.

Shaffer likes to take trips like this for three reasons: One is to help spread new ideas about what researchers and practitioners can and should do in education. Another is to meet interesting people with whom he might collaborate in the future. And he hopes to find at least one good new idea to make the trip worthwhile. In this case, he accomplished all three.

Shaffer had one short day to see Astana, the capital city. He says the National Assembly and Presidential Palace were about as impressive as one might expect, given all Kazakhstan’s oil money. He says he tried horse meat and horse milk in a local restaurant and, for what it’s worth, he does not recommend the latter.