School Lines Pact with PaCT, or PaCT a PACT? Lester Flockton
feedback, feedforward, Feedup, feeddown lester.flockton@otago.ac.nz
National Standards were foisted (forced, if you prefer) to the matter by earmarking over $5 million for developing a upon New Zealand’s schools, teachers and children by a framework to ensure consistency of teachers’ judgments against Government that naively, yet steadfastly, believes this is the national standards. He announced: “This initiative will make sure way to overcome underachievement for a fifth or so of our that the data teachers use to assess students against the standards children (call them learners if you want; everyone is a learner is reliable and valid, so that it can be used across the education but only children are children). America has No Child Left system to make sure teachers’ judgments and the measurement Behind (and the disadvantaged are still behind). England has of student progress are nationally consistent.” Surely Mr English Key Stage Assessments with a detailed reporting bureaucracy must know by now, that no matter how much political spin is entangled throughout the system (and underachievement ladled out, nothing “makes sure”, particularly when it comes to continues apace among the disadvantaged). Australia has constructs of reliability and validity. The pretence, however, is NAPLAN, which is denting schools and important, especially if there are other teachers, but not underachievement Surely Mr English must agendas sitting in the saddle bags – such statistics. Yet until the Key government as performance pay. arrived on its high horse saddled up know by now, that no matter The long and the short of the “make with Bill English’s national standards sure” initiative is the Ministry’s creation ideas, we had no such all-consuming, how much political spin of a progress and consistency tool: time-robbing, resource-diverting, is ladled out, nothing PaCT. It is perhaps pleasing, therefore, ill-proven, technocratic industry. So that one of the major concerns with thanks to politically packaged half- “makes sure”, particularly national standards data management baked knowledge about determinants is being tackled, but this won’t properly of learning and achievement, and when it comes to constructs solve the problem until the wobbly outright dismissal and intolerance of reliability and validity. definitions and descriptions of the of any robust counter-evidence, we actual standards from which the data too have climbed aboard and are playing games like America, is derived are overhauled and re-written. But also bear in mind, England and Australia with non-negotiable rules pronounced none of all of this will solve the true problems underlying from on high and policed by the Minister’s Ministry. underachievement among disadvantaged or difficult children. New Zealand’s game plan, however, differs in one very According to the Ministry, PaCT is being developed to support significant aspect from others. Judgements of children’s progress teachers’ professional judgments in relation to the National and achievement are made not according to test results, but on Standards, and to improve the measurement of learner progress the basis of teacher overall professional judgements (which may over time. include non-mandatory test results). This, like it or not, believe The progress and consistency tool is intended to support it or not, is enlightenment – a bright spark in a darkly dug damp teachers’ overall judgments and assist with increasing the hole. But paradoxically, this bright spark ignited a volatile hue consistency of judgments across the country and across and cry that spread like wildfire. Early widespread recognition time periods. The tool is based on the same principles as that the standards were shamefully concocted, confusing, riddled the standards themselves, requiring a variety of evidence with curricular flaw (notably in reading and writing), and open of learner achievement as the basis for overall judgments, to widely varying interpretation, meant that variability in teacher and will enhance the ability to track learners’ progress in judgements from Kaitia to Bluff, from Room 1 to Room 10, from relation to the National Standards. It is intended that the child to child, would be commonplace. primary source of evidence informing judgments will be Protestations were ablaze about the predictable inconsistency teachers’ observations, with test information being used of teacher assessments in relation to the standards, and the as a mechanism for self-moderation. worthlessness of data manufacture so cherished by the Ministry of (http://assessment.tki.org.nz) Education. Clearly, Government was landed with a genuine crisis that was usurping its own invented crisis (that without national Some of us have had the opportunity to view and discuss standards, one in five children will continue to fail in our schools). the design of the tool with Ministry people involved in its But Governments have all sorts of devices and resources at development (as opposed to “sales”). Those I met with impressed their beck and call. So Bill English’s 2011 budget resolved to deal with their genuine desire to ensure the tool would be useful and
safe, but when it came to the hard questions they were obviously compromised. It was clear that answers could only come from decisions that would be made “higher up” in the bureaucratic and political chain of command. So what were some key observations from this viewing? I will restrict these to just a few of particular importance. PaCT has potential to be a useful tool for those schools that require support to attain a system that allows them to have confidence in the strength of reliability (accuracy) of their judgments of children’s achievement and progress on National Standards. ■■ Critical determinants of consistency of judgments both within schools across schools nationally are the clarity, validity, and manageability of the criteria against which judgments are made. If these determinants are not properly accounted for, then regardless of the technical design and operation of PaCT, reliability of the data is likely to be fragile. There should be concerns that current literacy (reading and writing) criteria do not sufficiently meet the standards required of the critical determinants. Their clarity is compromised by too much specialist jargon, and the volume of ‘aspects’ and ‘criteria’ is likely to have unwanted side effects because teachers are faced with having to handle far too much unnecessary detail. These weaknesses need to be addressed in a co-constructive approach that values and draws more substantially on practitioner knowledge and experience. The systemic misalignment and confusions between curriculum levels and year-by-year standards (progressions) is a matter that is badly in need of being resolved. ■■ The Ministry is having its own internal ‘conversations’ about whether PaCT should be mandated or not (this is a control issue). We know from much experience that regulation for ■■
compliance is a device that undermines trust in professional responsibility, and leads to a mentality of expedience – factors that would significantly jeopardise the resulting data produced from PaCT. Yes, the tool should be available to those schools that would find it helpful, but making it compulsory for every school does not make good sense. ■■ It is not clear how PaCT would sit in the digital conduit between schools and the Ministry, and whether the Ministry would want a set-up that allows it to access any school’s data reports from PaCT. If this were to be the case, then there must be major concerns about the safety, use and misuse. Experience is telling us that grounds for confidence in this regard are very shaky indeed, and that protocols are no guarantee in an environment of shifting governmental motives and decrees. The current National Administration Guidelines specify the National Standards data that schools must report. PaCT should be viewed as a tool to assist schools with producing that data, not as a tool for further and deeper level of data reporting to the Ministry. That is, the tool should be for in-school use only. Researchers and advisers wanting to access detailed data for their various purposes should follow standard ethics and research protocols, rather than having open access to a centralised data warehouse that is a depositary for every school’s minutia.
Only once the full design, procedures, and uses of PaCT are officially revealed will it be possible to decide whether it is professionally advisable to enter into a PACT with PaCT. If the tool, which we are told is to support teachers with judgment making, is also to be used for more and more funnelling of data to the Ministry of Education, then it is likely to become yet another PACT: Performance Accountability Control Tool.
Equipping the Performing Arts in Schools Nationwide
Lighting
Makeup
Scenic Products
Drapes
Specialist Supplier to the Education Sector Huge Catalogue Online: www.adena.co.nz
TV/Video
Sound
Adena Limited PO Box 756, Hamilton Email: sales@adena.co.nz Free Phone: 0800 782-438