Formative Evaluation for Leadership

May 13, 2013

The No Child Left Behind initiative focused research attention on schools that succeeded in closing achievement gaps and raising achievement for all students.

When WCER researchers Rich Halverson, Carolyn Kelley, and Eric Camburn look at these schools, they see the importance of school leadership in promoting student learning.

Their 2011 study of how principals are evaluated in Wisconsin found that the evaluation systems are defined and administered inconsistently across districts. None of the systems displayed the power to promote changes in practice. The study also found that half of the principal evaluation systems in Wisconsin were more than 10 years old. Few school districts had defined what it means to be an effective principal. Where definitions did exist, they did not align with the evaluation system. In most districts, evaluators designed their own systems. Comprehensive policies, procedures, or guidelines for principal evaluation did not exist.

A high quality formative evaluation system should link the principles of effective evaluation to a clear definition of effective leadership. Results should then inform principal professional development.

Tasks assessed in the CALL survey were identified through research on successful schools. The survey defines leadership practice as composed of five macro tasks and 115 micro tasks. Macro tasks refer to general organizational duties, such as providing adequate resources and designing professional development. Micro tasks articulate these general responsibilities into the day-to-day activities of school leaders.

With that guiding concept, Kelley and colleagues designed a new formative school leadership assessment and feedback system. Their Comprehensive Assessment of Leadership for Learning (CALL) assesses specific leadership practices and tasks characteristic of high-performing middle and high schools. 

The CALL online survey captures leadership practices and school cultures across five domains:

• Focus on learning,
• Monitoring teaching and learning,
• Building nested learning communities,
• Acquiring and allocating resources, and,
• Maintaining a safe and effective learning environment.

The survey captures leadership practices rather than opinions about leadership. That’s because feedback should motivate and direct improvements in performance. It should focus on the tasks and task performance, not on the individual person or his or her self-concept. 
The CALL assessment measures the presence of formal and informal leadership practices—distributed throughout the school—that promote student learning and that advance learning equity for children at risk. CALL provides three levels of feedback:

• A summary report showing results of leadership practices by domain and subdomain;
• Information about effective practices for each domain and subdomain, drawn from research literature; and,
• Guidance on specific steps to take, and tools to use, to strengthen distributed instructional leadership in the school.

Why distributed leadership? Many people in schools are responsible for establishing the conditions for improving student learning. For example, the school special education staff might provide critical feedback to teachers in classrooms, and the school literacy coach might coordinate the assessment practices that help teachers understand how to support student learning. Assessing only the quality of principal misrepresents the practice of leadership across the school organization.

Focusing on distributed leadership relieves principals from a defensive posture toward critical feedback. Instead, survey feedback helps principals model effective leadership behavior and teach those around them to become stronger instructional leaders.

Why focus on practices rather than individuals? If an assessment system were to identify the principal as a weak communicator, the path toward strengthening communication skills may be unclear. In contrast, if the system identifies the school as having weak communicationpractices, such as the lack of a clear system for communicating student progress to parents and families, the path toward addressing this gap is clearer.

For that reason the CALL survey asks questions about specific practices carried out within the classroom, in interactions between teacher and other staff members, and across the school. For example, to measure instructional leadership practices by teachers in small peer groups, the survey assesses the frequency of teacher conversations with other teachers about student work, test scores, and instruction. The CALL system then provides formative feedback to strengthen instructional capacity.

In 2012, Kelley and colleagues administered the CALL survey in 120 middle and high schools across the country. Afterwards, practitioners told Kelley that taking the survey gave them an opportunity to think about what they should do, what they do well, and what they need to work on in their leadership practice.

Read more about CALL here.