Smorgasbord: August looms

A chart from one of my recent sessions

Teacher evaluation is about relationships and learning, not about scores

Long ago, I co-wrote a policy paper advocating for a teacher evaluation system that acknowledges that evaluation is a conversation that requires the context of a professional learning community, with input not only from the administration, but furthermore one’s peers.

While a pre and post-conference is included in most current eval systems, the reality is that the focus is on 1) compliance (paperwork), 2) the stakes/consequences attached to that paperwork, and 3) the demands of a very subjective rubric, rather than on the practices and content that will move learning forward for students.

So it should come as no surprise that few teachers are rated poorly by their principals. These systems have become all about summative evaluation, rather than formative feedback, and thus have lost sight of the real purpose of the system in the first place — to improve teacher practice and student learning. Effective principals will use the system to have those conversations — but they won’t rate their teachers poorly on paper unless they are intent on pushing them out the building.

Principals Are Loath to Give Teachers Bad Ratings, Ed Week

Research shows: Elect Democrats to fight segregated schools

Partisan tensions between individualism/choice and systems/regulation in action.

Want to reduce racial segregation? Elect a Democrat to school board, study says, Chalkbeat NY

Andy Rotherham argues against safe spaces

“. . . challenging people to become bigger than themselves is at its core an act of respect and love. Shielding them from challenge, especially in their most formative years, is fundamentally deeply disrespectful to them and their education.”

He’s talking about higher ed. But this also applies–arguably, even more importantly–in K-12.

Challenge Students, Don’t Shield Them,” US News

John King and Arne Duncan plead for sanity in regulations to protect students

“Protecting students and taxpayers shouldn’t be a partisan political issue.”

It shouldn’t. Unfortunately, however–in our country, in these times–it is.

Trump administration is putting profits over students, The Hill

The Problem with Robot Teachers

“I . . . worry that we’re slowly evolving toward a system where the affluent get that kind of education and the poor get automated schooling.”

Are The Robots Coming? Is The K-12 Sector Allergic To Accountability? Cheating In DC, College Access, David Harris Goes TEDx, Claudio Sanchez On ESSA, Jeff Walker On Systems Entrepreneurs, Curbing Eliteness, Cow Horse, More!, Eduwonk

A middle school in the South Bronx harnesses the power of testing & practice

This Bronx school is applying what we know from decades of research: repeated quizzing and practice of key skills and concepts, spaced out over time, transfers learning into long-term memory.

Kudos to MS 343. When you think about just how much of an outlier this approach is, it’s pretty disturbing. Most schools do not have a coherent and systematic approach to what they teach, nor consider how they are reinforcing what is most essential to learn across grades and classrooms.

Why this Bronx middle school believes in second — and third — chances, Chalkbeat NY

Speaking of practice, here’s 10 teaching techniques worth practicing

This is a useful list of a few pedagogical methods worth spending time mastering from UK educator Tom Sherrington, which are based on Deans for Impact’s advice for deliberate practice.

Ten teaching techniques to practise – deliberately., Teacherhead.com

NYCDOE is pressing ATRs into schools

Dan Weisberg writes an op-ed in The 74 against the move, claiming that “Principals would go back to hiding vacancies and would justifiably argue that they can’t be held accountable for student learning if they don’t get to pick their teams.”

His claim appears to be justified, as a recent Chalkbeat article reports:

“I’m going to make sure my school doesn’t have a vacancy,” said one Bronx principal who wished to remain anonymous due to the sensitive nature of the topic. “I’m not going to post a vacancy if someone will place an ATR there. I’ll be as strategic as I can and figure out another way.”

I think Weisberg’s suggestion makes much more sense: set a time limit on how long someone can be in the ATR pool.

Weisberg: Paying Teachers Not to Teach Is Absurd — but Reviving NYC’s ‘Dance of the Lemons’ Hurts Kids, The 74

New York City principals balk at plan to place teachers in their schools; some vow to get around it, Chalkbeat NY

Randi Weingarten calls Devos’s brand of choice what it is — but what is her union doing to fight segregation?

I think Weingarten is pointing out an inconvenient truth by calling vouchers a “polite form of segregation,” given their history and the folks that most typically foam at the mouth over them.

But I do wonder what exactly she and her union are doing to fight segregated schools. Public schools are doing plenty on their own to contribute to segregation without any consideration of charters nor vouchers.

TEACHERS UNION CHIEF: SCHOOL CHOICE ROOTED IN SEGREGATION, AP News

Give teachers some privacy

By PICOL- PIctorial COmmunication Language (http://picol.org/) [CC BY 3.0 (http://creativecommons.org/licenses/by/3.0)], via Wikimedia Commons
By PICOL- PIctorial COmmunication Language (http://picol.org/), via Wikimedia Commons

“…many trucking companies use cameras that automatically record a driver whenever there’s sudden braking, swerving or speeding up. But in one company Bernstein studied, the videos never go to management and are not used in performance reviews (unless the driver is texting-at-the-wheel dangerous). Instead, a team of coaches, whose only job is help drivers improve, receives the videos. Drivers, he says, like and trust that the system is there to help them, because it keeps their mistakes within a trusted circle of people who are not wielding power over their lives.

…In instituting these four forms of privacy—privacy within team boundaries, privacy limits on employee data, privacy in decision-making, and privacy about time—the organizations Bernstein studied refused the temptation to observe (or try to observe) everything. That refusal did not cost them profits or effectiveness. Instead, respect for privacy enhanced their success.”

–David Berreby, “Want People to Behave Better? Give Them More Privacy” on PsychologyToday

Observations of teacher practice have become a rote chore of paperwork to try and please accountability mavens. Those same mavens then get in a huff when they don’t find a whole bunch of teachers rated ineffective.

How about you just let teachers observe one another and give each other feedback, and administrators just sit in on that process every now and then? You might find that to be much more effective than the rigmarole of compliance that teacher evaluations have become.

My current views on testing, in answer to my past views on testing

While up in Albany a few weeks ago, I was interviewed by someone from NYSED about what I might say to parents who are considering “opting out” their child from state testing. You can view the video here*.

Someone on Twitter, “WiffleCardenal,” voiced a critique to me regarding the video, in contrast to things I’ve said in the past on testing. In fact, they even tweeted quotes of my own words! I deeply appreciate that someone out there is actually listening, and willing to take the time and effort to hold me accountable to them. I have elected to respond here, since Twitter isn’t the greatest venue for nuanced discussion, especially at the end of a long day, and I also hate typing things on my phone.

This is in reference to a live chat I did back in 2012 on The Nation‘s website with journalist Dana Goldstein and educator Tara Brancato. Have my views shifted since then? I would say they have in some ways.

You know, honestly, they’re not as terrible as I thought back then. I proctor these tests each year and go through the experience of answering the questions along with my students. The questions are often cognitively demanding and require multiple reappraisals of the text in question. A few of them are duds, certainly, but having tried to write many of my own text-dependent questions since then, I’ve come to appreciate a well-written multiple choice question. Check out this post from Joe Kirby (UK educator) on the rationale for using multiple choice questions for assessment.

Unfortunately, this continues to hold true. In reaction to this, the Center for American Progress recently created a “testing bill of rights” to advocate for better aligning tests with a more meaningful purpose.

This doesn’t mean, however, that I’m opposed to having test scores factor into my own evaluation or my school’s evaluation. When scores are considered over multiple years, I think they can be an important and useful measure of teacher effectiveness. But they are extremely variable, so I would only want them to be considered alongside other data that can provide adequate context.

One of the things I’ve become more aware of over time is that while our testing and evaluation schemes are extremely problematic, if we look at the big picture, accountability and testing do bring transparency to serving populations of students that were traditionally ignored. No Child Left Behind was certainly faulty and overzealous policy — but it also brought attention to holding school districts accountable to serving students with disabilities and other underserved populations based on data. This was entirely new, and it has raised awareness.

This is why the NAACP, the National Disability Rights Network, and other national civil rights groups oppose anti-testing movements.

Yes, I continue to believe this. Test measures are only one source of data that need to be coupled with qualitative observational data and other forms of understanding. Fortunately, I do feel like our focus, at least in NYC, has shifted to better match this understanding.

To give further context on my statements on the NYSED video, I was speaking about how I use testing data, which I do every week when developing IEPs for my students with disabilities. I compile all information I have on a student, including multiple years of state test data, in-house assessment data, such as reading, writing, and math scores, GPA, attendance, psychoeducational evaluations, social histories, etc. When viewed all together, in tandem with teacher observations and student and parent interviews, I find aggregate state testing data useful!

So it’s important to understand I’m not advocating now and never have advocated for a state test score as a singular reference point to judge myself or a student. But when viewed with appropriate context, I do find state testing data to be useful. (More on how I use that to develop IEPs here.)

No, unfortunately. While I do think that test scores should factor into an account of an individual teacher’s effectiveness (only in aggregate and when considered in terms of growth, not proficiency), we’re creating incentives for competition, rather than collaboration.

If I could set the rules for how we use test scores for accountability, I would do something kind of radical: I would hold all grade-level teachers accountable for student scores on literacy tests. And I’d stop labeling them “ELA” tests and call them “literacy” tests. Why? Because if we are honest about what we’re really testing, we’d acknowledge that the knowledge required to understand complex texts comes not solely from ELA, but furthermore from science, social studies, music, art, and so forth. (More on my argument on this here).

Furthermore, I’d try to better level the playing field for all students by requiring test makers to broadcast one year in advance which texts would be tested (not specific passages, just the general title/author). I would allow parents and educators an opportunity to vote on which texts they wanted tested that year as well to make it more reflective of current interests. The reason I would do this is that this would provide an opportunity for all students to build up the requisite vocabulary and background knowledge to access a text. Right now we just give them random texts, as if every child will be bringing equivalent knowledge and vocabulary to them, which is false.

Yes, unfortunately this continues to hold true in too many schools. But this is also why I have been a consistent supporter of Common Core standards, which have become synonymous with testing in some people’s minds. Yet the Common Core standards provided us an opportunity to move away from test prep, because they are fundamentally about building student knowledge and academic vocabulary through engagement with rich and complex texts — this is the exact opposite of test prep!

This speaks to the problem of making state tests so high stakes, and why we need multiple measures, such as direct observation, to hold schools accountable. It also is the reason for why I would advocate for the seemingly radical measure, as per above, of communicating which texts would be assessed that year so that “test prep” instead would simply be about reading and studying and discussing the rich texts that were selected for that year’s assessment.

Yes, it can be inhumane when a student is several years behind in reading ability or struggles in coping with anxiety and stress.

While computerized testing brings a whole new set of problems, I do believe we should move in this direction, because with computerized testing, we can use adaptive testing that can better scale to meet a student where they are. Otherwise we end up punishing students who are struggling, for whatever reason. Unfortunately, the needs of students with disabilities never seem to be factored into test design except as a final consideration, rather than from the ground up.

But there’s another side to this, too. I think we have to ask ourselves, as a teacher, a school, and a system, how do we prepare all of our students to be able to engage with a challenging text independently? And in what ways are we sequentially building their knowledge and skills and vocabulary in order to prepare them for doing so? It is the failure to do so systematically and adequately that we are failing students who most need those skills and knowledge.

Pearson is out of the picture, in case you didn’t know. I have no idea what Questar tests will be like, though I imagine they will be comparable.

From what I’ve heard, PARCC assessments are far superior to the cheaper assessments NY decided to get from Pearson. I think we get what we pay for, and if we want better test design, we have to be willing to fund them.

Personally, I think if we’re going to just use tests for accountability purposes, then we could make them every 2 or 3 years instead of every year to save money, and they could still continue to be used for that purpose.

What would be awesome is if we could move more towards performance based assessment. There’s a great article on them in the most recent American Educator. This seems like the right direction to go in if we truly interested in assessing the “whole child.”

Well, don’t know if all of this fully says everything I would like to say about testing, but I’m seriously tired after a long week, so this will have to do.

WiffleCardenal, whoever you are, thank you holding me accountable and I welcome continued critical dialogue on these issues.

* This was after a long day of a train ride from NYC and meetings with legislators, so I apologize for my shiny face. Won’t apologize for the winter beard, however. And no, I was not paid for that interview nor given a script. As ever, I speak my own mind (or so I like to think. Certainly let me know if it ever seems like I don’t).

Classroom Observation Significantly Influenced by Context

“Despite the intense focus on the use of student test scores to gauge teacher performance, the majority of our nation’s teachers receive annual evaluation ratings based primarily on classroom observations (Steinberg & Donaldson, in press). These observation-based performance measures aim to capture teachers’ instructional practice and their ability to structure and maintain high-functioning classroom environments. However, little is known about the ways that classroom context—the settings in which teachers work and the students that they teach—shapes measures of teacher effectiveness based on classroom observations. Given the widespread adoption of high-stakes evaluation systems that rely heavily on classroom observations, it is critical that we have a clearer understanding of how the composition of teachers’ classrooms influences their observation scores.

. . . We find that teacher performance, based on classroom observation, is significantly influenced by the context in which teachers work. In particular, students’ prior year (i.e., incoming) achievement is positively related to a teacher’s measured performance captured by the FFT.” [Bold added]

—Matthew Steinberg, University of Pennsylvania and Rachel Garrett, American Institutes for Research, “Panel Paper: Classroom Context and Measured Teacher Performance: What Do Teacher Observation Scores Really Measure?

VAM and School Ecosystems

Wiring diagram for “Wilson’s Central Pattern Generator” By Thomas.haslwanter (Own work) [CC BY-SA 3.0]
An excellent essay published in March’s Educational Researcher (1) by Susan Moore Johnson (2) brings the lens of complex systems to bear on the controversial issue of VAM.

Johnson takes to task the hasty moves of policymakers to bring VAM to the forefront of teacher evaluation. She focuses specifically on the problematic assertion by proponents that VAM is an effective measure to isolate and measure individual teachers. Johnson brings what she terms an “organizational perspective” to the issue—in other words, she considers how VAM might play out from the perspective of a school as an ecosystem, rather than as a set of individual, isolated teachers.

Johnson’s perspective is unique in that she explicitly acknowledges the impact of social networks on teacher and student performance. She refers to a 1988 James Coleman analysis, “Social Capital in the Creation of Human Capital,” which suggests that:

. . . whatever human capital schools acquire through hiring can subsequently be developed by interactions among teachers, principals, and others within the organization through activities within subunits such as grade-level or subject-based teams of teachers, faculty committees, professional development, coaching, evaluation, and informal interactions. In the process, the school organization becomes greater than the sum of its parts, and in this way, the social capital that transforms human capital through collegial activities in schools increases the school’s overall instructional capacity and, arguably, its success.

It probably goes without saying that the recognition of a school organization as “greater than the sum of its parts” is something we’re pretty adamant about here on Schools & Ecosystems.

Johnson asserts that the traditional structure of a school as an “egg crate,” in which teachers are isolated from one another, is a problematic model:

Compartmentalized school structures limit the potential development of individual teachers, who lack direct access to their colleagues’ expertise. However, social capital theory would suggest that if provided systematic opportunities to engage with their peers outside their classroom, the human capital of individuals—in this case, their instructional effectiveness—could be shared and augmented. Given this line of argument, the more robust the teachers’ instructional repertoire and the more opportunities they have to exchange and integrate promising ideas and techniques into their own teaching, the more likely it will be that all students—not only those assigned to the more effective teachers—will experience the benefits of expert teaching. This analysis suggests that teachers are not inherently effective or ineffective but that their development may be stunted when they work alone, without the benefit of ongoing collegial influence.

I’ve personally worked as a teacher in two very different kinds of school environments: one in which I was mostly isolated from my peers (due to divisive internal politics and management), to one in which I meet daily with colleagues to discuss student work and instructional practices. And I can attest to the significant impact that a collaborative and positive organizational environment has had upon my professional development.

Johnson would phrase my experience as an example of “how social capital augments human capital.” She points to the influence of context and peer learning as a factor in professional development:

. . . changing the context in which teachers work could have important benefits for students throughout the school, whereas changing individual teachers without changing the context might not (Lohr, 2012). Given that possibility, it is worth learning more about the components of a teacher’s workplace that promote greater satisfaction and more interdependent work.

. . . both theory and empirical evidence suggest that students and their schools stand to benefit when teachers work closely and collaboratively with colleagues.

Johnson also explores the “unintended consequences” of relying on VAM to evaluate individual teachers, a warning that we’ve echoed here in response to Gov. Cuomo’s misguided push for greater weight on student test scores, and which also echoes similar issues that Dana Goldstein has pointed to in her historical exploration of teacher evaluation.

 

She concludes that “that expanding the use of VAMS in teacher evaluations (even if it represents no more than 30% of the teacher’s total score) might compromise the school’s potential for improvement.”

Johnson therefore suggests that “reformers should lead the way with efforts to improve the school throughout as an organization that supports effective teaching and rich learning.”  She suggests including teachers in the hiring and professional development process in their schools, as well as in recognizing and sharing instructional practices from teacher leaders, such as Peer Assisted Review (PAR) programs.

This is the kind of thinking that I wish we heard more of in education reform circles—not hasty, breathless championing of the latest panacea, but instead reflective approaches that consider schools as whole organizations, complex adaptive systems ripe with social interactions and relationships.

1 Educational Researcher, Vol. 44 No. 2, pp. 117–126 DOI: 10.3102/0013189X15573351 © 2015 AERA. http://er.aera.net

2 I was alerted to Susan Moore Johnson’s essay by a link to a Marshall Memo from an April 9th USDOE Teacher’s Edition newsletter.