Accountability Measures
We had a leadership team meeting on Friday, where some heavy-hitting, paradigm shifting ideas about instruction and fluid scheduling were presented, discussed, and thunk on. We spoke also of a big-time approach to instructional program evaluation that takes us into a school-wide analysis of exactly what was taught and how often. We were talking about our instructional obesity, where kids get fat on the repeated teaching of certain skills, while remaining weak in others. There was some big-think happening, and I managed to use the word goldilocks as a verb (work that one through your noodle, why dontcha), but the whole thing was tempered by the need to develop a conception of the product we were looking for. What do we want? Truly, what will be produced?
I think the progression of comments in the previous post have reached this same point, with the inevitable surfacing of the I'm-down-for-accountability-but-on-different-terms argument, which itself dove-tails nicely with the standardized-testing-is-grossly-unfair-almost-always assertion. So let's talk product. What would an acceptable measure of accountability look like?
First principle: Growth
Student growth and progress is the a priori starting point for accountability, and its almost the whole ballgame. Where did they start and where did they end? How far past time-1 are they in time-2? We cannot measure student learning as a series of end-points, viewed in isolation. As always, it's not whether or not you crested the peak, it's how far you climbed. This is all the more critical for those teachers working with SpEd, EL, or generic low-achievers, and prevents folks from hiding behind those distinctions. It eliminates the I-can't-be-held-accountable-because-my-kids-start-too-far-behind argument.
Growth-based accountability must be quantified. How much growth is acceptable? This varies, of course, by where students begin. In talking to the POY, I suggested three classifications: extreme growth, growth, maintenance. Student readiness determines both the classification, and the corresponding expectations of progress as measured by either scaled scores or inter-quintile growth -- and I think the former is the way to go. By focusing on a sliding-scale measurement of growth, you prevent the problems tied into a binary measure of achievement -- proficient or not -- that ignores massive gains made at the bottom, and you also prevent a negative assessment of teachers with high-performers whose progress is less about making multi-year improvements, and more about maintaining and tightly growing learning.
Second principle: Accept the big tests
They aren't perfect, and no doubt there are individual questions or individual students who remain unable to demonstrate learning adequately on the tests, but these are not systematic inequities. In California, our tests are well-tied to the standards, the standards are appreciably rigorous, and the test measures by and large fair. I reject the notion that the tests are unduly stressful, or that it is unjustifiable to measure a year's worth of progress on just a few days -- these are, after all, summative measures. Moreover, I reject the idea that this would lead to a rash of the vile "teaching to the test." Teaching to the test also goes by the name of effectively leading students toward mastery of the standards, and is in no way vile.
I can make no argument for student performance on the CSTs being excluded from any evaluation of my teaching performance.
Third principle: Go beyond the big tests
The tests are fine, but they are blunt instruments, capable of shedding limited light onto the question of student growth and achievement. This is not a reason to dismiss them outright, but it does underscore the necessity of including additional, local measures of performance measuring.
In addition to a formalized expectation of scaled-score growth on summative standardized measures, teachers and administrators need to discuss classroom specific goals related to the nature of the course. These in turn vary according to the content, and can move between skill acquisition, work product, process-oriented participation and the like. Okay, you teach 10th grade honors English, what can I expect from your students by the end of the year? What will they produce? How does this demonstrate academic and intellectual progress? Okay, you teach Newcomers. We know they'll all be classified as Far Below Basic. How will they progress otherwise, in terms of language acquisition? Where will they score on the CELDT? How many will be ready for inclusion in less sheltered classrooms?
And so on.
These conversations can be extremely powerful things, professional development in their own right. Localized goal setting that exists under an umbrella of high expectations and clearly outlined student outcomes for which you are then accountable in a real way is some serious hard-hitting stuff. This is like calling in some baseball-bat wielding [guys] and getting medieval on the process of teaching and learning.
Fourth principle: Eliminate the perfect-from-the-start roadblock
Education is absurdly hidebound in its approach to innovation and the implementation of change. Merit pay, instructional freedom, local budget control, local hiring control, and a host of other factors get crunched under the sweeping blade of those who cry we can't! we can't! the system could be abused! Sure it could and sure it does and sure it will. Show me the field or industry where there is no nepotism, cronyism, gamesmanship, manipulation, misdirection, or outright fraud. The existence of all these remains unfortunate, but at the risk of waxing philosophic, some of this is the nature of the human condition and goes with the territory. We've certainly accepted the possibility that our political, economic, and religious leaders have not always arrived at their exalted positions by virtue of merit alone, and we've certainly accepted the existence of all manner of not-so-ethical practices throughout daily life.
Now, personally, I believe this oh-well acceptance of malfeasance sucks, but while it exists so pervasively elsewhere, to inhibit educational reforms on the basis that some people some where will find loopholes through which to pour their bad acts is plain silly. Previous commenter Allison makes this point when she writes, "No professional disciplines allow this crap about is X the best method for assessment? They agree on methods; best, better, whatever--they pick a set. They agree to them, peer review them, and if they aren't working, they revise them."
Yes. We sit back and wring our hands constantly on the issues change and improvement, determined to find an absolutely perfect method before trying anything. It's ridiculous, and we end up in this cycle of debating abstractions in hypotheticals, because we lack the will to try, to implement, to analyze and improve.
I think the progression of comments in the previous post have reached this same point, with the inevitable surfacing of the I'm-down-for-accountability-but-on-different-terms argument, which itself dove-tails nicely with the standardized-testing-is-grossly-unfair-almost-always assertion. So let's talk product. What would an acceptable measure of accountability look like?
First principle: Growth
Student growth and progress is the a priori starting point for accountability, and its almost the whole ballgame. Where did they start and where did they end? How far past time-1 are they in time-2? We cannot measure student learning as a series of end-points, viewed in isolation. As always, it's not whether or not you crested the peak, it's how far you climbed. This is all the more critical for those teachers working with SpEd, EL, or generic low-achievers, and prevents folks from hiding behind those distinctions. It eliminates the I-can't-be-held-accountable-because-my-kids-start-too-far-behind argument.
Growth-based accountability must be quantified. How much growth is acceptable? This varies, of course, by where students begin. In talking to the POY, I suggested three classifications: extreme growth, growth, maintenance. Student readiness determines both the classification, and the corresponding expectations of progress as measured by either scaled scores or inter-quintile growth -- and I think the former is the way to go. By focusing on a sliding-scale measurement of growth, you prevent the problems tied into a binary measure of achievement -- proficient or not -- that ignores massive gains made at the bottom, and you also prevent a negative assessment of teachers with high-performers whose progress is less about making multi-year improvements, and more about maintaining and tightly growing learning.
Second principle: Accept the big tests
They aren't perfect, and no doubt there are individual questions or individual students who remain unable to demonstrate learning adequately on the tests, but these are not systematic inequities. In California, our tests are well-tied to the standards, the standards are appreciably rigorous, and the test measures by and large fair. I reject the notion that the tests are unduly stressful, or that it is unjustifiable to measure a year's worth of progress on just a few days -- these are, after all, summative measures. Moreover, I reject the idea that this would lead to a rash of the vile "teaching to the test." Teaching to the test also goes by the name of effectively leading students toward mastery of the standards, and is in no way vile.
I can make no argument for student performance on the CSTs being excluded from any evaluation of my teaching performance.
Third principle: Go beyond the big tests
The tests are fine, but they are blunt instruments, capable of shedding limited light onto the question of student growth and achievement. This is not a reason to dismiss them outright, but it does underscore the necessity of including additional, local measures of performance measuring.
In addition to a formalized expectation of scaled-score growth on summative standardized measures, teachers and administrators need to discuss classroom specific goals related to the nature of the course. These in turn vary according to the content, and can move between skill acquisition, work product, process-oriented participation and the like. Okay, you teach 10th grade honors English, what can I expect from your students by the end of the year? What will they produce? How does this demonstrate academic and intellectual progress? Okay, you teach Newcomers. We know they'll all be classified as Far Below Basic. How will they progress otherwise, in terms of language acquisition? Where will they score on the CELDT? How many will be ready for inclusion in less sheltered classrooms?
And so on.
These conversations can be extremely powerful things, professional development in their own right. Localized goal setting that exists under an umbrella of high expectations and clearly outlined student outcomes for which you are then accountable in a real way is some serious hard-hitting stuff. This is like calling in some baseball-bat wielding [guys] and getting medieval on the process of teaching and learning.
Fourth principle: Eliminate the perfect-from-the-start roadblock
Education is absurdly hidebound in its approach to innovation and the implementation of change. Merit pay, instructional freedom, local budget control, local hiring control, and a host of other factors get crunched under the sweeping blade of those who cry we can't! we can't! the system could be abused! Sure it could and sure it does and sure it will. Show me the field or industry where there is no nepotism, cronyism, gamesmanship, manipulation, misdirection, or outright fraud. The existence of all these remains unfortunate, but at the risk of waxing philosophic, some of this is the nature of the human condition and goes with the territory. We've certainly accepted the possibility that our political, economic, and religious leaders have not always arrived at their exalted positions by virtue of merit alone, and we've certainly accepted the existence of all manner of not-so-ethical practices throughout daily life.
Now, personally, I believe this oh-well acceptance of malfeasance sucks, but while it exists so pervasively elsewhere, to inhibit educational reforms on the basis that some people some where will find loopholes through which to pour their bad acts is plain silly. Previous commenter Allison makes this point when she writes, "No professional disciplines allow this crap about is X the best method for assessment? They agree on methods; best, better, whatever--they pick a set. They agree to them, peer review them, and if they aren't working, they revise them."
Yes. We sit back and wring our hands constantly on the issues change and improvement, determined to find an absolutely perfect method before trying anything. It's ridiculous, and we end up in this cycle of debating abstractions in hypotheticals, because we lack the will to try, to implement, to analyze and improve.
posted by Kilian Betlach at 2:32 PM
4 Comments:
You've been hitting quite a few out of the park lately. Keep it up.
Thanks Ken. There's been some dry spells this year, and I was thinking of hanging em up a time or two. Maybe those thoughts were premature.
I salute the sentiment, tmao....
CSTs aren't the best measure but let's not punk down from them.
Just don't compare last year's students to this years; follow the progress of each student, like you say. Teachers can keep a portfolio of student work -- beginning and end of year efforts (if they're way more organized than me) -- to show in defense of test scores they think don't do justice to their efforts....
As for the necessity to make imperfect reforms, you put it about as well as I've heard it put....
What if we obliterated every law that was difficult to enforce?
When Ed Koch became mayor of NYC in the 1980s he told New Yorkers that he wasn't going to try to fix the whole city. That was impossible. But one pothole at a time he could start to repair things.
Just don't compare last year's students to this years; follow the progress of each student, like you say. Teachers can keep a portfolio of student work -- beginning and end of year efforts (if they're way more organized than me) -- to show in defense of test scores they think don't do justice to their efforts....
Huzzah!
That always struck me as one of the more asinine bits of state-mandated testing--the notion that this year's class should score higher on tests, depite the obvious and glaring facts that a) they're different kids and b) the freaking tests change from year to year. AYP? Sure. But for each student, not for aggregate classes.
Post a Comment
<< Home