Dawn of the Dumbest Data…or
Data-Driven Dimentia…or
Data: It Keeps Teachers Busy
Take your pick. But these cats at my school really have to be stopped.
As you may suspect, we here at my school are “data-driven”. That’s right. There is no substitute for data. And the best thing about it, from an administrator’s point of view, must be that you don’t have to worry about how long it takes teachers to collect the data or if it is really of any value in the first place. Just collect that data, tell everybody you are collecting it, and that you are using it to make data-driven decisions.…for the kids. The rest, my friend, will fall into place. No worries.
Here is our scenario:
At the beginning of every course we give a “diagnostic” exam which covers the content of the course. About 30 or so multiple choice questions. Each question is to be matched to a standard. There may be more than one question per standard. After the exams are graded, each answer to each question, from each student is put into the “standards mastery tracker” spreadsheet. This is a “tool”, if you know what I mean. (Can you just imagine how excited they were when they found this? It must have been something to see.)
Over the course of the course we are to “track” each students’ mastery of each standard and create reteaching “action plans” and all manner of whatnot, driven by the data, to ensure student mastery yadda yadda yadda…
I will comment on the general stupidity of this in a moment. But first I want to mention this semester’s addition, which is sure to close the Achievement Gap very soon: It is that we must now not simply code answers to the questions as, for instance, 1= correct; 0= incorrect. We must now also indicate which of the three possible wrong answers each child chose for each question.
Now the general critique.
1) When I test students on the content of a course they have not yet taken, and then I find that they score poorly, I am not sure what I have learned. For example, the last time I did this, the overall number of correct answers was 30.2 percent. On a multiple choice exam with four choices, that is pretty much exactly what you would expect if people were just guessing. So I have learned that the students do not yet know the content of the course they have not yet taken. Is that about it? And all I had to do was enter 2500 data points.
2) Then I was required to create an “action plan” based on this data. Seriously. OK…you dummy, my action plan is to now teach the course. Jeezus. Where did you come from?
3) Why does it matter how every student does on every standard? Isn’t that what quizzes, midterms, projects and finals are for? Do any of us really need to be absolutely sure that you understand federalism or foreshadowing or folic acid. The point of looking at all of these assessments as a group is that at the end of the class we can look at a student’s work and see if he or she basically got it or not, how well, etc. If there is a systemic problem, like 80 percent of the class thinks folic acid comes from farts and methane is needed to produce red blood cells…THEN we need an action plan. Not before.
4) These questions from which we glean our precious data may or may not be good questions. They are just questions pulled from websites, written on the train on the way to work or after a couple glasses of wine at dinner. There is no quality control or testing of these questions that would give us any certainty that answering them correctly would constitute “mastery” of anything.
I have worked for three years at a university survey research center and I bet I have taken more college and graduate courses in statistics and research methodology than all of my administrators combined. What we do to and in the name of data here might be legal, but I’m sure it’s a sin.
Mr. Teachbad
The good thing about you, Teachbad, is that you never fail to share a situation that makes the rest of us realize that no matter how bad we seem to have it at our humle academy or ecole, at least we do not teach at Teachbad’s school. You have done us all a tremendous service. Carry on.
Holy Toledo! A pre-test is a pre-test is a pre-test. It ain’t diagnostic.
Once agian, you have hit the nail exactly where it needed to be smacked. Around where I teach, “data” has become just another four letter word-yes, lumped with all those other four letter words that connote everything bad you can imagine. When using one set of results in a faculty “data dump” (honestly, I couldn’t make this stuff up if I tried)–in which 98% of the students failed mastery of every single strand in the math assessment–the question was asked by the adminiweenie in charge–“Okay-so what is your plan to regroup to address the needs of the students who failed to meet the standards?” NOBODY passed ANYTHING-so wtf does it matter if you regroup or not?
We got something similar– “What are you going to do about your students’ horrible benchmark scores?” My inner thought (I didn’t dare speak out loud) was, “Uuuuh, teach????” I know I had the “dumb teacher” look on my face.
Great post. I’ve been thinking a lot about data collection in schools these days. It seems to have become more important than actual educating. Also, the extraordinary amount of time and money spent on collecting data and then doing everything under the sun that can be done with and to this data is mind boggling. It’s sucks the life right out of teaching and learning, doesn’t it?
At my old school in the great state north of me we did that on a department basis for those of us who taught the same course, for UNIT TESTS ONLY. It helped us refine the test and see which of us was teaching a specific content topic well. In that regard it was helpful. But it wasn’t collected, and micro-analyzed by people who didn’t know WTF the content was about. And it certainly wasn’t collected for our pre-tests. What would be the point of that? You haven’t taken Chemistry, I therefore assume you know no Chemistry.
Jason Kamras was never happy whenever I pointed out that I could always show growth in my class if we were looking at Value Added BS. Kids coming in know nothing, they left knowing something. How did I not add value? It made his IMPACT BS for my non-tested subject seem petty and small. Because it is petty and small. We’re just not supposed to point that out.
Sigh.…I’m sorry you still work for idiots.
Hey…chemistry teacher…was my joke about folic acid and methane good?
Can’t speak for chemistry teachers, but at least one biology teacher thought it was.
Yes, as Chemistry teacher, I can say that was very funny!
It is my suspicion that a good number of administrators get off on data—literally. Once the teachers have collected it, these data-pervs gather ’round it in one of their offices and have a circle jerk.
Well, I guess we’ve all been dancing right around that, haven’t we? Thanks for stepping up.
Holy carp, thanks for the laugh. I couldn’t articulate why, but that response was classic. Teachbad, you’re my hero!
My pleasure. If ever you need a crude and crass response, I’m your homo sapiens.
This absolutely made my day.
Crude — I don’t think my posting went through and now I don’t remember what I said — wahhh
( — I know I said I wasn’t a teacher but even I can see how dumb this is, but basically it was to the effect of testing before teaching a subject is really dumb and spending all this money on “data collecting” is really dumb, not allowing the teachers to teach what they know. Taking a test before hand does have its benefits in that it can see if there are any strong areas that the kids may be okay with that you wouldn’t have to concentrate so much on that particular aspect of a subject but test before and then after to see how much progress they have made and then grade that part of the course.
Good luck getting those dimwitted, brainless, clueless, (love this word from someone here) “adminiweenies” to see the error of their “data collecting” — E
————
Elysabeth Eldering
Author of the Junior Geography Detective Squad, 50-state, mystery, trivia series
Where will the adventure take you next?
JGDS series blog
JGDS series website
Every building in my district has a “data wall” where every teacher and team is required to post quarterly data. But that is only enforced twice a year when the superintendent comes to visit. The super himself has a “war room” (did I mention that he is a former 2 star major general?) where he has the data from all the schools in the district arranged by quadrants on the walls. We do not give diagnostic pre-tests, but interim tests to predict projected outcomes on state-wide standardized tests. Data. It’s what’s for dinner.
Is this in Colorado, per chance? Or just another super from the Broad school of educational destruction?
Just what I needed to read the night before a 2.5 hour “predictive” assessment for the ELA exam (did math 2wk ago), after which reams of paper will be generated filled with every kid’s answer, which distractor questions were answered (you better find out why that kid chose “b” and not “d”), analyses of classes, genders, ethnic groups, groups based on who had cereal vs. waffles for breakfast (just kidding), and individual printouts. All of this will be gushed over by admin at next month’s teacher detention session, er, faculty conference, only to be added to my (already 15lb) “data folio”, and stay there until the last day of school when I empty the binder into the trash.
That should read “distractor responses”, not questions. Ooops.
Thinking about your # 4 up there. It won’t be long and you won’t be allowed to come up with the questions at all because they KNOW we cant’ be trusted with that either. We had a grade level pre-assessment we created for math. But sure thing, next quarter, they have taken that away, told us how ineffective it was because it measured three digit addition and how most of our kids can’t do single digit addition. What do they replace it with???? Oh this “great” universal screener they found that measures, single, double, & triple digit addtiion, subtraction AND 1 & 2 digit multiplication!!!! Then we have to hear how great they are and how awful we are.
Beautiful. Great post.
We are starting something called PDSA this year. That stands for Plan Do Study Act. We are supposed to do many PDSA cycles in which we choose a goal for our students. We plan how we will help our students achieve this goal. We implement those strategies. We study the data and then act on it by either continuing the same strategies, implementing new strategies or choosing a new goal if the previous one was met. We are required to have a data wall, and one of these days I will create mine. My favorite part of all of this was how we first heard about PDSA two Fridays before winter break. The week we returned from break, admin did walk throughs of all classrooms looking for a data wall. Uh, when were we trained on exactly what PDSA was? Oh that’s right. We weren’t! When were we supposed to get together to choose a grade level goal like admin suggested? We were all a little busy trying to wrap up the end of the quarter, so we could spend our winter break doing report cards. Were we supposed to choose a goal the week before winter break when we were still trying to do end of quarter assessments?
Next year, we will be a PDSA school which means we will be doing PDSA cycles of data collection throughout the entire year. This year, we are only responsible for one cycle. We have to involve our students in the data collection. They have to help record the data, discuss the data and come up with ideas for ways to improve. I’m a big believer in goal setting with students, but this whole process takes so much of my time and my students’ time. I am at a K-8 school, and every grade level must participate, even the half day kindergarten classes that barely have enough time to teach anything.
I feel like all of this quantitative data interferes with my qualitative data collection, which is often more valuable to me than the numbers on a bar graph. I don’t know if upper grade teachers feel the same way about anecdotal notes, but the notes I take on my 2nd graders really allow me to know my students and plan for their needs. Since my notes can’t necessarily be represented as numbers, they seem to be less valued by admin though my principal always asks me to share my note-taking forms with my colleagues. I can’t put my notes on a colorful bar graph or use them to have kids move their rocket ships into new categories on a wall chart. I can use them to “drive my instruction”, and isn’t that the purpose of data?
“What we do to and in the name of data here might be legal. But I’m sure it’s a sin.” Sad, but true.
Your qualitative notes mirror my own experiences in teaching. I get a lot more useful data from watching and talking to students than tests/quizzes can ever show.
Grrr.…we have changed assessments each year due to some administrator getting a hard-on for a particular type of test. Each time the results are the same: One teacher posts abysmal scores, most are in the middle, and the same few are always way on top. So what does this data tell admin? You would think they would do something about the guy who obviously isn’t teaching anything well, but no. He carries on year after year with no intervention. Why should the rest of us care if data doesn’t produce action? Most of us are doing fine.
We’re being “asked” to do something called data teaming next year. Does anyone know what this is and how badly my life will suck when they force us to do it? Anything that combines data with more meetings has got to be from Satan himself.
My school isn’t data crazy but I do have a second job teaching college classes where we’re required to use a pre and post test that are so old and pop-stained, they are brown and crunchy. When we “report’ data at the end of the semester, it doesn’t even ask for pre and post scores, just how many scored 70% or more. I don’t give the tests and just make up the data. No one looks at it anyway.
One year, during an “external data review” (ie when idiots from the ISD come to “listen” to us talk about our data — really, is there anything more insane?), one of the reviewers was going on and on about some little piece of data that was missing and how it was so important. Finally the teacher in question turned to her and said “well is this data for YOU so you can THINK I’m teaching or is it for ME so I CAN teach?!” It shut that reviewer up (for once)
Oh, oh oh! I know where this comes form. Our school district went over to ExamView, which gives this feedback on every question. When they started this 2 years ago, admin. made all the tests, all the kids answer questions on the aminisitrations own special multiple choice cards, and then we were to send them in for the district to grade and evaluate. Scanning a card through a machine and having the computer add up how many kids got each score — you wouldn’t think it too hard for a data-obsessed administration, would you?
We teachers got one set of feedback, 4 weeks after the first test, and we were asked what we were going to do about it. We said, nothing, the data was too late. So then they told us we had to do the scoring and data analysis by hand. I said no — if they can’t do it with an entire administrative staff and computers, how could I, a pathetically useless math teacher, possibly accomplish it?
So now we do a pretest mean score ( I include the 0 from any student who refuses to do it) and a posttest mean score, which we can usually manage to make at least 20% higher than the pretest, and we can claim that we improved. I have argued, so far without complaint, that if students had learned their earlier work, they would come in at a 50% level (it is math, and very repetitive) and be at 70%. As it is, they are going from 10% to 30%. If only they would be put in to retake the class, then they could go from 30% to 50% and then have another go and get to 70%.
Nobody’s challenged me on it yet.
As a little hint, I have always had success arguing that I am such a bad teacher that they cannot possibly expect me to do whatever the new fad is. What can they do? they can scarcely admit that I might not be that bad…
This is all so freakin’ scary. WHEN is the pendulum going to swing back to just solid good old teaching methods? Probably never–things have gone WAY too far over the edge. Common sense—where art thou????
Sorry, there is no money to be made from tried and true, time-honored teaching techniques. Fads=CASH!
when others ask why we are doing (insert fad here), my go to response is JOB SECURITY…
From your first paragraph, I just knew you had been sneaking in and listening during our faculty meetings since you quoted the administrator word for word!!
Our district has joined forces with Discovery Education this year to use their benchmark tests and data collection system. Think of the money that was spent on that partnership!! Now we are being pressed to create our own “probes”. (Sounds too much like a colon exam to me, but whatever.) We are to use this data as we create our lesson plans so we address all the weaknesses. I know it is hard for the higher ups to fathom, but it seems like actual TEACHING instead of probing and benchmarking would produce smarter kids.…..
We had something similar called AIMSWeb — it was so full of errors and the scoring was based on such ridiculous criteria (for instance, if a math question is 3x5, and the student answered 5, they would still get full credit for having the five in the right place! Yeah, that’s the student you want planning the launch of the space shuttle someday…) that no one could take it seriously. It ate up huge amounts of class time. And then kids who didn’t do well on it (and some kids were so exhausted from being constantly tested that they just threw down their pencils) were then pulled out of their core classes to attend special classes to focus on whatever area this poorly pieced together product declared them to be weak in. Not surprisingly, their grades fell, causing a whole new wave of panic. It was a sham and a farce, but admin. could point to it and tell the state they were doing something to get us up to NCLB standards. We were a “safe harbor” school because, like many schools in this area, the special ed. kids couldn’t, through no fault of theirs or anyone’s, pass the state standardized test. So even though the rest of the school has 95% passing in every subject, because those poor special ed. kids, who are special ed for a reason, can’t pass the state exam after they’ve been given modified tests all year, the entire system is punished.
Just one more reason why I quit.
I love the stupidity of Action Plans. In our district they’re a bit different though. If a student is receiving a D or F in a core subject, we have to fill out an action plan, which is similar to yours. We have to put down three other staff members that will help implement different strategies to ensure that the particular student’s grades will improve for the next marking period. I always want to write something along the lines of, “Student must actually do classwork and homework”, “Student must get a brain”, or “Student must give a sh*t about school”. These things are so asinine that I actually have just stopped doing them, and since my school is so large, my administrators haven’t even noticed. Big surprise!
Oh, shoot. No more comments. I kept scrolling down, LMAO–wanting them to never end. We need to have an education summit of our own. I’ll bring the chips and dip–y’all bring the data! This is one hot mess we’ve gotten caught up in. Dontcha just wanna let ‘em all take over some time, let the edformers have their way, and see where it all ends up. If I didn’t need this damned job so badly, I would tell’em: “Here, you show me how it should be done.” In our state, business rules. The suits and solicitors are incharge of telling teachers all about education. Let them face down my third period class every day for a while. I want to see anybody motivate little Ronnie.
Yeah…we’ve all got a Ronnie or two. He’s really a dick.
Dear Teachbad,
I am now convinced that we teach in the same school. And if I’m right, please bring me a beer tomorrow. I think we both need it.
Your friend,
the angry music teacher.
Great post teachbad…supposedly at my school there is this “machine” that will check which questions the students got correct based on the standard. Where is this machine? It’s become sort of like the Lochness monster or Bigfoot. We hear about it…but no one has ever seen it. Anyway, who in the world would have the time to grade all this stuff, analyze the data and them come up with a re-test to regrade to see if they actually learned it. By the time all of that happens the year is over! Geniuses…
Wait, wait! You left out the step about RETEACHING. Three years ago, our district got hooked completely on reteaching. Formative assessment (FA)–a very big concept that they love to bandy about–was supposed to show teachers what and who needed RETEACHING. Imagine my science class of 35, with me going on with the standard curriculum and at the same time trying to reteach the last misunderstood, misremembered, never saw it/heard it/got it concept to the 7–10 kids whose FA clearly showed they missed it. There was no way to convince them that this could not work. Meanwhile you are held accountable to a pacing chart and a very full curriculum with no room to extend or expand, let alone RETEACH.
My favorite example of this kind of bullshit is when there is one question out of 25 for my 4th-graders I get “the talk.”
“Mrs. Tired of Being Treated Like Shit, in looking at the data, apparently 90% of your students didn’t understand essential skills 4.23A. Are you using our method? You must not being using our method because this is common assessment number five and your students should have mastered this by now considering that they failed that very skill when they took the BOY.”
“Mrs. Tired of Being Treat Like Shit. What ARE you doing in that classroom all day long?”
“What? WHAT did you just say? Did you just say something about the children doing just fine on all the other essential skills that were tested and that maybe the person at Pearson Education who wrote that one single question smoked a joint before he wrote the question? Is that what you said, Mrs. Tired of Being Treated Like Shit?”
My answer? “No that’s not what I said. I said you can take this job and shove it.”
That’s what I’m talkin’ about!
Mrs. ToBTLS, I am in love… U Rock sister.
#4 is the key. The entire premise is faulty. How can educated educators NOT SEE THIS? Because ALL THEY WANT is data in a notebook of the type that Kathy0766 dutifully maintains and then trashes each year.
Jesus.
I think people in my building have data orgasms too. Maybe that’s why they ask for so much of it.
For ToBTLS: This is the kind of crap that makes me wonder what the hell the administrator does all day if they have the kind of time to meet with individual teachers and pick through test results in such a lame-brained and obsessive-compulsive manner. Around here, principals are supposed to have made the transition from building managers to INSTRUCTIONAL LEADERS. Your guy has transitioned to chief nit-picker. He/she needs to get a life.
We’re tryin’ like hell to get her one but we can’t find any takers. Can’t say as I blame ‘em either.
I hate 3-ring binders. Every year we get a new one. I’ve got so many required binders on my shelf that I have little room for books or other essential materials. Never mind that the rest of the world ditched them a decade ago and use a lap top or smart phone to stay organized instead. You’d think that someone would figure out exactly why schools received large “donations” of 3-ring binders, Day Planners, and portfolios from area businesses several years ago. I guess our leaders are convinced that binders are symbols of success and importance.
At my school everyone has multiple binders and the students are expected to keep them as well. So much for individual learning styles. Nope, just give Johnny whose physical organizational skills are less than stellar a binder and insist that he dutifully file all his handouts under the appropriate tab. Evidently a well-organized binder is surely a key component of being a high achiever. We even have weekly binder checks overseen by administrators and support personnel to ensure consistency. It makes no difference that Johnny has one of the highest averages and is capable of doing complex algorithms in his head. If his binder is not in order he must be taught the right way to organize his papers and take Cornell notes.
I use my laptop for everything — personal, finances, work, grad school.…. I was just told yesterday that I am NOT allowed to bring my laptop to grade level planning meetings anymore– where we are “supposed” to be planning our lessons for the upcoming week, and where I was helping take notes on the electronic form our principal requires, and where I was writing down the objective #s, essential questions and ideas other teachers offered in my lesson plan form so I would have it all there when I was ready to write my lesson plans.
So much for the 21st century!!!
Just be glad that their attention is apparently focused on the students for now. Wait until they get back to you teachers. Dum-da-dum-dum,…
Which admin actually has the time to do this kind of low-level checking???
And here I was all stressed that my class data wall isn’t completed for the pre-assessment question for the current literacy pathway in time for the next required professional learning community meeting where my community will discuss our SMART goal.
I give it another couple years before someone decides teachers have had it all wrong with all this data (silly teachers), and should start collecting anecdotal notes instead. Except they’ll give it a fancier, updated name, like “descriptive feedback”.… oh wait. That’s right. I’m supposed to be doing that too.
May I just say that you all make me feel so much better at the end of the day? Seriously, I come on here to get some stress relief. It helps to know that my dog-pound isn’t the only one, that there are many other colleagues going through this hell. I wish you all weren’t either, but at least I know it’s not me and I am not alone.
I guess this data just doesn’t satisfy anymore:
1. What grade did the student get and what is the class average?
2. What percentage of their class work and homework are they completing?
3. How many days have they been absent?
Guess I’m just a simpleton to believe that this is all the (concrete) data we need.