High School Concerns - List of Considerations

This is a running list of concerns from people in my building & neighboring schools. Please feel free to add to these issues and concerns so we can classify/address them.
Cindy Rudolph
What about kids who are dumped into classes and do not meet pre-requisites but there is no other place for them to go? That happens a lot in CTE.

Thanks for your work with the value-added team. Here is my list of things that I believe need to be considered. Once again, most of these things, though significant and clearly tied to student test scores, cannot be figured into a "mathematic formula" and will require human interaction such as the keeping and checking of documentation.

1. In some situations, even though a student gets the same achievement level, he/she has actually shown growth. For example, the 8th grade English EOG tests only reading. The 9th grade EOC tests reading and writing (editing and revision skills), and the reading passages and questions are more challenging. Thus, if a student gets a LEVEL 2 on his/her 8th grade EOG and then gets a LEVEL 2 on his/her 9th grade EOC, then he/she has grown because he/she got the same score on a harder test. However, if a person just looks at the record and sees "8th Grade EOG: LEVEL 2" and then sees "9th grade EOC: LEVEL 2," it will look like there was no growth. This is not accurate, though, because in any subject area, if a student is able to get the same score on a harder test, at least some growth has occurred.

2. In some situations, comparing last year's test of "Subject X" to this year 's test of "Subject X" will not be an accurate comparison and will thus be an inaccurate indicator of growth. The 9th grade English EOC is all multiple choice, mostly reading, with writing-related questions but no actually writing. The 10th grade English EOC has no multiple choice questions and no reading and is nothing but actually writing. If a student is a strong reader but a poor writer (or visa versa), then the results will be skewed. Also, even if the 9th grade EOC only covered writing skills, it's still a multiple choice test, and comparing a multiple choice test to a writing test is not an accurate comparison.

3. Looking at previous scores is not a consistently accurate indicator of expected growth. What if a student had a great year last year (health, home life, friends, school life, teachers, extended family, etc.), and he also happened to score a LEVEL 3 on all his EOCs. The "system" expects that student to show growth on this year's EOCs. So what if the student develops kidney problems, loses his dad to a divorce, loses his best friend because of a move, finds out his grandma has cancer, etc., etc., etc.? He may become a completely different person and lose all motivation to do well in school, thus scoring a LEVEL 2 on his EOCs this year. None of this can be plugged in to a mathematic formula, but all of it can be documented and brought to the attention of the supervising administrator.

4. What about teachers who teach all standard level students versus all honors students? A student in an honors class who has the same (or lower) abilities of a student in a standard class may actually achieve more simply because the environment in an honors class is often more conducive to achievement because there are usually more motivated students and usually less interruptions and less time spent on classroom management. How will these scores be compared accurately? Schools cannot "equally divide" honors students and standard students among all teachers because the numbers and the schedule do not work out that way, so we will have to deal with the issue that the learning environment will not be the same for all of these students, which is out of a teacher's control, but the test scores for all of these students will be judged equally, which will be connected directly to the teacher. Additionally, they need to take into account that standard and honors students all take the same exam with no curve added to scores of the standard and yet we are judging their growth on the same scale. What other class in any school would give the same final exam for all students, regardless of level, without adjusting the final scores in some way to compensate for the different levels? Even if the state is not going to do anything about this in their final scoring & reporting of the tests, at least CMS can do something before they start determining pay based on these scores.

5. Looking at socio-economic factors does not always accurately indicated expected growth for a student. A student who has wealthy parents and a seemingly "perfect learning environment" in his life may still choose to turn his/her back on a teacher's instruction and attempts to help. See the next point for further explanation of "behavioral" issues.

6. What about students who reject the help & instruction that is given to them, sleeping in class, sleeping through the exam, seldom or never completing work, seldom or never coming in for help, seldom of never paying attention, missing class, etc. If a teacher has made an acceptable effort to instruct a child, contact home, apply other interventions, etc. and the student has not responded, then the teacher should be released from any responsibility connected to that student's score. The criteria for being "released from test score responsibility" can be established by the district in conjunction with teachers and administrators. Teacher can keep documentation of interventions and student responsiveness, and this documentation can be presented by the teacher to the evaluating administrator. It will be up to the evaluating administrator (or perhaps a committee) to decide if the teacher will be "released from test score responsibility" for any particular student.

7. What about test length? Is it fair to hold a teacher responsibile for a student's score on a test that may take as long as four hours to complete? Also, how can we be certain that a student's actual ability and knowledge is the only thing that is being tested when this student is also having to endure through this test with two brief silent breaks and not meaningful interaction with the teacher and/or other students? How can a teacher prepare a student to endure through this? It's not like we have a bunch of four-hour-long practice tests we can give or even a bunch of two-hour long practice tests to give. If we did, when would we give them since we only have 90 minutes? The standard course of study does not indicate that we are to teach "test-taking endurance" to our students, so why are we being held accountable for a student's ability to endure that long? How can we expect a student to do his/her best in this environment, especially when he/she has been instructed in a radicallydifferent environment (meaningful teacher and student interaction, movement in the classroom, frequent breaks, music during transitions, visual aides, etc.). If we want these tests to be accurate indicators of a student's knowledge and ability, which reflects a teacher's performance, then we must look at the testing environment and length and consider how we might change it. At the very least, we must appeal to the state to let us split each test in half, at least the most challenging ones, so that we can administer them in two parts.

8. Some students get extra time to prepare for the EOCs. This practice needs to be factored in, or it needs to be made consistent throughout the district. For example, at some schools, students get one semester to prepare for the English I EOC, in other schools the standard students get two semesters (Foundations of English, which has the exact same goals & objectives as English I, and English I), and in other schools the standard AND honors students get two semesters (Same two classes mentioned above). This will clearly influence scores, so as I said, it must be factored in, or the preparation practices must be made uniform throughout the district.

9. Some students are appearing on rosters and EOC score lists even though they have never been in a teacher's class. This seems like a deceitful practice to begin with, so it needs to be addressed. Regardless, these students should not count for or against a teacher because the teacher has had no influence on these students.

10. It seems odd that the district already has "value added" scores on file even though they are still working on the "value added" process & formulas. These scores are being evaluated, ranked by school, sent to schools, and even published in the paper. Why are they publishing these scores as if they are accurate representations of schools and teachers when they are still developing the "value added" process. And, more importantly, when they start determining pay based on "value added" scores, will they use past data or new data, and if they use past data, will that data be recalculated using the fully developed value added formula? If not, those past scores will not be accurate representations of students or teachers.


1) How will classes that are required for graduation be measured as opposed to classes required for graduation? (students may care to attend, study, pay attention, etc...to classes that are required for graduation)

2) If CMS is using mean as a way to assess teacher effectiveness this is one of the most inaccurate ways. Mean is not a good measure of central tendency because of outliers that can skew the distribution of a bell curve.

3) Will CMS consider students that are reading at grade level to have shown growth? (Ex: a student scores a 3 on their 9th grade English exam. As a 10th grader they also score a 3. To me, this shows they have grown a year because the reading exam should be a year harder than the year before.)

4) How will CMS account for home life situations that teachers may not be aware of. (Ex: domestic violence in the home, single parent homes, students have to care for their siblings while their parents are at work, etc...)

5) I'm concerned that this value-added approach will cause unhealthy competition rather than collaboration amongst teachers.

6) What if we (teachers) contact parents, talk to the student about their grades and missing assignments, tutor the student, etc... and the student is still not able to meet the requirements of the class (honors or AP) what happens then?

7) Juvenile law protects students under 16 from having their record public. If the teacher doesn't know that the student has prior charges or current charges how will CMS take that into account when evaluating students?

8) What incentive does CMS have to give teachers money if they are considered effective? The current budget wouldn't allow that type of pay. It seems that CMS would benefit more (monetarily) if they didn't give teachers the value-added credit causing teachers to loose trust that CMS will keep their word on fairly rewarding teachers.


I think we should also have pre-requisites to take some courses, especially honors and AP. Pre-requisites should be certain grades and also teacher recommendations.

If we implement those, we can change the culture and maybe increase performance.


1) Number of student absences- greatly impacts test scores
2) EC/504 status- can impact test score
3) Previous EOC and EOG scores- I think middle school EOG reading scores should be taken into account for history, they need to be able to comprehend to take this test. If student has a record on not doing well on EOC's that is a cause for concern as well.
4) Free/reduced lunch status
5) Record of any criminal/arrest record can impact test scores
6) Teacher input- communication logs, notes about students sleeping in class. If they are going to assess us on test scores then teachers need to be able to defend themselves showing communication or lack of communication by parents and students who are lazy and not working in class.
7) Failures- students who fail class usually don't do well on tests
8) District training- is the district providing teachers with tools and resources necessary to get good test scores. Ex- The district will not pay for me to attend AP training and nothing is offered locally.
9) Environment- this includes class size, room temperature, inability to hang resources on walls because it will not stay put, lack of technology in classrooms (why do some teachers have smartboards and others do not), and there are many other things in this category that can greatly impact student achievement and test scores.
10) Loss of instructional time- student assemblies, etc that interrupt our instructional time. I have lost at least 3-4 days of instructional time in my 2nd block this semester due to unnecessary meetings (Class ring reps, discipline assembly, etc.)
11) Suspensions- leads to absences which impacts achievement and test scores

Sorry for the long list. I could go on but I think you get the point. There are so many variables that impact student achievement and test scores, how are they going to account for them all? What will determine which variables are included and which ones are not?


In general, I think that teachers entered the profession with the expectation that it would be a profession where competent practice of the art would be compensated according to a professional pay scale. Low but predictable.

The forced ranking of the Value Added Model changes this into a contest. There is no criterion on competency, only competition. If I do my job competently, and someone else finally figures out how to teach, my score goes down.

It is ironic that we have been using forced ranking (percentile scores) to mark our students for years, but when percentile scores are applied to teachers, it feels odious.

I work in the EC department. The IEP process has been devised by compassionate lawyers to protect and enable the children. Each year the IEP teams compose goals for the students. If the EC teachers are measured effective by how well those annual goals are met, they will be more careful in agreeing to them. I have seen some goals that are not attainable. The rules say you can not have the same goal two years in a row.


#1
Recently, I sent some follow up questions to Mr. Muri after he had attended an SLT meeting.

Below are the questions and the explanations behind those questions. Question #1 deals with using test data to evaluate teachers. I asked Mr. Muri at the meeting if "student motivation" was going to be a part of the value added formula, and he simply said "no" without offering any explanation. Question #1 is my follow up to that response.

#2
I am also concerned that some schools use an entire year to get students ready for an EOC test by putting students in certain related classes first semester (eg. Foundations of English) while others do not. This year, we chose to try a different option (for reasons we can discuss later), so most of our standard level English I students only have one semester to get ready for the test. This is not the case for many other CMS schools, some of which (according to Teri Lawson) are even having their English I honors students take a prep class in the fall.

Nancy Kaufman said that she went to a meeting where someone asked if this "different prep time" would be taken into consideration when calculating value added scores. The folks there said "no". As I have told Jeff, Ann, and Kristie in a previous e-mail, it seems that the district needs to come up with a uniform way to approach preparing students for the EOCs or figure out a way to adjust scores to make the comparisons fair. Further, someone needs to investigate if the state even approves such "extra prep" classes since years ago, it was against state guidelines to use more than one year (or one compact semester) to prepare students for any EOC.

I look forward to the meeting, and as I said before, please know that I respect you and am glad you are on this team.

Todd



January 7, 2011
Mr. Muri,
Thank you for your time and attentiveness at Hopewell's January SLT meeting. I know this was a sacrifice for you. I am glad to have had the chance to speak with you and hear your thoughts. I certainly benefitted from our time together.
I understand that you have a daunting task ahead of you, and I admire your willingness to confront it. I hope that I can help in some way in the future. Perhaps the questions and thoughts below can begin that process.
I know that I have much to learn about all of this, so if you have any input for me, please do not hesitate to send it.
Thank you,
Todd Humphrey
English teacher, Hopewell High School (t.humphrey@cms.k12.nc.us)

Additional questions regarding performance-based pay in CMS:
1. Why you are not including “student motivation” in the value added formula?
(If needed, please see further explanation below.)
2. Have you folks considered that the MCREL teacher evaluation may be so overwhelming for teachers that it may have a negative impact on student achievement and may even drive away good teachers (and administrators who have to evaluate those teachers)?
(If needed, please see further explanation below.)
3. Since “student achievement” is so crucial, do we have any data to support that teachers with ratings above “proficient” actually have higher-achieving students?
(If needed, please see further explanation below.)
4. Have you folks considered adding more "unannounced, ongoing observations" to the teacher evaluations instead of relying so much on teacher-created records and test scores?
(If needed, please see further explanation below.)
5. Do you understand why teachers with good reputations and good track records are considering other employment opportunities and may not stay around to see how it all works out and why even top teaching candidates in the universities may not even decide to teach after all?
(If needed, please see further explanation below.)

Further explanation for QUESTION #1
You said you have it “all figured out” when it comes to fairly interpreting the test scores of standard-level students versus honors-level students. You said you have a formula for doing this. However, when I asked you if “student motivation” is a part of that formula, you said, “No.” This concerns me greatly for these reasons:
1. Other than “knowledge & ability” I can’t think of anything more important to student achievement than student motivation.
2. Student motivation is one of the most (if not the most) challenging aspects of teaching, particularly when it comes to teaching lower-level students. A student may have been taught well, but if this student is not motivated to learn, he/she will not learn.
3. Even if the student is motivated to learn, if he/she is not motivated to perform, particularly on a state test, then this student will look like he/she was not taught well and did not learn much.
4. Even if a student is motivated to perform on an EOC test, will that student be motivated enough to work on that test for three to four hours? Is his/her score an accurate reflection of how well he/she has been taught? Or does it actually reflect how well he/she can endure and remain motivated for three to four hours of testing in the same seat with no meaningful interaction from the other students or from the teacher (both of which play a significant role in student motivation).
Related to this is your use of data from Bill Gates’ MET program and the THINKGATE program. The test data in these programs comes from tests students take for which they know they will not receive grades and therefore have little to no motivation to do their best on these tests. Once again, this data is greatly skewed by a lack of student motivation and yet student motivation is not being considered in the data.
With all of this in mind, it seems that a “value added” score that does not consider student motivation will not be completely valid in measuring student achievement or teacher effectiveness.
Further explanation for QUESTION #2
Here are a few of the overwhelming characteristics of the new MCREL teacher evaluation:
1. It has four levels of evaluation (Developing, Proficient, Accomplished, and Distinguished) with a total of 147 points of evaluation.
2. If a teacher wants to move beyond the lowest acceptable performance level (proficient), he/she must consistently accomplish tasks beyond the classroom, which may be helpful for the school but can often distract from the vital needs of the classroom and the students, our highest priority.
3. It is not acceptable to accomplish a few of the items above proficient or even most of the items above proficient. A teacher must accomplish all of the items in proficient as well as all of the items in the other levels, or he/she will move back down the scale.
4. It is not good enough to perform all of the items on the list. Teachers must now also prove they are doing these items by writing reflections and collecting artifacts.
5. There is no indication as to how much evidence needs to be supplied for each point of evaluation, which makes the process even more overwhelming. Is a teacher to provide daily evidence, weekly evidence, monthly evidence, bi-yearly evidence, or yearly evidence?
6. Trying to accomplish all of these items as well as reflect on these accomplishments and compile artifacts for all these accomplishments is an enormous task. I tried to do this at the beginning of the year, and my students suffered because of it. I was spending so much time trying to prove on paper that I was above proficient that I did not have the time or energy to actually be proficient or above in the classroom.
After the first month or so of school, I finally decided to be “content with proficient on paper” while striving to be “accomplished” and “distinguished” in as many areas as possible by focusing my extra time on my students’ most pressing needs. I actually wrote a letter to the staff explaining my “journey” and my decision. I heard from many teachers who had experienced the same challenges and were grateful for the freedom to be content with proficient on paper. The administration has even encouraged teachers to consider adopting this approach in order to get through the new process.
Of course, this approach is fine for now, but when “proficient” is at the bottom of the pay scale, it will probably result in a pay cut and will also make teachers less “marketable” for other jobs. Thus, in the near future, teachers will be pressured to move up the evaluation scale which will require them to do more as well as provide evidence that they have done more. This will be an incredible burden for all teachers who are increasingly overwhelmed as it is.
Sadly, some teachers even feel this pressure now because they are concerned that the MCREL evaluations they receive over the next three years may be used to determine their income when the program officially begins. Sure, this option for determining pay has not been made official, but no other option for determining pay has been made official (as you stated), so this option seems possible.
All of this impacts administrators and principals as well because they have to spend the time evaluating teachers based on this extensive process while also completing their own version of the MCREL evaluation for themselves. I cannot imagine how much time it must require for them to complete all of this in the midst of their incredibly demanding jobs. It seems like this must have an impact on how effectively they can help and evaluate teachers, handle safety and discipline, lead and envision the school, handle various administrative issues, meet with parents, spend time in the hallways and classrooms, etc.
Further explanation for QUESTION #3
There is no denying that doing well on the MCREL evaluation instrument will require more time for teachers, something teachers will have a hard time finding in an environment where time is already so precious and demands are increasing. And when a teacher does not have the necessary time to devote to planning, evaluating, contacting parents, tutoring, sleeping, exercising, spending time with family & friends, etc., that teacher’s effectiveness is impacted. As a result, student achievement is impacted.
Further, it seems like completing what is needed to attain the accomplished and distinguished rankings is somewhat similar to completing what is necessary for a masters or doctorate degree, especially since many of the aspects of these higher rankings address accomplishments beyond the classroom. It seems that teachers who devote more time to reflecting and collecting artifacts may end up looking better on paper even though their students may not achieve more. As with the advanced degrees, teachers will be doing things beyond the classroom to get recognized and get paid more, but those things may or may not directly impact student performance.
Yes, I know the “value added” scores will help balance this somewhat, but regardless, it seems like the connection between the MCREL evaluation instrument and student success needs to be more carefully investigated and communicated before we put so much weight on this means of evaluation. It would be helpful to see actual data that connects “MCREL success” to “student success.”
In fact, I would like to meet a teacher who has been able to effectively and consistently accomplish and prove all 147 points of evaluation, keep the rest of life in balance, and still have the time to help his/her students achieve success.
I could learn a lot from someone like this!
Further explanation for QUESTION #4
It seems like we could discover our strongest and weakest teachers more effectively and more quickly by having more "eyes" in the classroom on a more consistent basis. So much can be discovered about a teacher in a 10-15 minute observation, and if teachers knew they were going to be observed daily or almost daily, they would have fresh motivation to grow in their effectiveness. This would also alleviate the need for teachers to make records of everything they do in the classroom because more of what they do in the classroom would be observed on a regular basis. Teachers would also receive more consistent feedback and help, something they do not receive now, at least not regularly.

Using this system instead of the extensive MCREL system would cut down on paperwork for teachers and administrators and provide them with more time to do other things that may have a greater impact on student achievement. Teachers would also feel as though they could be recognized for doing good things in the classroom, even small things, without feeling the obligation and burden to remember all of these actions and/or comments and then find the time to record them later.

I know it would be challenging to increase teacher observations, but it seems we need to consider all options for accomplishing this. One teacher suggested that we hire retired teachers, paying them a small salary and asking them to do brief observations each day and report back to the administration any concerns or commendations. Another teacher suggested that we put cameras in every classroom so that teachers knew they could be corrected or commended for anything they did on any given day. Administrators would have the freedom to watch excerpts from teachers classes at any given time on any given day, and video data would be gathered that could be used in other ways as well (research, idea sharing, accountability, personal reflection, etc.).

Of course, we could have select teachers participate in this process, going a few steps beyond the present "peer evaluations" that we use. Perhaps even students can be involved in some way. Including student surveys and interviews in the evaluation process is certainly a valid consideration for a variety of reasons. In many ways, students know best what good teaching looks like. They know who cares and who helps them most in the learning process. Parents, too, can be included in this process through surveys, interviews, classroom visits and observations, etc. All of these processes would not only help evaluate teachers, they would also help inspire teachers to be more effective.

I am not sure the best way to get more "eyes" in the classroom, but I am sure that this is something that would help improve teacher performance and help improve the process for effectively and accurately evaluating teacher performance.

Further explanation for QUESTION #5
Teaching public school students is already one of the most increasingly overwhelming and often increasingly discouraging jobs in the market, particularly for teachers who really care and work hard every day (and most nights) to help their students succeed. Teachers are tempted to be even more overwhelmed and discouraged when they are told the following:
1. Teachers will soon be paid based on their performance and yet the tool for measuring their performance has not yet been determined.
2. Teachers will be evaluated using the extensive MCREL evaluation (explained above), which will require significant effort, not only to accomplish the tasks on the criteria but also to record evidence of accomplishing those tasks. Even maintaining the lowest acceptable level on this evaluation (proficient) requires significant work and proven effectiveness in the classroom, yet teachers who are at this level will still be at the bottom of the pay scale.
3. Teachers will be evaluated primarily by what is recorded on paper (MCREL evaluation & value added scores) and not what is observed consistently in the classroom. This means that anything they do in the classroom that may be at the “accomplished or distinguished level” will not be recognized unless it is documented by them, something that can take a significant amount of time, time that is most often needed for dealing with more pressing student-related needs.
4. Teachers will be evaluated using a “value added” score that is determined by student test performance and yet “student motivation” is not going to be considered when evaluating these test scores even though student motivation plays a vital role in student achievement.
5. Money is running out and there is no guarantee that teachers will make much money even if they meet all the criteria above.
It seems that adding all of this to the ever-increasing daily demands of teaching is going to make it increasingly challenging to keep good teachers as well as increasingly challenging to attract good teachers.
For the sake of these teachers and for the sake of the students they represent, I hope I am wrong.
Thank you for your time and consideration,
What happens if a teacher is out on extended medical leave? Ie. maternity, disability, personal etc. (Mawardi, Manners, Brown)

Are they given a test the first day and one the last day? will they be the same test?

What about students who miss 20+ days?

What about students who have a devastating event happen during the semester? ie. death of a parent/significant loved one, parent loses job, lose home, MCV kids

What if they just start performing poorly because of something that happened in their lives? (break-ups (we know how that can affect some kids), drugs, divorce, etc.

These are just a few concerns. I'm sure Ill think of more.

Cindy,

After looking at my value added data report, I am wondering if you can help answer some of the questions running through my mind. First, I am wondering why there is no standard deviation when looking at these results? I am being hurt the most by kids that scored at or higher than the achievement level they scored on the previous year's assessment. It just seems wrong to me that if a student made a C on his 8th grade test and makes a B on his 9th grade exam that I am marked down because he didn't earn a high enough B. I was especially hurt by the students who were projected to make A's. Despite the fact that the majority of these students still made A's, many of them hurt me on my value added data. If a student was projected to earn a 96 and he earns a 94, have I truly failed at my job? It seems that without a standard deviation, 5 months of work is boiled down to just a few questions. What is the incentive to teach an honor's class if there is less room for growth and more room for a decrease in achievement?

Second, I am wondering where the benchmark assessments fit into the picture. My students showed a lot of growth from the benchmark assessment to the EOC and yet that doesn't seem to be taken into account.

Finally, what other factors are being discussed as far as judging teacher performance is concerned? I am a fan of the new portfolio system, but at the end of the day, it seems that the only thing that still truly matters is how your students do on the EOC compared to what this "magical" formula, which no one seems to be able to explain to me, predicts they will do.

As someone who has attended some of the meetings on teacher effectiveness, I am hoping that you could provide a little clarity for me on some of these issues. Thank you very much for your time.