View Video Transcript
Developing reliable evaluators of teachers is a key component to the success of any school district’s teacher evaluation system. Teachers and students benefit from principals and evaluators who are trained and certified to observe, analyze and rate instruction before they ever step into a classroom to observe a teacher for the purposes of high-stakes evaluation.
My name is Colin Ripmaster. I’m MASSP’s Associate Executive Director.
Research is clear that in-depth Rater Reliability training should happen annually. Calibration Tune-Ups should occur regularly to ensure content clarity and reduce the natural drift in understanding and practice that occurs the further removed evaluators are from initial training.
Greater reliability occurs when an observer of instruction uses an evaluation rubric to observe a lesson and can consistently score each rubric indicator in alignment with expert raters.
MASSP, in partnership with the Center for Educational Leadership, offers 5D+ Rater Reliability Training Institutes to provide opportunities for principals and district staff to become certified using the 5D+ Teacher Evaluation Rubric.
Participants who successfully complete the institute are certified in their ability to use a video observation to align instructional practice to the 5D+ Rubric. These sessions engage leaders in discussions of best practices in evaluation and calibration and provide significant opportunities to practice scripting, coding and rating classroom videos.
Through discussion with their colleagues, administrators become increasingly aligned in their calibration to the rubric.
There are multiple Rater Reliability series. Series 1 is for first-time attendees and is three days.
Day 1 focuses on Rater Reliability protocols and research, scripting and coding practices, identifying evidence connected to the five dimensions and 5D+ indicators using video observations, and scoring indicators with 35% or greater accuracy.
Day 2 continues to develop effective scripting and coding practices, identifying evidence connected to indicators within the rubric using video observations, and scoring with at least 55% accuracy.
Day 3 further refines effective scripting and coding practices and participants’ ability to identify evidence connected to indicators within the 5D+ Rubric through video observation. Successful participants achieve 70% or higher accuracy in scoring the 25 classroom practices within the 5D+ Rubric.
Follow-up Rater Reliability series are only two days, paralleling Days 2 and 3 of Series 1 with new videos. Calibration Tune-Ups are designed to be completed more regularly in shorter chunks of time across the year in half- or full-day intervals.
Educational experts agree that quality teaching is the single most important factor for improving educational outcomes for all students, followed closely by building administrators.
Rater Reliability provides a safe and engaging space for school and district leaders to hone their knowledge and skills in order to cultivate the expertise of teachers to deliver high-quality instruction for all students.
To learn more about MASSP and how we might work with your team to deepen understanding and strengthen your practice, please contact our office.
MCL 380.1249(5) is a new requirement for each individual who conducts an evaluation of one or more teachers to complete rater reliability training by not later than September 1, 2024, and every 3 years thereafter.
Developing reliable evaluators of teachers is a key component to the success of any school district’s teacher evaluation system. Teachers and students benefit from principals and evaluators who are trained and certified to observe, analyze and rate instruction, before they ever step into a classroom to observe a teacher for the purpose of evaluation.
Districts must collectively bargain the type of rater reliability training required locally, and whether or not certification is expected. As the authorized provider of the University of Washington’s Center for Educational Leadership 5D+ Rater Reliability training, MASSP provides two options to comply with MCL 380.1249(5):
- Certification Option: 5D+ Rater Reliability Certification Training
- 3-Day Initial RR Certification Training
- 2-Day Recertification Training
- Non-Certification Option: Calibration Tune Ups
- 1-Day Non-Certification Training
5D+ Rater Reliability Certification (18 SCECHs)
5D+ Rater Reliability training provides opportunities for principals and district staff to become certified as reliable evaluators using the 5D+ Teacher Evaluation Rubric. These sessions engage leaders in discussions of best practices in evaluation and calibration, and are a significant opportunity to practice scripting, coding and rating classroom videos. Through discussion with their colleagues, evaluators become increasingly aligned in their calibration to the rubric. *Prerequisite: All six days of 5D+ Framework training and implementation.
- Day 1: 5 Dimensions of Teaching and Learning; 5D+ Teacher Evaluation Rubric examples, Rater reliability protocol; Scripting and coding practices; Identifying evidence connected to 5D dimensions and 5D+ indicators using video observations. Match scripted evidence to 5D+ indicators with 35% accuracy.
- Day 2: Continue to develop and redefine effective scripting and coding practices; Continue identifying evidence connected to 5D dimensions and 5D+ indicators using video observations. Match scripted evidence to 5D+ indicators with 55% accuracy.
- Day 3: Continue to develop and redefine effective scripting and coding practices; Continue identifying evidence connected to 5D dimensions and 5D+ indicators using video observations. Match scripted evidence to 5D+ indicators with 70% accuracy.
Cost: $3,500/full day (contracted in-district; $10,500 total) or $650/person at a host site or online via Zoom
View Rater Reliability Certification Trainings
5D+ Rater Reliability Recertification (12 SCECHs)
5D+ Rater Reliability Recertification training provides opportunities for principals and district staff to re-certify as reliable evaluators using the 5D+ Teacher Evaluation Rubric. These sessions engage leaders in discussions of best practices in evaluation and calibration, and are a significant opportunity to practice scripting, coding and rating classroom videos. Through discussion with their colleagues, evaluators become increasingly aligned in their calibration to the rubric. *Prerequisite: 5D+ Rater Reliability Certification training and implementation.
- Day 1: 5D and 5D+ Rubric Revisited; Rater reliability research, challenges and protocol. Scripting and coding practices; Identifying evidence connected to 5D+ indicators using video observations. Match scripted evidence to 5D+ indicators with >55% accuracy.
- Day 2: Continue to develop and redefine effective scripting and coding practices; Continue identifying evidence connected to 5D dimensions and 5D+ indicators using video observations. Match scripted evidence to 5D+ indicators with >70% accuracy.
Cost: 2 Full Days: $7,000/in-district, 2 days In-Person at a host site or online via Zoom: $450/person
View Upcoming Rater Reliability Recertification Trainings
Calibration Tune-Up (6 SCECHs)
Calibration Tune-ups provides opportunities for principals and district staff to observe instruction, rate evidence, and engage in calibration of ratings to deepen content clarity and increase internal reliability in scoring evidence using the 5D+ Rubric. Through discussion with their colleagues, evaluators become increasingly aligned in their calibration to the rubric.
Cost: $3,500/full day (contracted in-district) or $250/person at a host site or online via Zoom