EMSC Committee

SUMMARY

Issue for Discussion

            Update on progress made to expedite the release of test scores for the Grades 3-8 Testing Program in English Language Arts (ELA) and Mathematics. This report is the first in a series of reports to the Board of Regents to help inform major policy decisions and provide an update on actions taken to dramatically improve our data and assessment system (in conjunction with the revision of the learning standards) as well as our school improvement and accountability system. This month’s report focuses on New York State’s assessment and data system. Prior to this discussion on data, the Board discussed phase one of the growth model (with phase two, the more comprehensive value-added model, to be discussed later this winter) and the move toward a differentiated accountability system.

One major goal in this broad reform is to reconsider all options and processes to make New York State’s assessment and data systems state-of-the-art and significantly more beneficial to students and educators. Improvement of the data system is possible through a partnership with ITS. The reformed system will more readily inform instruction and improve the ability of educators to help students learn.   

In addition to an update on progress with the release of 3-8 test results, the Board will be engaged in a policy discussion of future directions that include benchmarking other states, extensive feedback from educators, as well as processing and test administration options for the 21st Century including the exploration of test taking on computers. The work underway will examine the 3-8 tests and all State required exam administrations, including scoring and reporting as we move forward with a new improved system for NYS in conjunction with the revision of the Learning Standards. This discussion will also include options and considerations for revisions to the schedule that the Board of Regents may wish to discuss. 

Reason(s) for Consideration

           
            For Information

Proposed Handling

            This item will be discussed by the Regents EMSC Committee at its December 2008 meeting.

Procedural History

 During the May 2008 meeting of the Board of Regents, the Senior Deputy described a plan to significantly expedite the release of the Grades 3-8 ELA and Math test scores for the current system of data collection and reporting. While the Board of Regents and the Department are pursuing the overhaul of the current system, it is important that the existing system is improved simultaneously. The goal described in May was to establish a system so that the current administration of the 3-8 test scores could be returned within 10 weeks of the last make-up exam administered versus the lengthier and unreliable schedules experienced since the administration of this exam. The creation of a system to achieve this 10 week turnaround has been accomplished for this testing cycle. The work underway is to examine these tests and all State required exam administrations, scoring and reporting as we move forward with a new improved system for NYS.

Background Information

Earlier Reporting of Grades 3-8 Test Scores

It is critically important for the education of our children that educators and the public receive test score information in a way that is timely and reliable to help inform instruction. While the 3-8 test scores are not diagnostic in nature they drive instruction and the educational activities of schools.  In the first year of the Grades 3-8 tests, 2006, it took 33 weeks to release the ELA scores and 28 weeks to release the Math scores because new psychometric processes were required for the new tests.  Those unacceptable timelines have subsequently decreased.
 
In May 2008, the Regents discussed a plan to further reduce this timeframe to 10 weeks.  To implement the plan, we worked  extensively with the field, benchmarked other states and received input from our consultants. Other states’ timelines range from seven weeks in Ohio where questions are graded by a test vendor rather than by certified teachers to as long as 4 months. The change to the current system was achieved with the work and input of the District Superintendents, School Superintendents, the leaders of the Big 5 school districts, the Regional Information Centers (RIC’s), CTB/McGraw Hill and others.  We appreciate the flexibility and willingness of all parties to work together to reconfigure their internal processes and timelines to implement the recommended plan for this testing cycle. Beginning with the next Grade 3-8 exam administration, the Department will shorten the time for test scoring and scanning, psychometric equating, and quality assurance based on new systems now in place.  This will allow us to reliably report the test results within 10 weeks after the tests are administered, which is a significant cycle time improvement of reducing the reporting period by over 2 months from last year to as much as over 6 months from 2006.  This improvement is illustrated as in the charts below:

Current Scheduling of Grades 3-8 Tests and Options for Changing the Schedule
           
When the grade 4 and 8 tests were introduced in the 1998-99 school year, both the ELA and Math tests were administered in May. At that time, many school administrators asked for a time separation between the administrations of the ELA and Mathematics Tests.  To respond to this request from the field, the Grades 4 and 8 ELA tests were administered in January while the Math tests were then given in May.

In 2006, the expanded federal No Child Left Behind (NCLB) requirements for Grades 3-8 testing began.  The Grade 3-8 Testing Program is an assessment system designed to measure concepts, processes, and skills as defined by the State’s Learning Standards and Performance Indicators. The purpose is to measure the extent to which individual students achieve the New York State Learning Standards in English Language Arts and mathematics and to determine whether schools and districts meet the required progress targets specified in the accountability system.  Unlike the Regents high school examinations, the 3-8 testing program is designed to be a point in time summative assessment of a student’s progress to date, versus an end–of-course summative assessment.  Under NCLB, test scores need to be reported early enough to allow school accountability decisions to be made as early as possible, preferably before the next school year.

The Department continued to administer the ELA tests in January but moved the administration of the Math tests from May to March in order to provide more time to comply with the NCLB reporting requirements. The Regents have discussed the possibility of a policy change that would provide testing data earlier specifically exploring the option of changing the administration schedule of the Grade 3-8 tests from the current winter (January/March) administration to the fall or the spring similar to when most other exams are scheduled. If the Regents decide on the need for a different test schedule, staff will provide a plan to transition to a new schedule with appropriate feedback from the field, an assessment of best practices in other states and any estimated costs. 

            The following information is provided to help inform the policy discussion of the option of a Fall or Spring alternative testing schedule:

Fall Administration   (3-8 ELA in October / Math in November)

Advantages and Challenges:

• Earlier test administration will enable scores to be returned to school districts by mid year (or sooner with system changes)

• Earlier test administration would  enable more timely placement into Academic Intervention Services (AIS) for students who score below the State Cut Score for Proficiency and for other related instructional decisions

• Less overlap with other tests  

• Tests prior year’s skills (Some educators see as an advantage. Others believe that these tests should examine the current status of student skills, as long as the return of the scores is timely).

• Administered after summer break  -- 2 months after the end of instructional period

• A change in the timeframe of the assessment will require a revision of the math curriculum to ensure appropriate content coverage and associated professional development for instructional staff. 

• Adequate time to meet NCLB test score reporting requirements

Spring Administration (ELA and Math in June)

Advantages and Challenges:

• Responds to educators in the field who have suggested that these tests should be administered at the same time as other end-of-course tests including Regents Exams.

• This time frame would help younger students to become accustomed to end-of-course tests.

• Tests current year’s skills and permits more time to teach the next year’s curriculum prior to the administration of the next test.

• The 3-8 tests would be given during the same time period as other State tests-some overlap of 8th grade exams.

• A change in the timeframe of the assessments will require a revision of the math curriculum to ensure appropriate content coverage and associated professional development for instructional staff. 

• Need to explore options to current system of teachers scoring the open-ended questions, possibly the hiring a vendor to handle centralized scanning and scoring. For a quicker turnaround time to meet NCLB reporting requirements. 

• Based on industry research and costs incurred in other states, vendor costs could range from $15 - $25 million per year.   

We look forward to a conversation with the Board of Regents on this subject. We will also continue to speak with the field, survey other states, and benchmark with best practices so that we can explore all possibilities to help bring New York’s testing system into the 21st century. We will study all options including changing from the paradigm of students using pencil and paper tests to computerized testing.

Recommendation

That the Regents discuss whether they wish to pursue a transition for changing the test administration schedule for the Grades 3-8 Tests.  

Timeline for Implementation

           N/A