The Honorable the Members of the Board of Regents


James A. Kadamus




Overview of the Grades 3-8 Testing Program


October 19, 2004




Implement Regents Policy


Goals 1 and 2






This report provides an update on New York State’s grade-by-grade testing program that is being implemented to comply with the No Child Left Behind (NCLB) Act.  It is one in a series on assessment issues that have been scheduled on the Committee's upcoming agendas. It updates the report provided to the Board in September 2004. 




The New York State Testing Program (NYSTP) is designed to evaluate the implementation of the State Learning Standards at the student, school, district, and statewide levels.  The Department currently tests students in English Language Arts (ELA) and mathematics in grades 4 and 8 and at the commencement level with the Regents examinations.  To meet the new mandates under NCLB, tests in ELA and mathematics are to be administered annually at every grade level from grade 3 to grade 8.


NCLB Testing Requirements


NCLB specifies that statewide tests must:

·        address the depth and breadth of the state content standards;

·        be valid, reliable, and of high technical quality; and

·        be designed to provide a coherent system across grades and subjects.


            With the exception of the small population of severely disabled students who qualify to take the State’s Alternate Assessment, all students, both general education and special populations, in all public school districts, will be required to take the exams. While the decision to test in grades 3-8 was initially mandated by the federal government, these new tests also present the opportunity to:

·        annually evaluate the implementation of the New York State Learning Standards;

·        measure individual student and cohort progress; and

·        gather data on student readiness for study at the next level. 


Philosophical Basis for Tests


            A number of test designs were considered to respond to the federal mandate for testing in grades 3-8.  A Committee of Practitioners was formed to guide the Department's thinking on the test design.  The Committee rejected the idea of developing multiple choice tests for grades 3, 5, 6 and 7 that could be used only to supplement the current Regents accountability system.  Instead, the Committee stressed that tests in all grades must test the New York State learning standards and provide local administrators and teachers with data to help struggling students and help schools improve instruction.  They sought tests that clearly tied assessment, curriculum and instruction most closely.  However, these new tests must be administered and scored more efficiently than ever before.


            The State Education Department has adopted this philosophy in developing the new grades 3-8 tests.


Test Development Process


CTB/McGraw Hill (CTB) was awarded the contract to design, develop, field-test, standard-set and scale the new grades 3-8 testing program in ELA and mathematics. CTB is required to adhere to the New York State Education Department's Examination Development Process, which requires extensive involvement of New York State teachers and administrators. Teachers and administrators have served or will serve on a number of committees and will be an integral part of the test development process.


Test Specifications


The first step in test development was to create new specifications for test questions. Committees composed of New York State teachers, State Education Department staff, and staff from CTB met to ensure that the test questions would be aligned with State standards and classroom instruction. Information from these committees also provided guidelines for writing the test questions by selecting standards and performance indicators most appropriate for statewide assessment, and by identifying formats best suited for assessing those standards and performance indicators. The English Language Arts committee also reviewed the reading passages to be used in the tests.


Item Development


State-developed specifications were used, along with the State Learning Standards, to train and guide experienced writers at CTB in developing appropriate test questions.  Test questions were subjected to a rigorous editing and review cycle by CTB, and were then reviewed by select committees of teachers and administrators from throughout New York State and by Department staff. Questions were reviewed from several perspectives, including:

·       alignment to State Learning Standards;

·       contexts that were appropriate in both grade level and content;

·       a range of the depth of knowledge needed to answer questions;

·       clear and concise language;

·       content-specific issues, e.g., calculator use; and

·       appropriateness of genre for literary texts.


              Based on the recommendations of the review committees, test questions were accepted, revised, or rejected. A final review with Department staff will determine the final questions that will be selected for field-testing. After field-testing, only questions shown by data analysis to be valid and reliable measurements of the State Learning Standards are selected for inclusion on the final operational test forms.


Test Structure and Content


The tests reflect New York State content/process standards in each grade and subject area. They are designed to signal priority content and to be instructionally sensitive. That is, they should be able to be used by teachers and administrators to inform curriculum, instruction, and locally developed assessment.


Tests in both subjects will be similar in format to the existing grades 4 and 8 assessments; all tests (ELA and mathematics) will contain multiple choice, constructed response (short answer), and extended response (essay) questions.  The largest change is that the Independent Writing Prompt, an open-ended essay, on the ELA grades 4 and 8 exams has been removed.   Additional detail on the test designs for each subject and grade level can be found in the accompanying slides.


Administration Time


All efforts have been made in the test development process to limit the length of the tests while ensuring the validity and reliability of the assessment program. Based on current tests formats, the grades 4, 6 and 8 tests will be somewhat shorter than the current grades 4 and 8 tests, with the grades 3, 5 and 7 tests being about one-half the length of the current grades 4 and 8 tests.  We have analyzed the testing times of some typical off-grade tests currently used by schools, and we believe that the total testing time for all tests, grades 3-8, will be the same or even less than the time schools currently devote to State and off-grade testing.


Participation of Special Populations


Federal law requires all students to participate in State assessments regardless of any special needs. For students with disabilities, the school district’s Committee on Special Education determines how a student with a disability will be assessed and the majority of special education students participate in the regular assessments.  For severely disabled students, the State provides the Alternate Assessment. To guarantee appropriate representation for special population students, Office of State Assessment


staff have worked collaboratively with staff from VESID and Bilingual Education in the design of the new testing system. 


Educators working with special populations (students with disabilities and English language learners) have participated in the test specifications/framework process, item selection, and item review process. VESID staff have provided assistance on the principles of Universal Design for Learning, which we are working to ensure are used in the grades 3-8 test development process.


The principles of Universal Design for Learning include, but are not limited to:

·       development of instructional and assessment materials that are varied and diverse;

·       development of tests that do more than accommodate physical, sensory, or cognitive disabilities;

·       development of tests that are flexible and promote alternatives; and

·       development of tests that are inclusive.




Field-testing is a fundamental component of a high quality statewide testing program and the Department depends upon school participation. The results of student performance from a stratified sample of schools across the State will help the Office of State Assessment to develop the best test items for the resulting operational tests administered to all students. While the new NCLB testing requirements in grades 3-8 do not go into effect until 2006, field-testing at grades 3-8 needs to be conducted in 2005 so that the results and data collected from these items can be used as the basis for the development of the operational tests to be administered in 2006.


The Department is aware that some schools see field-testing as intrusive. The Department and CTB have taken many steps to limit schools’ field-testing responsibilities, while still collecting robust stratified samples from the field. The Office of State Assessment has designed a new Five-Year Field-Test Sampling Matrix for the Elementary and Middle School State Tests, similar to the system that has been successfully implemented for Regents examinations.


During the 2004-2005 school year, in addition to the operational English Language Arts and mathematics tests administered in grades 4 and 8, field-tests in both content areas will also be administered in grades 3, 5, 6 and 7. The table below lists the 2005 test administration dates for ELA and mathematics at each grade level.


Content Area

Grade Levels

Test Administration Dates

English Language Arts Field-Test

3 through 8

February 7 – 11, 2005

Mathematics Field-Test

3 through 8

March 14 – 18, 2005

English Language Arts Operational Test

4 and 8

January 31 – February 4, 2005

Mathematics Operational Test

4 and 8

May 10 -11, 2005




First Administration of Operational Tests


The grades 3-8 testing program is scheduled to begin during the 2005-2006 academic year. ELA will be administered in late January-early February 2006 and mathematics will be administered in March 2006. Testing on these dates will allow the Department the time necessary for the scoring and certification of the testing system, while still getting test results back to school districts in a timely manner.




We are still studying the best ways to score these tests and get results to schools as quickly as possible.  There are many challenges associated with scoring these tests in a way that preserves the reliability of the scoring while not unduly affecting instructional time.  Over the summer we issued a Request for Information related to printing, scanning and scoring these tests.   Based on the information we received, we are considering a number of models, including distributive scoring. Such a scoring model could allow the use of technology to alleviate some of the scoring burden on schools, while assuring that teacher expertise is not lost.  Under a distributive scoring model, test results would be scanned into a computerized database and teachers would be able to score essays or math problems by accessing them from an online database.  A number of factors, including cost, will be considered in deciding the most reliable and efficient process of scoring to use. 


Currently, CTB staff, Department staff, and members of the Department’s Technical Advisory Group are studying the design of the scaling system to be used for this new family of tests. The tests will still produce a scaled score that places a student into the 4 current performance levels: Level 1, Serious Academic Deficiencies; Level 2, Basic; Level 3, Proficient, and Level 4, Advanced. Furthermore, the Department is studying the possibility of creating scales that will measure individual student growth.   Such a scaling system, combined with the new individualized student information system, will give schools more precise information to track the progress of both cohort and individual student growth over time.


Reporting of Scores


            The Department will contract with a vendor to produce uniform score reports by school, classroom and individual students.  There are a number of such systems currently in existence in other states.  These basic score reports will be put in a computerized database to be accessed by authorized school personnel.


            The Department is currently studying ways to decrease the turnaround time from testing to reporting of scores.  A key factor to shorten the length of time from testing to reporting will be the type of scoring process used.  In addition, the implementation of the individual student information system should make recording of individual student scores easier.


            Turnaround time will also depend on better training of local school staff who enter data.  In our regional meetings on grades 3-8 testing, we will be emphasizing ways to make this reporting faster.


Accountability Implications


            We do not believe that using the results of tests in the additional grades will adversely affect schools, since they will continue to receive one accountability “score” per school.  However, there may be a slight performance “dip” at the beginning, especially in mathematics, where the development of the tests and the introduction of revised standards are happening simultaneously.  Therefore, we have begun discussions, including with USDOE, to make adjustments in the accountability system as needed to ensure that all schools are fairly treated. 


As we move forward with the planning and implementation of this testing program, we will continue to keep the Board of Regents fully apprised.