Meeting of the Board of Regents | April 2008
THE STATE EDUCATION DEPARTMENT / THE UNIVERSITY OF THE STATE OF NEW YORK / ALBANY, NY 12234
April 9, 2008
Goals 1, 2 and 4
Issue for Discussion
Are the actions to implement an interim growth model as required by Chapter 57 of the Laws of 2007 well aligned with Regents policy objectives?
Reason(s) for Consideration
Review of policy.
The question will come before the Regents EMSC Committee in May.
In April 2007, the Legislature passed and the Governor signed into law Chapter 57 of the Laws of 2007, which contains provisions regarding development of growth models. In early April 2007, the Commissioner provided a summary of the provisions of the law in his briefing memo to the Board of Regents on the enacted budget. At the February 2008 Board of Regents meeting, Dr. Brian Gong of the Center for Assessment in Nashua, New Hampshire presented on growth models. On February 28, 2008 another discussion on growth models took place in Syracuse, New York. At the March 2008 Regents meeting, an oral report on the Syracuse meeting was provided to the EMSC committee.
Chapter 57 of the Laws of 2007 states that by the start of the 2008-09 school year the Regents shall establish, using existing state assessments, an interim, modified accountability system based on a growth model, subject to the approval of the United States Department of Education. In turn, by the 2010-2011 school year, the Board of Regents will, subject to USDE approval and that of external experts in testing and measurement, implement an enhanced accountability system based on an enhanced growth model, using revised or new assessments and, where feasible, a value-added assessment model.
When No Child Left Behind was first enacted, states were not permitted to use growth models as the basis for determining whether schools or districts have made Adequate Yearly Progress (AYP). In November 2005, United States Education Department Secretary Margaret Spellings announced a pilot growth model project that would permit up to ten states to participate in the pilot. In December 2007, after nine states had had growth models approved, Secretary Spellings announced that all states would now be permitted to submit growth models for approval. States that wished to use a growth model based on 2007-08 school year data were required to submit a proposal by February 15, 2008. On February 15, 2008 Commissioner Mills wrote to Secretary Spellings informing her of New York’s intent to submit a Growth Model Proposal to the U.S. Department of Education (USED) for implementation in the 2008-2009 school year.
The attached document provides a summary of the purposes and uses of growth models, the seven core principles that a growth model must met in order to receive USED approval, the guidelines that Department staff have been using to inform the work of developing a growth model, the capacity of the Department to implement a growth model, and the timeline for completion of this task.
Staff recommends that the Regents endorse the strategies outlined in this item to implement the requirements of Chapter 57 of the Laws of 2007 related to the development of a growth model.
Timetable for Implementation
In May 2008, Department staff will present to the Board of Regents a draft growth model proposal. With the endorsement of the Regents, staff will then seek public comment on the draft growth model proposal. In July 2008, staff will bring to the Regents a proposed final growth model proposal. Upon Regents approval, the proposal will be submitted to the United States Department of Education. Once approved by USED, staff will submit to the Regents proposed amendments to Commissioner’s Regulations to implement the growth model for use with 2008-09 school year results.
Strategies for Implementing A Growth Model for New York State Schools and Districts
Certain information and requirements pertaining to growth models are already in law and USED policy.
What do Chapter 57 and the United States Department of Education mean by a growth model and what is a growth model intended to accomplish?
The term “growth model” is used by different persons to refer to different concepts. Most fundamentally, a growth model is a means to measure the change in the performance of students on specified assessments over time. Chapter 57 of the Laws of 2007 specifically defines a growth model as “the assessment of a cohort of students, or individual students, over time that measures the academic progress made by those students.” Chapter 57 further clarifies that the interim growth model to be established by the Board of Regents shall be implemented by the start of the 2008-09 school year, based on existing state assessments, designed to secure USED approval, and serve as a basis for the creation of an enhanced accountability system that incorporates to the extent feasible and consistent with Federal law a value-added assessment model.
A key question in the design of a growth model is to determine how “academic progress” over time is to be measured and how much growth “is enough.” Targets for growth can be policy driven, data driven, or a combination of both. Policy driven targets start with a policy goal (what should be) and then establish the targets for performance that are necessary to achieve this goal. Data driven growth targets start with historical performance data (what has been) and use that as a basis to project what should be expected of the units to be measured. Policy driven growth models often seek to hold schools and districts accountable for ensuring that students obtain a common standard (i.e. all students will be proficient) even though this creates differentiated challenges for the units of accountability in that some will find it easier than others to achieve this goal. Data driven growth models often seek to standardize across schools and districts the challenge of achieving the targets (i.e. schools and districts have an equal likelihood of achieving the targets regardless of student demographics or resource availability) by creating differentiated standards for acceptable student outcomes.
The “growth to standards” models, in which states, to receive USDE approval, must demonstrate that all students are either proficient or on track to proficiency as of the 2013-2014 school year, are examples of policy driven growth targets. To meet the requirements of Chapter 57 and secure USDE approval, New York’s interim growth model must be anchored in a methodology designed to measure whether students are either currently proficient or on track to obtain proficiency in English language arts and mathematics within a specified period of time.
What are the United States Seven Core Growth Model Principles?
In order to receive approval from USDE to implement a growth model, states must demonstrate that their proposals adhere to the following seven core principles:
- Ensure that all students are proficient or on track to proficiency by 2014, and set annual goals to ensure that the achievement gap is closing for all groups of students;
- Set expectations for annual achievement based on meeting grade-level proficiency, not on student background or characteristics;
- Hold schools accountable for student achievement in reading / language arts and mathematics separately;
- Ensure that all students in tested grades are included in the assessment and accountability system, hold schools and districts accountable for the performance of each student subgroup, and include all schools and districts;
- Include assessments in each of grades 3-8 and in high school for both reading/language arts and mathematics that they have been operational for more than one year and have received approval through the NCLB peer review process. The assessments must also produce comparable results from grade to grade and year to year;
- Track student progress as part of the state data system; and
- Include student participation rates and student achievement on a separate academic indicator in the State accountability system.
Given these parameters above, there are still many policy questions that must be answered to guide development of the interim growth model. For example, how do the Regents define growth? What is the purpose of a growth model of accountability? How will the interim growth model meet federal criteria for approvable growth models? Since the passage of Chapter 57 of the Laws of 2007, we have been involved in internal discussions, meetings with interested constituent organizations, and consultations with recognized national experts on growth modeling and assessments. In particular, we have received extensive technical support from Dr. Brian Gong, Executive Director of the National Center for Assessment, and Dr. Robert Linn of the University of Colorado. Scott Palmer and Amy Starzynski of Holland and Knight have also provided insights into how New York can build a robust growth model that adheres to the requirements stipulated by USED. Based on these discussions, Department staff have developed these proposed questions and answers for designing a growth model that we believe will fulfill Regents P-16 strategy and meet Chapter 57 and USED requirements:
- What should be the purpose of New York's interim growth model?
Hypothesis. New York State's interim growth model should be designed to further enhance the accuracy and utility of state accountability determinations by adding to current status measures a focus on the extent to which students are making significant progress toward proficiency, with a particular focus on closing achievement gaps toward proficiency. The interim growth model’s purpose is to permit us to make more refined determinations of student progress, identify with greater precision schools and districts that are high performing and rapidly improving, and support greater differentiation in the interventions and services provided to schools and districts identified for improvement.
- How should New York's interim growth model fit federal USED requirements?
Hypothesis. New York State's interim growth model should be designed to best serve New York's core education goals while also acting within (and pushing as necessary) federal growth model requirements to promote USED approval (as required by state law).
The primary purpose of those growth models that have been approved by the USED is to allow states to make better determinations as to which schools and districts should be designated as making Adequate Yearly Progress. The USED has recognized that there are schools and districts in which the percentage of students who are proficient may be low but the percentage of students who are demonstrating growth towards proficiency may be high. The combination of using a status model (i.e. comparing the performance of this year’s students in grade four to performance of the students last year in grade four) with a growth model (i.e. comparing the performance of students this year in grade four to their performance last year when they were in grade three) allows for a fuller, more richer analysis of school performance. This, in turn, permits the Department, districts, and schools to develop more nuanced and differentiated support and intervention strategies depending on both the level and rate of growth of student performance.
3. How will the interim growth model pave the way for a value added approach to accountability?
The interim growth model is intended to be the first stage of a two stage process that leads, by 2011-12, to the development of a more comprehensive system that, consistent with federal requirements and the requirements of Chapter 57, includes a value added model. The interim growth model is intended to support these purposes. Over time, work will also go forward to build upon the interim growth model a value added component. A value added component allows us not only to determine how much growth has occurred but also to associate or attribute this growth to various factors or conditions. Combined with other evaluation methodologies, a growth model can help districts and schools to determine which programs, strategies, and interventions are effective with which groups of students.
- How should New York's interim growth model look at student growth across all performance levels?
Hypothesis. New York's interim growth model should create incentives for promoting continuous growth for all students, including higher performing students who are above proficiency, while maintaining the focus for school AYP on promoting significant progress among the lowest performing students.
- What other considerations should guide the development of a growth model?
- The interim growth model will be based upon current state assessments and will not require the collection of any new data from school districts.
- The first year in which AYP decisions will include a growth measure will be those based on 2008-09 school year results.
- The interim growth model will be based on an open architecture; that is SED will publish exactly how it calculates growth decisions and the result will be a single, clear, unambiguous determination of AYP for each English language arts and math accountability criterion.
- The interim growth model will allow additional schools and districts to make AYP; no school or district will be at risk of failing to make AYP because of the addition of a growth component. However, the determination of which schools are designated as high performing or rapidly improving may change as the result of implementation of the interim growth model.
Does the SED have the capacity necessary to implement a growth model?
For the past two years, SED has been challenged to provide schools and districts with timely and accurate accountability determinations. The implementation of a growth model will add to the complexity of making AYP decisions for schools and districts. In order to implement a growth model, SED will need to compare the performance at two points in time for each student to be measured and determine whether that student is proficient or on-track for proficiency. Substantial effort will be required on the part of SED staff to compile very specific and explicit business rules, both for how the accountability rules will work and how those rules will be reflected in the necessary computer programming; review prototypes and final products; and explain the rules and calculations to the field, as well as providing ongoing technical assistance as the system is implemented. This analysis and establishment of business rules will be affected by the size and complexity of the student and school situations and the need to create a system that is both fair and understandable As part of this work SED will have to modify output reports so that they can provide information regarding both status and growth.
This system requires a change from working with approximately 700 school districts and approximately 4,400 school buildings to working with 2.9 million student records over multiple years. Even given these challenges, SED should be able to successfully implement a growth model because the task, while complex, is largely self-contained. Implementation of the interim growth model requires no modification to the state assessment system and no need to collect new information or to collect information in a different way, though data will need to be analyzed differently. Although the costs for designing and implementing a module to make growth determinations will be less than the total costs of administering the State’s standards, assessments, and accountability system, they will not be inconsequential. Furthermore, since determinations using a growth model will first be made in the summer of 2009, the Department should have sufficient time to complete the work, so long as USDE approval of the model occurs during the Fall of 2008 or Winter of 2009.
If we were to bring this process back to SED, additional professional and technical staff would be needed, not only to implement the model, but to provide intensive training and technical assistance, as well as ongoing assistance to districts as they endeavor to understand this new system and how calculations are made. For example, the additional staff may have to include accountability program experts, data analysts and computer programmers. We would also have to ensure that SED has the capacity to provide intensive training and technical assistance, and provide the training and district assistance described above.
In summary, implementation of the growth model will require crucial additional resources. Should the Department seek to do this work itself, which appears to be the preferred option, these additional staff resources would be needed before we begin the start-up phase of the process. Absent additional resources, the Department may not be able to complete this assignment in addition to fulfilling its present work commitments.
What is the timeline for the development of the growth model?
With the consent of the Board of Regents, staff propose the following timeline:
April 2008 SED staff provide Board of Regents with briefing on development of the growth model
May 2008 SED staff provide Board of Regents with draft growth model and request approval to seek public comments
May& June 2008 SED staff engage in discussions with interested constituencies on the proposed growth model
July 2008 Full discussion at the Board of Regents meeting on the Growth Model; Approval of Model Requested.
If approved by Board of Regents, staff will submit model to the United States Department of Education and request that it be peer reviewed.
Fall 2008 Upon receipt of approval by the United States Department of Education, the Board of Regents will adopt regulations, either by emergency or regular action depending on the date USDE approval is received, to implement the growth model for accountability in the 2008-2009 school year.
2008-20009 Implement Growth Model, using 207-08 school year data as base and 2008-09 school year data to measure growth
2009-2010 Use data from 2008-2009 assessment results to make AYP decisions for schools and districts.