EMSC Committee

SUMMARY

Issue for Discussion

Is the attached proposal for an interim growth model as required by Chapter 57 of the Laws of 2007 well aligned with Regents policy objectives?

Reason(s) for Consideration

For Information
             

Proposed Handling

The question will come before the Regents EMSC Committee in June.

Procedural History

In April 2007, the Legislature passed and the Governor signed into law Chapter 57 of the Laws of 2007, which contains provisions regarding development of growth models.  In early April 2007, the Commissioner provided a summary of the provisions of the law in his briefing memo to the Board of Regents on the enacted budget. At the February 2008 Board of Regents Meeting, Dr. Brian Gong of the Center for Assessment in Dover, New Hampshire presented on growth models.  On February 28, 2008 another discussion on growth models took place in Syracuse, New York. At the March 2008 Regents Meeting, an oral report on the Syracuse meeting was provided to the EMSC Committee. During the April 2008 Board of Regents Meeting, staff presented a discussion item outlining the primary policy questions that the growth model should address; the Board of Regents approved these questions and directed staff to develop the model for the Board of Regent’s review.

Background Information

The term “growth model” is used by different persons to refer to different concepts.  Most fundamentally, a growth model is a means to measure the change in the performance of students on specified assessments over time.  When No Child Left Behind (NCLB) was first enacted, states were not permitted to use growth models as the basis for determining whether schools or districts have made Adequate Yearly Progress (AYP). In November 2005, United States Education Department (USED) Secretary Margaret Spellings announced a pilot growth model project that would permit up to ten states to participate in the pilot.  In December 2007, after nine states had had growth models approved, Secretary Spellings announced that all states would now be permitted to submit growth models for approval.  States that wished to use a growth model based on 2007-08 school year data were required to submit a proposal by February 15, 2008.  (On February 15, 2008 Commissioner Mills wrote to Secretary Spellings informing her of New York State’s intent to submit a Growth Model Proposal to the USED for implementation in the 2008-09 school year.)

Chapter 57 of the Laws of 2007 requires that by the start of the 2008-09 school year the Board of Regents shall establish, using existing state assessments, an interim, modified accountability system incorporating a growth model, subject to the approval of the USED.  In turn, by the 2010-11 school year, the Board of Regents will, subject to USED approval and that of external experts in testing and measurement, implement an enhanced accountability system incorporating an enhanced growth model, using revised or new assessments and, where feasible, a value-added accountability model.

Chapter 57 specifically defines a growth model as “the assessment of a cohort of students, or individual students, over time that measures the academic progress made by those students.”   A key question in the design of a growth model is to determine how “academic progress” over time is to be measured and how much growth “is enough.”  Targets for growth can be policy driven, data driven, or a combination of both.

• Policy driven targets start with a policy goal (what should be) and then establish the targets for performance that are necessary to achieve this goal. Policy driven growth models often seek to hold schools and districts accountable for ensuring that students obtain a common standard (i.e., all students will be proficient) even though this creates differentiated challenges for the units of accountability in that some will find it easier than others to achieve this goal.

• Data driven growth targets start with historical performance data (what has been) and use that as a basis to project what should be expected of the units to be measured.  Data driven growth models often seek to standardize across schools and districts the challenge of achieving the targets (i.e., schools and districts have an equal likelihood of achieving the targets regardless of student demographics or resource availability) by creating differentiated standards for acceptable student outcomes.

The “growth to standards” models, in which states, to receive USED approval, must demonstrate that all students are either proficient or on track to proficiency as of the 2013-14 school year, are examples of policy driven growth targets.  To meet the requirements of Chapter 57 and secure USED approval, New York State’s (NYS’s) interim growth model must be anchored in a methodology designed to measure whether students are either currently proficient or on track to obtain proficiency in English language arts (ELA) and mathematics within a specified period of time.

              Working with Scott Palmer and Amy Starzynski of EducationCounsel in affiliation with Nelson Mullins and Brian Gong of the Center for Assessment, staff developed a proposal for an interim growth model.  This proposal complies with the current requirements established by the USED regarding participation in the Growth Model Pilot.   

              The attached document provides an overview of the core principles of the proposed growth model, a more detailed description of how growth calculations would be made and incorporated into school accountability decisions, and a question and answer document that addresses the primary issues associated with the development and implementation of the growth model. The document also provides a strategy for outreach to interested constituencies and provides a modified timeline for the development of the growth model.  

As you review this proposal, a key point to remember is that there are many possible growth models that are designed to serve different purposes.  The State’s interim growth model is designed to support the Board of Regents policy to close the achievement gap and improve student performance.  This model gives schools and districts credit for students making sufficient progress towards becoming proficient and meeting NYS learning standards.   The NYS Grades 3-8 growth model methodology has many similarities to the North Carolina model that has been approved by the USED.  The high school growth model employs a value tables approach that has been used by states such as Delaware and also been approved by USED. However, at both the elementary-middle and high school levels, the models contain unique features specifically designed to make the model the most appropriate one for use by NYS.

Other prominent growth models are designed for other purposes.  For example, the New York City growth model compares schools primarily against like schools within New York City, and the Sander’s value-added model (Education Value Added Assessment System) used by some districts in the state is designed primarily to attribute growth to particular teachers and schools.  As a result, different growth models may draw different conclusions from growth data. For example, a school may have a large percentage of students who are demonstrating higher rates of growth than students in schools with similar student populations. This information might be one factor to consider in judging the effectiveness of a principal in relation to his peers or the efficacy of certain targeted interventions. However, even though the students in this example are showing strong growth compared to students in “similar” schools, the students may not be on track to become proficient within the next several years. In this circumstance, a model such as New York City’s may give this school a high rating because of its relative performance while a model based on NCLB growth principles may still conclude that the school is not making AYP towards the goal of having all students proficient or on track to proficiency by 2013-14.

Recommendation

Staff recommends that the Regents endorse the principles outlined in this item to implement the requirements of Chapter 57 of the Laws of 2007 related to the development, review, and implementation of a growth model.

Timetable for Implementation

              With the endorsement of the Board of Regents, staff will seek public comment on the draft growth model proposal. In July 2008, staff will submit a final growth model proposal to the Board of Regents.  Upon Board of Regents approval, the proposal will be submitted to the USED. Given USED’s likely schedule for review of state submissions, it is important to keep to this timeline. Once approved by USED and after the Board of Regents have determined that sufficient resources are available for the Department to implement the growth model in a timely and accurate manner, staff will submit to the Board of Regents proposed amendments to Commissioner’s Regulations to implement the growth model for use with 2008-09 school year results.

Overview of New York’s Proposed Growth Model

Introduction and Background

              In April 2008, staff proposed that the development of an interim growth model be guided by the following principles:

1. NYS's interim growth model should be designed to further enhance the accuracy and utility of state accountability determinations by adding to current status measures a focus on the extent to which students are making significant progress toward proficiency, with a particular focus on closing achievement gaps toward proficiency.
2. NYS's interim growth model should be designed to best serve NYS's core education goals while also satisfying federal growth model requirements to obtain USED approval.
3. The interim growth model is intended to be the first stage of a two stage process that leads, by 2010-11, to the development of a more comprehensive system that, consistent with federal requirements and the requirements of Chapter 57, includes a value-added model.
4. NYS's interim growth model should create incentives for promoting continuous growth for all students, including higher performing students who are above proficiency, while maintaining the focus for school AYP on promoting significant progress among the lowest performing students.
5. The interim growth model should be based upon current state assessments and will not require the collection of any new data from school districts.
6. The first year in which AYP decisions will include a growth measure will be those based on 2008-09 school year results.
7. The interim growth model should be based on an open architecture; that is the New York State Education Department (NYSED) will publish exactly how it calculates growth decisions and the result will be a single, clear, unambiguous determination of AYP for each English language arts and mathematics accountability criterion.
8. The interim growth model should result in additional schools and districts making AYP; no school or district will fail to make AYP due to the addition of a growth component. However, the determination of which schools are designated as high performing or rapidly improving may change as the result of implementation of the interim growth model.

To accomplish these goals, staff proposes to:

• Submit to USED a “proficiency plus” growth model for Grades 3-8 with enhancements for middle school and high school growth.  A “proficiency plus” model is one in which, for AYP purposes, schools get “full” credit for all students who are either proficient or on track to proficiency. This model is sufficiently similar to the North Carolina model as to give NYSED high expectations that USED would approve the model.
• Require in the basic model that students be proficient within four years of first NYS test administration or Grade 8, whichever is earlier.
• Initially use two years of test data to establish growth trajectories. Over time, consider using more years of data to establish growth trajectories.
• Establish growth trajectories only for students who have valid test scores in successive grades over two years.
• Seek USED approval, simultaneous with the submission of the “proficiency plus model” to implement an “enhanced” Grades 3-8 growth model that:
• Allows new growth trajectories to be established when students enter middle school; and
• Allows these growth trajectories to be based on whether students are on track to be proficient on the Regents Comprehensive Examination in English and the Regents Examination in Integrated Algebra rather than by the end of Grade 8.

These “enhanced” model provisions are ones that NYSED believes are consistent with USED growth model principles, although they are ones that are not contained in the growth models that USED has thus far approved.

• Submit to USED a “value table” model for use for high school accountability in which students who enter high school as Level 1 or low Level 2 students and who pass Regents examinations with a score between 55-64 prior to their senior year will be deemed to be “on track towards proficiency” and credited to schools as students making sufficient growth.
• Incorporate a “growth for all” measure into the process for identification of high performing and rapidly improving schools and districts. 

We believe that integrating the basic model into our existing system combined with the middle and high school provisions built into our enhanced model would give New York State one of the best NCLB growth models in the nation. This growth model will help us move forward as we begin development of a value-added model.

Core Principles of the Growth Models

              The general approach of the “Proficiency Plus Model” we propose is:

1. If a student scores proficient or above in the current year, include that student’s results in the Performance Index as is done under the present status model. 
2. Use growth to check whether students who did not yet score proficient have grown enough that it is likely they will become proficient within a designated amount of time. 
3. For purposes of calculating the Performance Index, give schools and districts “full credit” for any student who either scores proficient or above or who is deemed to be on track for proficiency.

Overview of Grades 3-8 approach:

1. Calculate two scores for each non-proficient student: the observed growth score (based upon two successive years’ of data) and the projected growth score (reflecting where the student will likely be if s/he continues to grow into the future). 
2. Use the projected growth score to determine whether the student would likely be proficient in the future.

Details of Grades 3-8 “Proficiency Plus” Approach:

• For non-proficient students, annually calculate observed growth score based on the most recent two successive years of data.  Project where that student will be after four additional years of growth, or Grade 8, whichever is less.  This is the student’s projected growth score.
• Annually evaluate whether the student will be proficient at the designated grade level based upon the student’s projected growth score.  If so, the school gets credit for the student being on-track to be proficient.  If not, the school does not get credit for the student being on-track to be proficient.
• Make Grade 8 the maximum target for students in this basic proposal, so as students move through the elementary grades they will have correspondingly fewer than four years projected growth to achieve proficiency.
• Each student has a maximum of four years of projected growth from the time s/he has an initial growth score calculated; that is, the student must be proficient within four years of projected growth for the school to receive credit for the student’s growth.

Special Cases

• Growth will be calculated only for students for whom there are two valid scores from the State’s regular assessment in adjacent grades in two successive years.  Students who have one valid score will be included in the accountability system as they would have, without growth.
• Students who participated in the NYS Alternate Assessment will not be included in calculations for growth; status will be used for these students.
• Students with limited English proficiency who participated in the English language proficiency assessment only will not be included for growth.

Middle School Enhanced “Proficiency Plus” Accountability Extension

              For school accountability to be meaningful, it must reflect the contributions of the school, or be something that the school can influence.  This Middle School Enhanced “Proficiency Plus” Accountability Extension addresses the following two issues in the basic proposal to make growth relevant to middle schools:

1. Every student would have a maximum of four years of projected growth from the time the first observed growth score was calculated.  That effectively means that all students who enter the system in Grade 3 have until Grade 8 to be proficient, but as students are promoted through the grades they have fewer years to become proficient. 

1. There simply is not enough time to make sufficient growth if all students have to be proficient by Grade 8, if they are beginning in a school in Grade 6.  Students in the elementary grades had three years to become proficient; a student whose baseline in the middle school was Grade 6 (growth from Grade 5 to Grade 6) would have only two years to become proficient.

To address these two shortcomings and make growth an equally relevant part of school accountability for middle schools, we propose the following:

• Students in middle school would be evaluated on whether they made sufficient growth to become proficient by the designated high school Regents examination.
• For the purposes of this growth accountability for AYP, the designated high school target is proficient on the Regents Examination in Integrated Algebra and proficient in the Regents Comprehensive Examination in English.
• Students in middle school would have until the target assessment to be projected proficient; the number of years permitted would be based upon the grade the student entered middle school. 
• This middle school extension will only apply to schools (and their subgroups), not to district AYP decisions in instances where students transfer among schools within a district.

The other aspects of calculating growth scores and incorporating them into AYP decisions are the same as in the “Proficiency Plus” Proposal.

High School Accountability Value Table Model

              The majority of states have not been approved for a high school growth model under the USED Growth Model Pilot.  This is largely because states lack the adjacent grade testing in high school and/or are using end-of-course exams which lack census testing of students in any one year.  NYS has both of these conditions, but has a strong and respected student-level accountability system built upon its expanded Regents examination system.  NYS strongly desires to extend growth accountability to high schools; this value table proposal is a possible way to do that within the constraints of the current assessment system. Staff proposes that the high school growth model:

• Allows students who enter high school having scored on Level 1 or at low-Level 2 on the Grade 8 ELA or Mathematics Tests to be considered on track towards proficiency if the student is able to score between 55-64 on the Regents Examination in Integrated Algebra and Regents Comprehensive Examination in English prior to their senior year. For purposes of the Performance Index, schools would receive “full credit” for the performance of these students.
• Allows schools to have five years for certain limited-English-proficient students, certain students with disabilities, and students who enter high school far below standards to demonstrate proficiency in English language arts and mathematics.

              These two modifications to NYS’s current accountability system will recognize high schools that are successful with newly enrolled low-performing students and will ensure the success of certain students who require an additional year of high school.

              This proposal uses an existing performance level that is meaningful within the NYS context.  Students must score 65 or above on a set of designated Regents examinations in order to qualify for a Regents diploma.  Students who score 55-64 may receive a local diploma.  The model gives schools credit for students achieving a score of 55-64 since that represents significant growth from the students’ performance in Grade 8, and represents a rate of growth that would likely result in the student achieving a 65 or better on the Regents examination if taken at the end of high school.  

An Example of How the Grades 3-8 “Proficiency Plus” Growth Model Might Work

Assume the following, on the Grades 3, 4, and 5 Mathematics Tests, the hypothetical scale scores are translated into performance levels as follows:

Level 1:             Below 620

Level 2:             620

Level 3:             650

Level 4:             690

              In Grade 3, Student A scores 614 or Level 1.  Since there is no basis for making a growth determination, Student A’s status is “not proficient” and his score is included in the Performance Index as a Level 1 score.

              Student A is now given a four year growth trajectory (this is a simplified version of the actual psychometrics that will be employed), requiring that he obtain proficiency by Grade 7.  Currently there is a 36 point gap between the scale score for proficiency (650) and Student A’s score of 614.  If Student A gains 9 scale score points each year, he will obtain proficiency by Grade 7. Therefore, to be on track toward proficiency, Student A must earn a scale score of 623 or better on the Grade 4 Mathematics Test.

              In Grade 4, Student A receives a score of 623 or Level 2. Because Student A is now considered on track towards proficiency, his school will get credit for Student A’s performance as being equivalent to a Level 3 Performance rather than a Level 2 performance.  This will raise the school’s Performance Index and make it more likely that the school will make AYP. For example, let us say there are 100 students in Student A’s school in the low-income group and their scores distribute as follows:

              In the example above, because 1 student who is Level 1 and 4 students who are Level 2 made enough gain to be considered on track towards proficiency, the school’s Performance Index increases from 130 to 136.  If this Performance Index places the school’s low-income group above its Effective Annual Measurable Objective, the school would be able to make AYP for that group.

              Since Student A’s performance in Grade 4 is 623, a new trajectory for being on track towards proficiency will be established for Student A. Now Student A must gain 27 scale score points in three years to be proficient by Grade 7 (650-623=27).  Since Student A has three years to make this 27 point gain, Student A is expected to make a gain of 9 scale score points per year (27/3=9). Therefore, Student A is “on track” for his proficiency target in Grade 5 will be 632 (627+9=632).

              In Grade 5, Student A achieves a scale score of 630, or Level 2.  Since Student A did not achieve his “on track” towards proficiency target of 632, his score will be included in the school’s performance index as a Level 2 score.  Student A now has two years to make up the remaining 20 point difference, so his Grade 6 target will be 640 to be included in the Performance Index as a Level 3 score.

An Example of How the Middle School Enhanced “Proficiency Plus” Extension Might Work

              To continue the previous example, Student A gained 9 points between Grades 3 and 4, and 7 points between Grades 4 and 5, and has 20 points to grow to be proficient by Grade 7.  When Student A enters middle school in Grade 6, under the basic “Proficiency Plus” Grades 3-8 growth model, there is tremendous demand for the middle school to help the student make up the difference that exists in part because the student did not grow as needed in the elementary school. 

              The middle school enhanced extension provides a fair incentive for the middle school and holds it accountable for the student’s growth towards proficiency.  In the enhanced extension, the student’s growth target is set four years out from entry into middle school. Thus the student is expected to grow enough to become proficient by Grade 10.

              When Student A enters middle school in Grade 6, Student A is given a four-year target based on becoming proficient, for example, in Integrated Algebra by Grade 10.  Assume that passing that end of course examination has a comparable scale score of 666.  Based on Student A’s Grade 5 scale score of 630, Student A has to grow 9 points each year.  The middle school is held accountable for helping Student A grow 9 points each year rather than the 10 previously given.

              NYSED staff is aware of the technical challenges associated with creating a projection model that is based on using the high school end of course examinations. We are engaged in additional research with Brian Gong to explore the feasibility of using this approach; if there are too many complications, we will use only the Grades 3-8 Testing Program data and use a different projection methodology that ends with the Grade 8 Tests.

An Example of How the High School Values Table Might Work

•               Assume that Student A for some reason makes almost no progress in middle school, and still is a low Level 2 in Grade 8.  The middle school has been held accountable for Student A’s performance.  But how should the high school be held accountable for Student A’s growth? This approach provides credit to schools for students who scored very low in Grade 8 and who scored at least at the level of qualifying for a local diploma. In the proposed high school extension, the high school gets credit if a student who scores at Level 1 and/or low in Level 2 in Grade 8 makes significant growth by Grade 11.  Significant growth is defined as scoring between 55-64 on the designated Regents Examination in Integrated Algebra or Regents Comprehensive English Examination.  The amount of credit may be 200 points (indicates on track to meet the Regents passing standard) or 100 points (considerable growth, but not on track to proficiency).
•  
•               If Student A scored a 56—and thereby qualified for a local diploma—by Grade 11, the school would receive credit for helping Student A grow enough that the school would receive 200 accountability points in the Performance Index. The 55-64 points will be linked to qualification for a local diploma until 2011-12, when all students will need to score at least 65 in order to qualify for any diploma.  However, growth from a low score in Grade 8 to a score of at least 55 on the designated Regents Examination will still represent significant growth.

An example of the value table is provided below:

              The High School Performance Index is calculated by multiplying the number of students by the value of their performance and then dividing by the total number of students.  Thus, if all of the students score less than 55, the school will have a Performance Index of zero. If all students score 65 or above, or low performing students make sufficient growth (sufficient growth is defined as students starting at Level 1 or Level 2-minus in Grade 8 who earn a scale score of 55-64 on Regents examination prior to senior year), the school will have a Performance Index of 200. NYS’s goal is that all schools and districts have a score of 200 by 2013-14. It should be noted that the value table is intended to be used only through the 2010-11 school year, when it will be replaced by value-added measures.

Implications of the Interim Growth Model Methodology

              This methodology can only help schools to make AYP.  Any school that is currently making AYP will continue to make AYP.  Some schools in which significant numbers of students are not yet proficient but are moving from Level 1 to Level 2 or from low Level 2 to high Level 2 may be able to get credit for making AYP as well.

Impact on Differentiated Interventions

              It helps in two ways.  First, it allows the NYSED to distinguish between schools with low status and low growth, which need one type of intervention, and those schools with low status but high growth, which need a different type of support. In addition, it also allows us to distinguish which groups of students within a school or district are not only of low status but also not making growth, which allows for more finely calibrated interventions. As we move to the requirement for all students reaching full proficiency by 2013-14, the ability to measure whether or not a student is on track to achieve proficiency will become even more important.

Impact on the Identification Process for High Performing Schools

              NCLB primarily focuses performance on students becoming proficient.  There is no incentive given for students to progress beyond proficient. However, with the use of growth, it is possible to create a state accountability approach that also holds schools responsible for helping high-performing students grow.

              A school that otherwise would qualify for the state’s “high performing” rating would not qualify for that rating if sufficient students who were proficient or above showed declining growth such that they were “on track to become non-proficient” (i.e., Level 2) within a designated amount of time.

              In addition, schools currently must have at least three disaggregated NCLB groups to qualify for the “high performing” rating.  Some schools, while high performing, do not have enough subgroups to qualify.  The addition of growth may be sufficient to provide such schools a “high performing” rating.

Department Initiatives for the Transition to Value-Added Accountability

              Chapter 57 requires that the State implement a testing and accountability system that measures growth of students at all levels.  With the use of growth models, it is possible to create a state accountability approach that holds schools responsible for helping high-performing students grow.  Staff is engaged in empirical research to identify the design of the next generation of the Grades 3-8 Testing Program to meet these needs. Staff is reviewing the technical reports from the research and is consulting with the Department’s Technical Advisory Group and Brian Gong regarding next steps for the scaling properties. To support a vertical scale, staff is researching the necessary test structure; this requires assessing the balance between multiple-choice and open-ended responses and the numbers of items needed to ensure that the tests generate enough data points to support the inferences for this type of scaling. The Department will then issue a Request for Proposals to design and build the new Grades 3-8 Testing Program to be implemented in the 2010-11 school year. Along with test development research, staff is researching prospective value-added methodologies to ensure that the test scaling properties are aligned to the subsequent accountability methodology. Finally, staff is reviewing how to ensure that this approach is consistent with accountability guidelines as stated in NCLB. The ability to develop and implement an enhanced accountability system that utilizes value-added methodology is also dependent upon the revisions that are made to NCLB, Title I accountability guidelines during the reauthorization process.

Field Outreach Strategy

              NYSED staff have already had numerous discussions with and made presentations to a large number of interested parties regarding the concept of a growth model.  With the consent of the Board of Regents, staff will schedule meetings with organizations representing parents, teachers, administrators, data and assessment coordinators, superintendents, school boards and other key groups during June and July to review the proposed growth model.  The NYSED will also post to its web site a copy of a draft of the NYSED’s proposed growth model submission to USED and seek public comment on the document.  Once approved, NYSED staff will hold a series of meetings throughout the State to familiarize educators and the public with how the growth model works.

Capacity Needs for Implementation of the Growth Model

              This system requires a change from working with approximately 700 school districts and approximately 4,400 school buildings to working with 2.9 million student records over multiple years.  With the addition of the statewide student identification number, NYSED had all the necessary data to implement the interim growth model.  NYSED will work with districts and school systems to ensure that necessary data elements and policies are established prior to the implementation of the more sophisticated value-added system, and that districts and schools are able to comply with any additional data requirements. 

              The implementation of the growth model will require additional resources.  Additional professional and technical staff would be needed to provide training and technical assistance to school districts.  Additional staff may include accountability program experts, data analysts and computer programmers. Should the NYSED seek to do this work itself, which appears to be the preferred option, these additional staff resources would be needed before we begin the start-up phase of the process if we are to complete this assignment in addition to fulfilling our present work commitments? This work will also require changes to NYSTART computer programming.

              It is recommended that the Board of Regents seek funding from the State legislature to ensure that NYSED has the capacity necessary to produce timely and accurate growth determinations for use in making AYP decisions. Implementation will not begin until NYSED staff has demonstrated to the Board of Regents that the NYSED has put in place the necessary infrastructure to make accurate and timely AYP decisions based on measuring the growth of students.

Timeline for the Development of the Growth Model

              With the consent of the Board of Regents, staff proposes the following revisions to timeline presented in the April Board of Regents Item:

June 2008                    NYSED staff provides Board of Regents with draft growth model and request approval to seek public comments.

June/July 2008            NYSED staff engages in discussions with interested constituencies on the proposed growth model.

July 2008                      Full discussion at the Board of Regents meeting on the Growth Model; Approval of Model Requested.

If approved by Board of Regents, NYSED staff will submit model to the USED and request that it be peer reviewed.

Fall 2008                      Upon receipt of approval by the USED and a determination by the Board of Regents that the NYSED has sufficient resources to implement the model, the Board of Regents will adopt regulations, either by emergency or regular action depending on the date USED approval is received, to implement the growth model for accountability in the 2008-09 school year.

                                      Staff will hold forums around State to explain the Growth Model.

2008-2009                   Implement Growth Model, using 2007-08 school year data as base and 2008-09 school year data to measure growth.

2009-2010                   Use data from 2008-09 assessment results to make AYP decisions for schools and districts.

Appendix A

Frequently Asked Questions About the

NYSED’s “Growth Model”

• Is the NYSED required to develop a “growth model” for school accountability? NYSED is complying with State law, Chapter 57 of the Laws of 2007, that requires the State to implement an interim growth model—if that model is approved by the USED—by 2008-09 and a more sophisticated “value-added model” by 2010-11, if such model is approved by USED and a panel of external experts in testing and measurement.
• Is NYSED implementing a growth model only because it is required to do so by law?  No.  NYSED has been considering for some time a refinement in how the State currently holds schools accountable for the achievement of all students.  A “growth model” would provide schools credit for students who are making progress towards becoming proficient.  NYSED believes that by basing accountability on the percentage of students who are proficient plus the percentage of students who are demonstrating that they are on track to become proficient, NYSED can make better, more nuanced judgments about school performance.
• What is “growth” and how is it different from what schools currently are being held accountable for? Currently, schools and districts are held accountable for the percentage of students who are proficient or partially proficient at a given point in time. The current “status” model makes schools and districts responsible for demonstrating that an increasing percentage of students are proficient over time. Under a growth model, when students are not proficient, schools and districts are expected to show that a student is making sufficient annual progress to become proficient within a specified period of time. A status model typically measures the performance of the same grades, but different students, over time, while a growth model measures the performance of the same students over time.
• What will the growth model look like? The growth model will follow the guidelines established by the USED for its Growth Model Pilot and subsequent proposed regulations.  In essence, schools will get credit for having students who are proficient and for students who have grown enough that they are on-track to become proficient within a set time.  NYSED is committed to using a transparent system that is accessible enough that parents, schools, and districts should be able to perform the accountability calculations themselves.
• What will students, schools, and districts need to do to participate in the State’s growth model?  The addition of growth will not cause any additional work for students, schools, or districts, if they are complying with current state policies regarding assessment and data integrity.  The NYSED will calculate growth using existing longitudinal data.
• When will the growth model take effect?  The growth model will take effect as soon as it receives necessary approvals from the NYS Board of Regents and the USED.  The statutory intent is to have the interim growth model take effect for school accountability determinations that incorporate student assessment data from the 2008-09 school year, if NYSED is successful in securing the approval of USED.
• What will be the effect of adding the interim growth model on school accountability?  The NYSED interim growth model is designed to give schools and districts credit for students who have not yet scored proficient on the State’s assessments, if they have made sufficient progress that they are on-track to becoming proficient within a set amount of time.  Thus, the interim growth model will not cause any school or district that would have made AYP under the current system to fail to make AYP because of the inclusion of the growth component.  The number and types of districts and schools that will get credit for making AYP because of the inclusion of the growth model in the AYP determination process will be determined by students’ performances in 2008-09.  Experience from other states with these types of growth models and preliminary empirical modeling of available data indicate that the impact on schools and districts initially will be quite modest.  However, it is likely that the ability to include students who are on track to proficiency in AYP calculations will be of increasing benefit to schools and districts as the Annual Measurable Objective increases each year.
• What will be the benefits for instruction and program evaluation of adding the interim growth model?  Parents, students, schools, districts, and others will be able to see individual and/or aggregated student growth on a yearly basis from the state’s assessments.  Like other large-scale assessments, the growth data provide useful information from credible assessments at a point in time.  As with any large-scale assessment, growth data probably will need to be supplemented with more diagnostic and immediate assessment results gathered at the local level.
• How will NYSED support schools and districts using the growth data in a valid way for professional development and instruction?  NYSED will provide thorough documentation of the growth model to schools and districts, along with regional training regarding the growth model and data.  Districts and schools will need to customize the interpretation and application of the growth results to their individual programs and circumstances.
• How is the State’s growth model similar to or different from other growth models?  There are many possible growth models that are designed to serve different purposes.  The NYSED interim growth model is designed to support the Board of Regents policy to close the achievement gap and improve student performance.  It gives schools and districts credit for students making sufficient progress towards becoming proficient and meeting the NYS learning standards.   The NYS Grades 3-8 growth model methodology has many similarities to the North Carolina model that has been approved by USED.  The high school growth model employs a value tables approach that has been used by states such as Delaware and also been approved by USED. However, at both the elementary-middle and high school levels, the models contain unique features specifically designed to make the model the most appropriate one for use by NYS. Other prominent growth models are designed for other purposes.  For example, the New York City growth model compares schools primarily against like schools within New York City, and the Sander’s value-added model (Education Value Added Assessment System) used by some districts in the State is designed primarily to attribute growth to particular teachers and schools.
• What will the State’s more sophisticated “value-added” model look like?  The current growth model under development is intentionally an interim model.  The state’s goal is to develop the best available growth model with additional features, as required by the State law, by 2010-11.  The more advanced model will draw on advances in data infrastructure, statistical models, and be linked to appropriate support and intervention models.
• How will NYSED ensure there will be a smooth transition from a status model to an interim growth model and from a growth model to a value-added model?  NYSED is working to ensure a smooth transition by making the interim growth model as clear and straight-forward as possible, with as close an integration as possible with the existing assessment and accountability systems.  NYSED is aware of the desirability for a smooth transition to the longer-term “value-added” model, and is keeping that consideration salient in all of its considerations.
• How will the growth data be reported to schools and districts?  NYSED will report growth results as it does for other assessment results.  Schools and districts will receive growth results in combination with the current year data at the individual student, classroom, school, district, and state levels.  Schools and districts may choose to disclose individual or disaggregated results to parents.  The report forms that will be used are currently being designed.
• How will criteria for “enough growth” be set?   The interim growth model follows the requirement established by the USED that schools are held accountable for students becoming proficient within a set amount of time.  The NYSED interim growth model complies with that requirement while advancing an accountability design that is more meaningful for middle and high schools.  NYSED is anxious to extend its research on growth to inform future model development, such as what growth is possible, what growth is typical, and how growth can be measured as precisely as possible.
• Are the state assessments technically adequate for this purpose?  Yes.  NYSED has undertaken a series of research analyses, both in-house and with consultants, to review the technical adequacy of the current assessment system in regards to issues such as scale sensitivity and uniformity, the basis for cross-grade comparisons of student performance, the comparability of grade-level achievement levels, the adequacy of assessments for students with severe cognitive disabilities and students with limited-English-proficiency, and so on.  Based on this analysis, the current State assessment system can be validly and reliably used to support the growth models that have been proposed.
• How will any revisions to the content standards be coordinated with creating interim growth and the longer-term value-added accountability measures?  As the content standards are revised, State assessments, upon which the interim and enhanced growth models are based, will also be revised as needed to measure student achievement of the new content standards.