SUMMARY

Issues for Discussion

              What was the Contract for Excellence monitoring process for the 2007-08 Contracts?  Did the Department find that the 55 Contract for Excellence school districts executed the terms of their approved Contracts?

Reason(s) for Consideration

              Policy Implementation.

Proposed Handling

              This item will come before Regents EMSC Committee for discussion at the September 2008 meeting.

Procedural History

Contract for Excellence proposals were submitted by school districts in July 2007.  These contracts were reviewed consistent with regulations adopted by the Board of Regents by emergency action in April 2007.  All Contracts were reviewed by August 15, 2007 and the Commissioner approved all final Contracts for Excellence on November 19, 2007.  The Department conducted monitoring visits to more than 180 schools in Contract for Excellence districts between March and May 2008.

Background Information

Consistent with the Regents P-16 Action Plan, Chapter 57 of the Laws of 2007 (“Chapter 57”) included a provision for a State Foundation Formula which, over the course of four years, would provide adequate funding for all New York State school districts at a sustained level to support the provision of adequate education services for all students.  The total amount of 2007-08 Foundation Aid in the enacted budget was $13.640 billion which included a $1.1 billion increase over the 2006-07 amount.  Chapter 57 required, for the first time, that certain school districts enter into Contracts for Excellence and spend a portion of their Foundation Aid increase for school year 2007-08 on certain allowable programs and activities.  The new statute required that any school district receiving an adjusted increase of 10 percent or $15 million and with one or more schools designated as Requiring Academic Progress, In Need of Improvement, Corrective Action or Restructuring, submit a Contract for Excellence.  Statewide, the total amount of 2007-08 State Foundation Aid for which a Contract for Excellence was required was $428 million, 5% of the total amount of Foundation Aid that Contract for Excellence districts received ($7.940 billion) or 3% of the total amount of 2007-08 Foundation Aid for all school districts in the State ($13.640 billion).  

The Department began working on guidance for school districts on the Contracts for Excellence as soon as the Governor’s Executive Budget proposal was released in January 2007.  In February and March of 2007, the Department began conducting meetings and discussions with groups of education partners including representatives from school districts that would likely be Contract for Excellence school districts.  As soon as the State budget was approved in April and without any additional staff or resources, the Department began providing information on implementing the Contracts to school districts, developed draft regulations for consideration by the Regents with extensive input from the field, began development of a Web-based system for Contract submission by school districts and began responding to numerous inquiries from school districts and others concerning implementation of Contracts for Excellence.  To facilitate the successful implementation of Contracts for Excellence for the 2007-08 school year, the Regents adopted regulations by emergency action at their April 2007 meeting.  All Contracts for Excellence were developed and submitted by school districts in July 2007 and the Department reviewed them all by August 15, 2007.  Between August and November of 2007 the Department worked with districts that needed to revise their Contracts to meet the statutory and regulatory requirements so that the final Contracts could be approved.  The Commissioner approved all 2007-08 Contracts for Excellence on November 19, 2007.

For the 2007-08 school year, consistent with Chapter 57, 55 school districts across the State - made up of over 1,500 schools with a total enrollment of 1.1 million students - put in place Contracts for Excellence to use new State Aid to implement proven, research-based practices to improve student learning in order to strengthen academic outcomes for students and schools most in need.  Contract for Excellence school districts are required to implement proven programs and activities primarily focused on students with the greatest educational needs. Contract for Excellence districts could use funds for class size reduction; time on task; teacher/principal quality initiatives; middle school/high school restructuring; and full day pre-kindergarten/kindergarten and experimental programs. 

This report provides the following information:

• The Monitoring Process
• General Overview of SED Monitoring Site Visit Findings
• NYCDOE Class Size Reduction
• Next Steps for all Contract for Excellence Districts Statewide

Attachment A - List of schools visited during SED monitoring

Attachment B – Monitoring summaries for C4E districts

I.  The Monitoring Process

To meet Chapter 57 and the Board of Regents expectations for ensuring fiscal, performance, and program accountability for the Contracts for Excellence resources and to improve student achievement, the State Education Department developed a coordinated approach to monitoring the 55 Contract for Excellence districts.  To ensure consistency and continuity in monitoring, the Office of Elementary, Middle, Secondary and Continuing Education, School Operations and Management Services, Office of Vocational and Educational Services for Individuals with Disabilities (VESID), and the Office of Audit Services developed monitoring protocols. 

Contract monitoring is a three step process:

• On-site monitoring. SED monitors visited 183 schools over a 3-month period to review a representative sample of schools where Contract for Excellence programs were being implemented. 

• Certified audit report.  By October 15, 2008 Contract districts are required to submit certified audit reports that show that systems are in place to separately track receipt and spending of Contract for Excellence funding for purposes of assessing that contract funding is targeted to schools consistent with the approved Contract and that the increase in total foundation aid and supplemental improvement plan grants have been used to supplement and not supplant funds allocated by the district in the previous year.

• A detailed breakdown of contract expenditures in a format prescribed by the Comptroller.  Due when districts have closed their books for the 2007-08 school year; this information will be crosschecked against the C4E system and will be reviewed by independent auditors in preparing their certified audit report.

Given the size of this new initiative and to give school districts some time to develop and implement Contract programs, (4-6 months), monitoring took place between March and May 2008.  Thirty-six members of the Department’s P-16 staff participated in the monitoring in addition to their other responsibilities. Fourteen of the 36 were new staff hired with resources provided to the Department by Chapter 57.  

Since it was not possible to conduct site-visits at all 1,500 schools impacted by the Contracts for Excellence, a risk-based approach was used to determine the number of schools to be visited in a school district.  The more schools identified as in need of improvement in a district, the more schools the Department selected for monitoring. 

Statewide, 183 schools were visited during the three-month monitoring process:

• Three to four schools were visited in 27 NYC Community School Districts (89 schools)
• The accountability status of schools under No Child Left Behind (NCLB) was used to create a random stratified sample, with over-sampling of schools farthest along the NCLB accountability continuum.
• Schools in New York City were visited as follows: Title I Restructuring Schools (38); Title I Corrective Action Schools (21); Schools Requiring Academic Progress (11); and Schools in Good Standing (19).  
• In each of the Big Four city school districts, four schools were visited per district (16 schools).
• In school districts the rest of the State, two schools were visited for each of the 27  school districts considered to be higher risk (54 schools), and one school was visited in each of the 23 school districts considered to be lower risk (23 schools). 

Attachment A is a list of the schools visited during the monitoring visits.  Monitoring summaries for each Contract school district are included in Attachment B.  Additional details on the monitoring visits are also available.

II.  General Overview of SED Monitoring Site Visit Findings

              The purpose of the monitoring was to ensure that the terms of the approved Contracts were being met.  Department monitoring visits assessed implementation in the following areas:

• Implementation of the 2007-08 Contract as approved;
• Public comment process for 2007-08 Contracts;
• Established procedures for parent complaint;
• Improved student performance and expectation of achievement of contract performance targets; and
• Public comment process for 2008-09 Contracts.

Implementation of the 2007 – 2008 Contract as Approved

In general, SED monitors found that sufficient evidence was available to verify that the majority of Contract districts substantially implemented the Contract provisions as approved by the Commissioner.  Evidence of compliance in each district is outlined on the attached monitoring summary reports.  In those districts where Department monitors had some concerns about implementation, detailed monitoring reports that included findings and corrective action steps that need to be taken were provided to the districts.  These districts must prepare a corrective action plan to correct deficiencies within 30 days.

              The statute does not permit State Aid dollars required to be spent on Contract for Excellence programs and activities to be carried-over from one year to the next; they must be used by the end of the school year (June 30th).   The regulations included a process for districts to request amendments to their contracts, if after implementation, circumstances changed.   Because the original C4E budgets submitted to the Department were estimates of costs and in some cases differed from the actual costs associated with the approved C4E program activities, 23 districts received approval for contract amendments that would permit them to fully use the C4E funds in the 2007-08 school year.

Public Comment Process for 2007-08 Contracts

              The public comment process required by statute for the 2007-08 Contracts for Excellence was very general.  Chapter 57 of the Laws of 2007 required that for the 2007-08 Contracts for Excellence, school districts must “solicit public comment on their contracts for excellence.”  All school districts monitored took steps to publicize the contracts and seek comment, but in general, public engagement was not extensive. Most school districts used Board of Education meetings, budget information sessions and other district meetings as the primary means of disseminating information about the Contracts for Excellence.  Public hearings were conducted in some districts and others posted the Contracts on their district Web sites.  While a number of districts made significant effort (e.g., New York City held hearings in each of the 5 boroughs), community turnout was low in many of them. Both the statute and the Regents regulations require that this process be strengthened in future years.   Those 2007-08 Contract for Excellence districts that are again required to submit Contracts in 2008-09 will be required to expand their public comment process.

Established Procedures for Parent Complaints

Chapter 57 included a requirement for there to be a mechanism for parents to file complaints about the implementation of a district’s Contract for Excellence with the school principal or school superintendent. If a parent is not satisfied by the response from the district to their complaint, they may then appeal to the Commissioner pursuant to Section 310 of Education Law. To date SED has not received any appeals.

Approximately half of Contract districts established specific C4E complaint processes.  Other districts relied upon existing district parent complaint processes such as the Title I parent complaint policies but in many cases,  parents were not notified in writing of the establishment of an opportunity to specifically register concerns about the Contract for Excellence. SED monitors alerted districts to the requirement establishing a Contract-specific parent complaint policy and have requested documentation of the adoption of such policies in those districts. 

Improved Student Performance and Expectation of Achievement of Contract Performance Targets

              As a result of the Contracts for Excellence, research-based practices to support student learning were targeted for 1,500 schools across the State for those students with the highest need.  Preliminary indications from monitoring visits reveal that districts’ efforts to implement the Contracts are positively impacting students.   Most school administrators interviewed in Contract districts report that they believe that student performance will continue to improve as Contract for Excellence programs continue to be implemented in their schools.  Some were able to share positive local assessment/test information.  The Department will be able to make a more informed assessment of trends in performance on State examinations in English and mathematics in schools receiving C4E funds when the full set of State assessment data for the 2007-08 school year becomes available.  It is also important to consider two points of additional information when gauging the success of the Contracts for Excellence initiative this year and in the long-term. First, C4E funds represented just a relatively small proportion of total education revenues in C4E school districts.  This impacts one’s ability to draw a direct causal relationship between Contracts for Excellence resources and performance growth.  It is expected, for example, that performance achievements, for example, will likely come as a result of the Contracts for Excellence initiative in conjunction with other funding and practices.  Second, it is important to note that English examinations were administered in January 2008, just shortly after Contracts for Excellence were approved on November 19, 2007.  It is important to exercise caution when assessing the efficacy of C4E funding during this first year of C4E program creation and implementation.  Similarly, other district factors during this period, (i.e., certified teachers, other non C4E district programs, etc.,) have to be considered when looking at student performance.   Nevertheless, monitoring visits have preliminarily revealed that the Contracts for Excellence initiative have had a positive impact on students.  After a full set of test results is available, the Department will be able to provide further analysis of the broad trends in student performance in districts and schools participating in C4E.

The Department has taken a number of actions to ensure that the Regents will have data on the effectiveness of C4E programs in improving student performance:

• Each school has been required to establish C4E performance targets for students impacted by C4E.  These targets focus on reducing the percentage of students in specific populations who are not proficient in the major subjects.  SED staff will report to the Regents later this year on the degree to which schools were successful in meeting these targets.

• The Department is currently consulting with representatives of school districts and others to determine whether the current process to set C4E performance targets should be modified to make them more meaningful for schools and districts.

• The Department is collaborating with the Education Research Finance Consortium on the development of a possible design to evaluate the C4E programs’ impact on student progress.  Examination of district assessment data for students involved in C4E activities may include:

• Exit rates from limited English proficiency status for English Language Learners;
• Systemic growth from level 2 to 3 for cohorts of students
• Graduation and dropout rates
• Regents test completion and test results
• Change in school accountability status

Longer-term, depending on available resources, the Department and the Research Consortium are considering program evaluations that could determine which strategies used by C4E districts have proven most effective in improving student performance. Studies may also be conducted to determine the degree to which C4E funding is cost effective in promoting improved student achievement.

Public Comment Process for the 2008-09 Contracts

              The statutorily required public comment process for the 2008-09 Contracts for Excellence are more specific than those for the first year (2007-08).  Chapter 57 requires districts to develop the 2008-09 Contracts for Excellence through a public process, in consultation with parents or persons in parental relation, teachers and administrators.  A public hearing must be held within each district, and in each borough in New York City.  A transcript of testimony presented at the public hearings must be made public before the Contract is submitted to the Commissioner for approval.  Based on public comments received, the Board of Regent, in their regulations, specified procedures that school districts must follow to ensure that the public is included in the C4E development process.   In addition to the public hearing process required by the statute, the Regents regulations require a 30-day public comment period; notice of the public comment process including specific information about the proposed C4E programs and allocations by affected student population groups (i.e., English language learners, students with disabilities, etc.); information on where to obtain a copy of the proposed C4E, and a description of the process for submitting written comments.  Regents regulations also delineate specific requirements concerning the notice of required public hearings and the conduct of the hearings which includes providing an opportunity for the submission of oral and written comments by any interested party.   Each district must prepare a public comment assessment within 12 days after the expiration of the public comment period or the conclusion of public hearings, whichever is later.  The public comment assessment must be posted publicly and made available upon request. 

              The public comment process for the second year of Contracts for Excellence (2008-09) is underway in Contract school districts.  Community focus groups, public forums, public hearings, Board of Education meetings and internet postings are being used to provide information to the public and generate public comment and input.  The Department is monitoring this process. 

• NYCDOE Class Size Reduction

Over the last several decades, average class sizes in New York City public schools have been significantly higher than the State average.  For example, BEDS data from 1990 to 2005 show New York City consistently reported average class sizes in Grade 7 English and High School Biology that were more than 20% higher than the statewide averages.  Chapter 57 included a requirement that the New York City Department of Education (NYCDOE) incorporate in its Contract for Excellence a five year plan for class size reduction.

NYCDOE’S enrollment targets, school budgets and hiring decisions for 2007-08 had to be made even before its Contract proposal was submitted to the Department in July 2007.  Approval of the Contract occurred in November 2007. Because enrollment and hiring decisions were already made, NYCDOE’S ability to fully implement the 2007-08 class size reduction plan was impacted.    

Summary of SED key findings regarding New York City’s actions to reduce class sizes and pupil-teacher ratios based on SED analysis of data provided by the NYCDOE:

• NYCDOE substantially increased the number of classroom teachers by 1,892 and made progress toward achievement of the citywide targets they established.  However, they did not fully achieve the 2007-08 school year citywide class size targets the NYCDOE established in their plan.    

NYC Average Class Size

                                      

• NYCDOE made progress toward achieving its 2007-08 school year pupil to teacher ratio targets.   The Contract called for a reduction in pupil-teacher ratios from 13.8 in 2006-07 to 13.4 in 2007-08. NYC reported that, excluding CSD 75 schools, ratios were reduced from 13.84 in 2006-07 to 13.46 in 2007-08. Two-thirds of NYC schools reported a reduction in pupil-to- teacher ratios between 2006-2007 and 2007-2008.  While overall NYC reported that pupil-to-teacher ratios were reduced by .38 citywide, schools receiving C4E funding on average reported almost twice as large a reduction in pupil to teacher ratio (.47 versus .25).

3.     NYCDOE and the State Education Department identified 75 New York City schools for high priority class size reduction, based on a combination of high absolute class size and low student performance. Forty-five of the 75 priority schools (60%) fully achieved the 2007-2008 reduction in average class size that was projected in the approved class size plan.  The average projected class size reduction for these 75 schools was 2.6% for 2007-2008.  Middle schools were more successful at achieving their targets, with 27 of 41 (65.9%) achieving them compared to 19 of 35 high schools (54.3%). In total, NYC reported that the average class size was reduced in 30 of the 41 priority middle schools and core class sizes were reduced in 32 out of the 35 priority high schools (one school had both middle and high school grades).

4.     A shortcoming of the first year implementation is that NYC cannot identify how many of these teachers are specifically attributable to Contract for Excellence funds.  NYCDOE projected it would hire 1,900 teachers above the staffing level needed to maintain existing 2006-07 class size and pupil-to-teacher ratios and that 1,400 of those teachers would be funded by C4E dollars. NYCDOE has reported that it   hired 1,892 additional classroom teachers in 2007-08.

5.     In 2007-08 658 schools allocated C4E funds to reduce class sizes or pupil-to-teacher ratios, of which 399 allocated $97,000 or more to class size reduction (the average cost of a NYC teacher, including salary and fringe benefits). Using C4E and other funds, at the elementary school level, 53.3% of City schools reported a reduction in average class size.  At the middle and high school levels the percentage reporting a reduction was 57.3% and 62.5% respectively. 

6.     There is evidence that NYC focused class size reduction efforts on those schools with the highest official class sizes at the elementary and middle school level and highest core class sizes in high schools.  Nonetheless, at the elementary school level, nearly 40% of schools in the quartile with the largest average class sizes did not report class size reductions in 2007-08.

7.     On average, schools in need of improvement or requiring academic progress had greater reductions in average class size than the schools in good standing.

8.     As is permitted by the law, much of NYC class size funding was used to reduce pupil-teacher ratios rather than reduce official class sizes.  In total, two-thirds of NYC schools reported a reduction in pupil-teacher ratio in 2007-08 compared to 2006-2007, including 70% of schools that received C4E funding. However, 18.6% of schools that allocated money to reduce class sizes or pupil-teacher ratios reported no reductions in either class size or pupil-to-teacher ratios, including 70 schools that received over $100,000 in C4E funding last year. These schools account for $19.7 million in class size reduction funding. SED is requiring that NYCDOE explain how C4E funds were utilized in these schools.

9.     Overall, 53.9 of NYC schools reported that either official or core class size or pupil-to-teacher ratio increased in 2007-2008.  In contrast, 46.1% of schools reported decreases in both class size and pupil-to-teacher ratio. 

What is being done to improve the second year implementation of NYCDOE’s class size plan?

NYCDOE will be required to improve implementation of the second year of its class size plan.

Based upon these findings, certain actions have already been taken by NYCDOE:

• For 2008-09, the timing of funding designations and C4E allocations has already allowed class size planning to occur when it is more likely to impact system and school level resource allocations. 

• The DOE also reports that its other school-level planning efforts, including the Comprehensive Education Plan (CEP) process and School Support Organizations technical assistance programs, have been aligned with class size reduction planning efforts, making it more likely and easier for schools to achieve reductions. 

• A distinct allocation category for C4E In the school accounting software also makes the funding more identifiable to schools and makes it easier to track.

• Revised reporting systems on special education and middle school programming will more appropriately track the progress that schools make in reducing class sizes.

Beyond these changes, SED is also requiring that the following actions be taken by the NYCDOE:

• NYCDOE must provide evidence of how C4E funds were used appropriately to support class size reduction in schools in the 70 schools that received $100,000 or more in C4E funding but in which class sizes and pupil-teacher ratios increased. In addition, all Contract districts, including New York City, are required to submit certified audit reports that show that systems are in place to separately track receipt and spending of Contract for Excellence funding for purposes of assessing that contract funding is targeted to schools consistent with the approved Contract and that the increase in total foundation aid and supplemental improvement plan grants have been used to supplement and not supplant funds allocated by the district in the previous year.

• NYCDOE must provide SED with class size and pupil-to-teacher targets for each school and a detailed description of how many additional classrooms are to be created in school year 2008-09 under the Class Size Reduction Option for each school, including the grades, subjects and/or special populations that are targeted for reduced teacher/student ratios for 2008-09.

• SED staff will analyze NYCDOE’s final class size submissions and expenditure reports and require NYCDOE to address any implementation issues in its 2008-2009 plan.

• For the 2008-09 Contract year, NYCDOE must submit a plan to implement the second year of the five year class size and pupil-to-teacher ratio targets that were approved by the Commissioner in November 2007.

• NYCDOE’s 2008-2009 class size reduction plan will only be approved after these actions have been taken.  To receive approval, NYCDOE must demonstrate that the elements of its plan will result in NYCDOE achieving its 2008-2009 average class size and pupil-to-teacher ratio targets.

The Commissioner’s Class Size Panel will make recommendations on future class size targets and provide guidance on NYCDOE’s five year plan. The panel will also recommend methodologies that can be used statewide to calculate class sizes and pupil-to-teacher ratios.  The Commissioner then must set class size targets and pupil-to-teacher ratio targets.

IV.    Next Steps for all Contract for Excellence Districts Statewide

1. We are using what we learned in the first-year of Contract for Excellence implementation and monitoring  to inform the approval process for the 2008-09 Contracts.

1. All first-year monitoring visits have been completed, and reports detailing the findings submitted for each district.  Based upon the findings, if there were problem areas, corrective action plans were required from the districts.

1. Examples of corrective actions that districts were asked to make include submission of:

• Additional documentation concerning computer equipment that was not yet in operation at the time of the monitoring visit.

• A revised Contract for Excellence parent complaint policy.

• Documentation detailing professional development activities that were conducted.

• Program descriptions for each of the C4E options implemented in schools for School Year (SY) 2008-09.  This description must include information on first time implementation or expansion of existing programs.

1. Other corrective action requirements include:

• Immediately informing all C4E schools of funding levels and requirements attached to the C4E program to ensure accountability for C4E funds.  Appropriate, standardized documentation will be required in every C4E school. 

• An immediate process to inform C4E schools of their student performance targets and the gap reductions required for designated groups of students.

• Continued efforts to recruit highly qualified staff as approved in the C4E.

• School districts required to file Contracts will publicly report the expenditure of total foundation aid, which will ensure full disclosure of the use of such funds. The Department has developed a methodology to report school-based expenditures by all Contract districts.

1. Building upon the knowledge gained in the monitoring visits, SED will:

• Continue to meet with C4E districts, provide technical assistance, and review final monitoring reports provided by districts.

• Review New York City’s class size data and detailed year-end expenditure reports.

• Once a full set of test results are available, provide the Regents with analyses of broad trends in student performance in C4E districts.

• Report to the Regents on the degree to which schools were successful in meeting performance targets

• Undertake a long-term evaluation of the success of the various C4E options – such as reduced class sizes, increased time on task, etc. 

• Review the monitoring schedule, protocols, instruments and sampling methodology, so that appropriate revisions will be implemented in 2008-09.

• The Department is also discussing potential consequences in the event that districts do not carry out the terms of their Contracts and/or fail to follow-through on corrective actions. 

Attachments

                                                                                                                                                           Attachment A

2007-08 C4E Monitoring Visits

Attachment I

Attachment II

Attachment III

The New York City Department of Education in its approved class size plan reported a 2006-2007 baseline of 24.9 and a target of 24.3 for grade 9-12. for 2007-08. Subsequent to submission of its plan, the New York City Department of Education, after consultation with interested organizations, modified its methodology for calculation of high school class sizes.  The new methodology is based on core subject classes that are required for graduation while the methodology used in the C4E plan included elective courses.  Since elective courses tend to be more advanced and have smaller enrollments, their removal from the calculation caused the baselines and targets to be higher under this new methodology. For analytic purposes SED computed a revised baseline and target for NYCDOE using the new methodology.