sed seal                                                                                                 

 

THE STATE EDUCATION DEPARTMENT / THE UNIVERSITY OF THE STATE OF NEW YORK / ALBANY, NY 12234

TO:

Subcommittee on Audits

 

FROM:

John B. King, Jr.

SUBJECT:

Update on Actions Taken to Respond to the Findings and Recommendations in the Office of the State Comptroller  Audit of Regents Examinations Scoring Practices

DATE:

April 7, 2010

   

AUTHORIZATION(S):

 

SUMMARY

 

Issue for Discussion

In response to the audit of Regents examinations scoring practices conducted by the Office of the State Comptroller, should the Audits Subcommittee approve the two-step recommendation for scanning Regents examination answer sheets?  Should this recommendation be forwarded to the Regents EMSC Committee for discussion and approval? 

Background Information

The Office of the Comptroller conducted an oversight audit of scoring practices on Regents Examinations. The final Audit report was issued on November 20, 2009.  In December 2009, the Regents Subcommittee on Audits discussed the Audit findings and recommendations and future actions to be taken by the Department to respond to the audit recommendations.  

Points to Consider:

 

 

Update:  Scanning Regents Exams:

The Department responded to the Office of the State Comptroller with a series of actions that we will undertake. One set of significant actions requires conducting Regents Examination Item Level Score Collections at both the representative sampling and full test taking census levels which will include erasure and response tampering analysis (this will require a contract with an external vendor) and conducting annual spike/cluster analysis of scores to identify schools with suspicious results.

             

              Regents Exams are scored immediately after their administration; schools use a variety of means to score and record the data. The Department has an interest in ensuring that Regents exams are scored accurately since these scores are used on local school report cards, student transcripts, and for accountability reporting.  There is no standardized process in place that allows the State to collect item level and test-taker demographic data that it needs for: USED Title I Peer Review; USED Race To the Top; IDEA & Title III Accommodation Analyses; Office of State Comptroller Test Scoring Monitoring (data forensics); and Technical Manual Development. Without scanning these answer sheets for the information that that they can provide, the Department does not have a mechanism to provide high quality data analysis for its own needs and for providing school systems with actionable data that can used to inform their instructional programs.

              Department staff has discussed requiring statewide scanning of Regents Exam Answer sheets, starting in June 2011, with District Superintendents, Regional Information Center (RIC) Directors, the NYSCOSS Curriculum & Assessment Committee, Big 5 School Districts, NYSSBA, and NYSUT. Currently, there is some scanning of Regents exams around the State, but it is not being done at scale.  We have a combination of RICs providing the service and a limited number of school districts that have purchased equipment, ranging from high-capacity scanners to single-sheet desktop scanners, to do the work.  Department staff has met with RIC Directors to determine answer sheet development and scanning costs (approximately $1.80-2.25 per student, per test), answer sheet design and production, scanning timelines, and reporting. Precise cost implications for districts would vary depending on the number of students tested per year per test and current scanning practices (e.g., some districts pay for scanning now and some have the equipment to do the scanning themselves).   

Recommendation to Consider

              The Subcommittee on Audits could approve the recommendation below and forward it to the Regents EMSC Committee for discussion and approval:

              The Department realizes that there will be costs associated with scanning Regents Answer sheets. To that end, we propose the following two step approach: