_Chapter 14 - Appealing the Ratings_ |
Providing superintendents with the opportunity to appeal accountability ratings has been a feature of the state accountability system since 1994. The opportunity to appeal is supported in the 2007 system as well.
Superintendents may appeal the state accountability ratings for both standard and alternative education accountability (AEA) procedures, by following the guidelines provided in this chapter. Below are the dates for appealing ratings. These deadlines are final. To maintain a fair appeals process, no late appeals will be considered.
June 21, 2007 |
Dropout/Completion Lists. Superintendents are given access to confidential lists of dropouts and lists of completion cohort membership. These reports provide a preview of the data that will be used to calculate the Annual Dropout Rate and Completion Rate base indicators for the state accountability ratings. |
|---|---|
July 20, 2007 |
Preview Data Tables. Superintendents are given access to confidential preview accountability data tables for their district and campuses showing all state accountability indicator data. Principals and superintendents can use these data tables to anticipate their campus and district accountability ratings. Appeals may be submitted by the superintendent after receipt of the preview data tables. |
August 1, 2007 |
Ratings Release. Due to the short timeline between the transmittal of the preview data tables and the ratings release date, no appeals will be resolved before the ratings release. |
August 17, 2007 |
Appeals Deadline. Appeals must be postmarked no later than August 17, 2007 in order to be considered. |
Late October, 2007 |
Ratings Update. The outcome of all appeals will be reflected in the ratings update scheduled for October, 2007. At that time the TEA website will be updated. |
A more detailed calendar can be found in Chapter 18 – Calendar.
The numbers shown on the data tables (and later on other agency products, such as the AEIS reports) are final and cannot be changed, even if an appeal is granted.
Appeals should be based upon a data or calculation error attributable to the Texas Education Agency, regional education service centers, or the test contractor for the student assessment program. However, problems due to district errors in PEIMS data submissions or on TAKS answer sheets are considered on a case-by-case basis. Also, statute permits consideration of data reporting quality in evaluating the merits of an appeal. Poor data quality is not a valid reason to appeal.
Only appeals that would result in a changed rating will be considered.
Appeals that follow these guidelines are not guaranteed to be granted. Each appeal is evaluated based on the details of its unique situation. Well-written appeals that follow the guidelines are more easily processed, but they are not necessarily granted.
One strength of the state accountability system is that the rules are applied uniformly to all campuses and districts. Therefore, a request to make exceptions for how the rules are applied to a single campus or district is viewed unfavorably and will most likely be denied. Examples of some appeals seeking inconsistent rule application follow. Because some examples apply to both standard and AEA procedures and some are unique to one set of procedures or the other, the examples are subdivided accordingly:
Examples applicable to both standard and AEA procedures:
- Campus Mobility. A request to include the performance of students who were excluded due to the appropriate use of the campus mobility subset criteria would likely be denied.
- Grade 3 and Grade 5 Cumulative. A request to alter the TEA methodology for combining the first and second administrations of grade 3 reading results, or for the first and second administrations of grade 5 reading and mathematics results would likely be denied.
- Rounding. A request to compute Required Improvement, student group percentages, or indicator values differently from the method described in this Manual would likely be denied.
- Minimum Size Criteria. A request to evaluate student groups using minimum size criteria different from those described in this Manual would likely be denied.
Examples applicable to standard procedures:
- Exceptions Provision. Exceptions are automatically applied; a request for additional exceptions or changes to the application of the Exceptions Provision would likely be denied.
- Pairing. A request to alter pairing relationships that districts had the opportunity to determine by April 27, 2007 would likely be denied.
- New and Academically Unacceptable. A request to assign the Not Rated: Other label to campuses that are Academically Unacceptable in their first year of operation would likely be denied.
- Floors. A request to waive the floor requirements when applying either the Exceptions Provision or Required Improvement would likely be denied.
Examples applicable to AEA procedures:
- Late Registration Requests. A request submitted after September 22, 2006 to be registered as an alternative education campus (AEC) in order to be evaluated under AEA procedures would likely be denied.
- At-risk Criterion. A request by AECs or charter operators to be evaluated under AEA procedures when they did not meet the at-risk criterion or applicable safeguards in 2006-07 would likely be denied.
If a problem is identified with data received from the test contractor, the TAKS data may be appealed. An appeal of the TAKS indicators should reflect a serious problem such as a missing grade level or campus. However, coding errors on TAKS answer sheets will be considered on a case-by-case basis.
- If the district has requested that writing results be rescored, a copy of the dated request to the test contractor and the outcome of the rescored tests should be provided with the appeal. If the rescored results impact the rating, these appeals are necessary since rescored results may not be processed in time to include in the assessment data used to determine the accountability ratings released on August 1.
- If other serious problems are found, copies of correspondence with the test contractor should be provided with the appeal.
- Coding errors related to student demographic or program participation fields on the TAKS answer documents will be evaluated by reviewing the student’s history in PEIMS.
As with TAKS appeals, an appeal of the SDAA II indicator should include copies of any correspondence with the test contractor. Other information available to the agency about special education students will be used in evaluating SDAA II appeals; for example, Performance-Based Monitoring (PBM) indicators pertaining to SDAA II will be examined in concert with the supporting documentation provided by the district. Any SDAA II appeals that result in raising a rating from Academically Unacceptable to Academically Acceptable will incur the use of an exception. For that reason, if an SDAA II exception was used in 2006, no SDAA II appeal can be granted in 2007, as the same exception cannot be used in two consecutive years.
Due to a number of factors—change in the definition of a dropout, changes to the PEIMS leaver data collection, the effect of students displaced by Hurricane Katrina on the 2005-06 dropout rate, and the absence of Required Improvement for the Annual Dropout Rate this year—the School Leaver Provision has been added for 2007. This means that leaver indicators (either alone or in combination) cannot cause a lowered campus or district rating. The School Leaver Provision applies to Completion Rates I and II, both Annual Dropout Rates (for grades 7-8 and grades 7-12), and Underreported Students.
The School Leaver Provision will be automatically applied. There is no need to appeal any of the leaver indicators, as none of them will cause a lowered rating.
Campuses that avoid being rated Academically Unacceptable in 2007 due to the application of the School Leaver Provision will be subject to technical assistance team (TAT) intervention requirements in the 2007-08 school year. Additionally, districts will be subject to identification and intervention under Performance-Based Monitoring (PBM) for dropout rates and leaver reporting.
For more information on the dropout definition changes, see Appendix I: NCES Dropout Definition. For more information on technical assistance teams, see Chapter 15: Responsibilities and Consequences.
Gold Performance Acknowledgments (GPA) cannot be appealed. Campuses or districts that appeal an Academically Unacceptable rating will automatically receive any GPA earned if their appeal is granted and their rating is raised to Academically Acceptable or higher.
Grade 11 assessments are administered multiple times during the school year. For accountability purposes, the performance of all juniors tested for the first time during the primary spring administration and some juniors testing for the first time during the October administration are included. (See Chapter 2.) A district may appeal to include additional grade 11 results for first-time tested students as part of the TAKS base indicator. These appeals will be evaluated on a case-by-case basis. As with all appeals, no changes will be made to the data shown on the reports.
The 2007 performance results of students who were displaced during the 2005-06 school year due to the hurricanes are included in the 2007 accountability data. This means that Required Improvement will be based on 2007 results that include these students, compared with 2006 results that do not.
A district may appeal to include the prior year performance of students who were excluded from assessment results in 2006, for purposes of meeting Required Improvement. Districts must provide evidence that inclusion of these students’ results in 2006 will have an impact on the campus and/or district rating.
In evaluating the appeal, TEA will consider the performance of all students coded KRI in 2006, not a subset of these students.
These appeals will be evaluated on a case-by-case basis. As with all appeals, no changes will be made to the data shown on the reports.
High schools created to serve special populations of gifted and talented and/or early college bound students may appeal the use of the district completion rate when the use of this district value is the sole reason for not achieving the next higher rating. Early college high schools are designed to produce graduates who earn both a high school diploma and a college degree. The appeal must provide justification for why the use of the district completion rate is not an appropriate substitute.
Superintendents appealing an accountability rating must transmit a letter prior to the appeal deadline that includes the following:
- A statement that the letter is an appeal of the 2007 state accountability rating;
- The name and ID number of the district and/or campuses for which the appeal is being submitted;
- The specific indicator(s) appealed;
- The problem, including details of the data affected and what caused the problem;
- If applicable, the reason(s) why the cause of the problem is attributable to the Texas Education Agency, a regional education service center, or the test contractor;
- The reason(s) why the change would result in a different rating, including calculations that support the different outcome;
- A statement that all information included in the appeal is true and correct to the superintendent’s best knowledge and belief; and,
- The superintendent’s signature on official district letterhead.
Other Information:
- Appeals for more than one campus within a district may be included in the same letter.
- Appeals for more than one indicator may be included in the same letter.
- Appeals of ratings issued under both standard and AEA procedures may be included in the same letter.
- Districts have only one opportunity to appeal each indicator for any campus or the district.
- When student-level information is in question, supporting information must be provided for review, i.e., a list of the students in question by name and identification number. It is not sufficient to claim data are in error without providing information with which the appeal can be researched and evaluated. Confidential student-level documentation included in the appeal packet will be processed and stored in a secure location and will be accessible only by TEA staff authorized to view confidential student results.
- It is the district’s responsibility to ensure all relevant information is included in the appeal as districts will not be prompted for additional materials.
- Envelope should be addressed to the Division of Performance Reporting as follows:
Attn: Accountability Ratings Appeal
|
- Appeal letter should be addressed to Dr. Shirley Neeley, Commissioner of Education (see letter examples, below).
- Appeal letter must be postmarked on or before August 17, 2007. Appeals postmarked after this date will not be considered. Appeals delivered directly to TEA by district staff must be time-stamped in the Division of Performance Reporting by 5:00 p.m. on August 17, 2007.
- Only send one copy of the appeal letter and/or supporting documentation.
- Districts are encouraged to obtain delivery confirmation services from their mail courier.
- Examples of satisfactory and unsatisfactory appeals are provided on the following page for illustration.
Satisfactory Appeal: |
Unsatisfactory Appeals: |
|---|---|
Dear Commissioner Neeley, This is an appeal of the 2007 state accountability rating issued for Elm Street Elementary School (ID 123456789) in Elm ISD. Specifically, I am appealing TAKS mathematics for the Hispanic student group. This is the only indicator keeping Elm Street Elementary from achieving a rating of Academically Acceptable. My analysis shows a coding change made to one student’s ethnicity on the answer document at the time of testing was in error. One 5th grade Hispanic student was miscoded as White on the answer document. Had this student, who passed the mathematics test, been included in the Hispanic student group, the percent passing for this group would have met the Academically Acceptable standard. Removing this student from the White student group does not cause the White student group performance to fall below the Acceptable standard. Attached is the student’s identification information as well as the PEIMS data for this student for the last six years (kindergarten through 5th grade) showing we have consistently reported this student as Hispanic. The second attachment shows the recalculated mathematics percent passing statistics for both the White and Hispanic student groups for Elm Elementary. We recognize the importance of accurate data coding, and have put new procedures in place to prevent this from occurring in the future. By my signature below, I certify that all information included in this appeal is true and correct to the best of my knowledge and belief. Sincerely,
J. Q. Educator attachments |
Dear Commissioner Neeley, I have analyzed the percentage passing for the economically disadvantaged mathematics students. The campus is allowed two exceptions. The floor for using the exception table is 40% for mathematics. The campus has 39%. Therefore, the campus was not able to use both exceptions. I am seeking consideration for the 39% in mathematics for the economically disadvantage student group. If granted, the school’s rating would become Academically Acceptable. Attached is a copy of the preliminary accountability data table. Sincerely,
J. Q. Educator attachment |
Dear Commissioner Neeley, Maple ISD feels that its rating should be Exemplary. The discrepancy occurs because TEA shows that the performance for Hispanic Writing is 89%. We have sent two compositions back for scoring, and are confident they will be changed to passing. If you have questions, do not hesitate to contact us, at 701-555-1234. Sincerely,
J. Q. Educator (no attachments) |
Once an appeal is received by the Division of Performance Reporting, the process for evaluating the information will be followed as outlined below:
- The details of the appeal are entered into a database for tracking purposes.
- Researchers evaluate the request using agency data sources to validate the statements made to the extent possible. The agency examines all relevant data, not just the results for the students specifically named in the correspondence.
- Researchers analyze the effect that granting a campus appeal may have on other campuses in the district (such as paired campuses), whether they are specifically named in the appeal or not. Similarly, the effect that granting a campus appeal may have on the district is evaluated, whether the district is named in the appeal or not. In single-campus districts, both the campus and the district are evaluated, whether the district submits the appeal as a campus or district appeal.
- Staff prepares a recommendation and forwards it to an external panel for review. Legislation passed in 2006 requires use of an appeals panel to ensure independent oversight of the appeals process. The use of an external, independent, three-member panel has been a feature of the state accountability system since 2004.
- The review panel examines the appeal, supporting documentation, staff research, and the staff recommendation. The panel determines its recommendation.
- The panel’s recommendation is forwarded to the commissioner.
- The commissioner makes a final decision.
- The superintendent is notified in writing of the commissioner's decision and the rationale upon which the decision was made. The decision of the commissioner is final and is not subject to further negotiation at this point. The commissioner will respond in writing to each appeal received.
- If an appeal is granted, the data upon which the appeal was based will not be modified. Accountability and AEIS reports, as well as all other publications reflecting accountability data, must report the data as they are submitted to the TEA. Accountability data are subject to scrutiny by the Office of the State Auditor.
When a rating is changed due to a granted appeal, the letter from the commissioner serves as notification of the official rating for the district or campus. Districts may publicize the changed rating at that time. The agency website and other state accountability products will be updated after the resolution of all appeals. This update will occur in October 2007 concurrent with the release of the Gold Performance Acknowledgments. Note that the update will reflect only the changed rating; the values shown on the report, such as percent met standard, are never modified. Between the time of receipt of the letter granting an appeal and the update of agency state accountability products, the agency sources will not reflect the changed campus or district rating.
2007 Accountability | Accountability | Performance Reporting | TEA Home