_Chapter 14 - Appealing the Ratings_ |
Providing superintendents with the opportunity to appeal accountability ratings has been a feature of the state accountability system since 1994. The opportunity to appeal is supported in the 2006 system as well.
Superintendents may appeal the state accountability ratings for both standard and alternative education accountability (AEA) procedures, by following the guidelines provided in this chapter. There are defined time limits under which appeals may be submitted.
June 20, 2006 Dropout/Completion Lists. Superintendents are given access to lists of official dropouts and lists of completion cohort membership. These reports provide a preview of the data that will be used to calculate the Annual Dropout Rate and Completion Rate I base indicators for the state accountability ratings. July 21, 2006 Preview Data Tables. Superintendents are given access to preview accountability data tables for their district and campuses showing all state accountability indicator data. Principals and superintendents can use these data tables to anticipate their campus and district accountability ratings. Appeals may be submitted by the superintendent after receipt of the preview data tables. August 1, 2006 Ratings Release. Due to the short timeline between the transmittal of the preview data tables and the ratings release date, no appeals will be resolved before the ratings release. August 18, 2006 Appeals Deadline. Appeals must be postmarked no later than August 18, 2006 in order to be considered. Late October, 2006 Ratings Update. The outcome of all appeals will be reflected in the ratings update scheduled for October, 2006. At that time the TEA website will be updated. A more detailed calendar can be found in Chapter 18 - Calendar.
The numbers shown on the data tables (and later on other agency products, such as the AEIS reports) are final and cannot be changed, even if an appeal is granted.
Appeals should be based upon a data or calculation error attributable to the Texas Education Agency, regional education service centers, or the test contractor for the student assessment program. However, problems due to district errors in PEIMS data submissions or on TAKS answer sheets are considered on a case-by-case basis. Also, statute permits consideration of data reporting quality in evaluating the merits of an appeal. Poor data quality is not a valid reason to appeal.
Only appeals that would result in a changed rating will be considered.
Appeals that follow these guidelines are not guaranteed to be granted. Each appeal is evaluated based on the details of its unique situation. Well-written appeals that follow the guidelines are more easily processed, but they are not necessarily granted.
A strength of the state accountability system is that the rules are applied uniformly to all campuses and districts. Petitions to make exceptions for how the rules are applied are viewed as unfavorable for appeal. Examples of situations unfavorable for appeal follow. Some examples apply to both standard and AEA procedures. Some are unique to one set of procedures or the other.
Examples applicable to both standard and AEA procedures:
- Campus Mobility. Requests to include the performance of students excluded due to the appropriate use of the campus mobility subset criteria.
- Grade 3 and Grade 5 Cumulative. Requests to alter the TEA methodology for combining the first and second administrations of grade 3 reading results, or the first and second administrations of grade 5 reading and mathematics results.
- Rounding. Requests to compute Required Improvement, student group percents, or indicator values using rounding methodology different from that described in this Manual.
- Minimum Size Criteria. Requests to evaluate student groups using minimum size criteria different from those described in this Manual.
Examples applicable to standard procedures:
- Exceptions Provision. Requests for additional exceptions or changes to the application of the Exceptions Provision.
- Pairing. Requests to alter pairing relationships that districts agreed to prior to April 28, 2006.
- New and Academically Unacceptable. Requests to assign the Not Rated: Other label to campuses that are Academically Unacceptable in their first year of operation.
- Floors. Requests to waive the floor requirements when applying either the Exceptions Provision or Required Improvement.
Examples applicable to AEA procedures:
- Late Registration Requests. Requests after August 31, 2005 to be registered as an alternative education campus (AEC) in order to be evaluated under AEA procedures.
- At-risk Criterion. Requests by AECs or charter operators that did not meet the at-risk criterion or applicable safeguards in 2005-06 to be evaluated under AEA procedures, instead of standard procedures.
If a problem is identified with data received from the test contractor, the TAKS data may be appealed. An appeal of the TAKS indicators should reflect a serious problem such as a missing grade level or campus. However, coding errors on TAKS answer sheets will be considered on a case-by-case basis.
- If the district has requested that the writing results be re-scored, a copy of the dated request to the test contractor should be provided with the appeal.
- If other serious problems are found, copies of correspondence with the test contractor should be provided with the appeal.
As with TAKS appeals, an appeal of the SDAA II indicator should include copies of any correspondence with the test contractor. Other information available to the agency about special education students will be used in evaluating SDAA II appeals; for example, Performance-Based Monitoring (PBM) indicators pertaining to SDAA II will be examined in concert with the supporting documentation provided by the district.
The dropout rate indicators are based on 2004-05 leaver data submitted for students in grades 7 and 8 (standard ratings) and 7 through 12 (AEA ratings). This information was reported by districts on submission 1 of the 2005-06 PEIMS data collection. Districts and campuses are held accountable for their official dropouts. Official dropouts are those students who:
- were reported by the district with leaver codes identifying the student as a dropout; and,
- were not located in other educational settings through the TEA automated comparisons of leaver data against other state data sources. For example, students found to be enrolled in a Texas public school or to have graduated or to have earned General Educational Development (GED) certificates are not included in the count of official dropouts.
In addition, the agency determines the appropriate campus of accountability (COA) for dropouts reported on campuses not permitted to have dropouts attributed to them (such as Juvenile Justice Alternative Education Program and Disciplinary Alternative Education Program campuses). See Appendix D - Data Sources for a list of the leaver codes that designate students as dropouts for accountability purposes.
The agency also determines the appropriate district of accountability (DOA) for certain dropouts reported in pre- or post-adjudication facilities or in residential treatment centers. The agency has developed rules to determine and assign responsibility for the dropout to a district the student previously attended, other than to the district where the facility is located. See Appendix D - Data Sources for more details about the COA and DOA processes.
Other Information:
- As shown in the calendar, in June the agency will provide superintendents access to lists of their official dropouts. For standard ratings, only students shown as official grade 7-8 dropouts on these lists may be appealed. For AEA ratings, only students shown as official grade 7-12 dropouts on these lists may be appealed. Also included are the reported dropouts who were located through the statewide searches. An explanation of why these dropouts are not part of the official dropout list is provided.
- Dropouts who have been designated as official dropouts but who are located by a district after the PEIMS resubmission due date (January 19, 2006) cannot be appealed. Only the status of a reported leaver by the resubmission deadline is relevant to a dropout appeal. This policy ensures that all districts have an equal opportunity to locate leavers.
- No more than ten official dropouts may be appealed for any campus or district.
- Data quality will be a consideration in evaluating the merits of a dropout rate appeal. Poor data quality is not a valid reason to appeal.
The completion rate indicator for the class of 2005 is based on the status of students who first attended 9th grade in the 2001-02 school year. A student's final status is determined to be either graduated, received a GED, continued high school, or dropped out. All data used to calculate longitudinal completion rates are derived from PEIMS data submitted by districts between 2001 and 2006 and the statewide GED file. See Appendix D - Data Sources for details of the PEIMS records used to calculate the completion rate.
As shown in the calendar, in June the agency will provide districts with access to lists of all students in their class of 2005 completion cohort. Only students shown in these lists may be appealed for the completion rate indicator. The final status of each student in the completion cohort will be provided. For the numerator, students with a final status of graduated and continued high school are counted as "completers" under standard procedures. Under AEA procedures students who received a GED are also counted as "completers". The denominator of the rate calculation is the sum of the students who graduated, received a GED, or continued high school, plus the students with a final status of "dropout." The list also includes two groups that are not part of the denominator-members of the cohort who left Texas public schools, and students with identification errors.
The status of no more than ten non-completers or one percent of the non-completers in the cohort (whichever is larger) may be appealed for any campus or district.
Data quality will be a consideration in evaluating the merits of a completion rate appeal. Poor data quality is not a valid reason to appeal.
Gold Performance Acknowledgments (GPA) cannot be appealed. Campuses or districts initially rated Academically Unacceptable are automatically eligible for GPA if their rating is later raised on appeal.
As described in Chapter 2 - The Basics: Additional Features, a district is prevented from being rated Exemplary or Recognized if it exceeds the standards for either the number or percent of underreported students. There is no minimum size criteria employed with respect to the number of underreported students. If a district exceeds the 2.0 percent standard for percent underreported due to a very small number of underreported students, the Commissioner of Education will consider a ratings appeal.
Grade 11 assessments are administered multiple times during the school year. For accountability purposes, the performance of all juniors tested for the first time during the primary spring administration and some juniors testing for the first time during other administrations are included. (See Chapter 2.) A district may appeal to include additional grade 11 results for first-time tested students as part of the TAKS base indicator. These appeals will be evaluated on a case-by-case basis. As with all appeals, no changes will be made to the data shown on the reports.
The assessment results of students displaced due to either Hurricane Katrina or Hurricane Rita will be removed from the accountability data as stated in the October 12, 2005 letter from the commissioner. The Katrina-Rita Indicator (KRI) code that is collected on the TAKS and SDAA II answer documents will be used to determine which scores to exclude.
Appeals that petition for rating changes due to problems with KRI coding will be evaluated on a case-by-case basis. Requests to change a student's KRI value will be evaluated against prior year attendance information to help confirm or refute the initial code value reported.
Requests to include results of students properly coded as KRI are unfavorable for appeal.
With the assistance of Education Service Center (ESC) staff in five targeted regions of the state, districts were identified as eligible for a special hurricane provision in the 2006 accountability system. A district impacted by Hurricane Rita, yet not identified may appeal to be afforded the same considerations as the identified districts if there are unique circumstances that warrant additional review. Also, districts and campuses with Not Rated: Other ratings due to application of the hurricane provision may appeal to have the system-generated rating applied instead. (See Appendix I.)
Superintendents appealing an accountability rating must transmit a letter that includes the following:
- A statement that the letter is an appeal of the 2006 state accountability rating;
- The name and ID number of the district and/or campuses for which the appeal is being submitted;
- The specific indicator(s) appealed;
- The problem, including details of the data affected and what caused the problem;
- If applicable, the reason(s) why the cause of the problem is attributable to the Texas Education Agency, a regional education service center, or the test contractor;
- The reason(s) why the change would result in a different rating, including calculations that support the different outcome;
- A statement that all information included in the appeal is true and correct to the superintendent's best knowledge and belief; and,
- The superintendent's signature on official district letterhead.
Other Information:
- Appeals for more than one campus within a district may be included in the same letter.
- Appeals for more than one indicator may be included in the same letter.
- Appeals of ratings issued under both standard and AEA procedures may be included in the same letter.
- Districts have only one opportunity to appeal each indicator for any campus or the district.
- When student-level information is in question, supporting information must be provided for review, i.e., a list of the students in question by name and identification number. It is not sufficient to claim data are in error without providing information with which the appeal can be researched and evaluated.
- It is the district's responsibility to ensure all relevant information is included in the appeal as districts will not be prompted for additional materials.
- Envelope should be addressed to the Division of Performance Reporting as follows:
Your ISD
Your address
City, TX zip
stamp
Division of Performance Reporting
Texas Education Agency
1701 North Congress Avenue
Austin, TX 78701-1494
Attn: Accountability Ratings Appeal
- Appeal letter should be addressed to Dr. Shirley Neeley, Commissioner of Education (see letter examples, below).
- Appeal letter must be postmarked on or before August 18, 2006. Appeals postmarked after this date will not be considered.
- Do not send multiple copies of the appeal letter and/or supporting documentation.
- Districts are encouraged to obtain delivery confirmation services from their mail courier.
- Examples of satisfactory and unsatisfactory appeals are provided on the following page for illustration.
| Satisfactory Appeal: | Unsatisfactory Appeals: |
Dear Commissioner Neeley, This is an appeal of the 2006 state accountability rating issued for Elm Street Junior High (ID 123456789) in Elm ISD. Specifically, I am appealing the grade 7-8 annual dropout rate that was used to assign a rating of Academically Unacceptable to this school. I have analyzed the leaver information for Elm Street Junior High and believe that one student counted as an official dropout in the statewide record reconciliation and assignment system should not have been counted. This student left Elm Street Junior High last spring but we did not find him enrolled in another district in PID Enrollment Tracking, and we didn't receive a request for records until after the PEIMS resubmission due date. However, I have reason to believe that this student has been enrolled at the transfer district since the beginning of the school year. Unfortunately, this student received a Z-ID during the leaver record processing, which is why I believe that this student could have been reported in current year enrollment but not matched. Attached is pertinent information to this appeal: Student name, student identification numbers, date of birth, and transfer documentation are provided. Assigning this record as other leaver rather than dropout should raise the school's rating to Academically Acceptable. By my signature below, I certify that all information included in this appeal is true and correct to the best of my knowledge and belief. Sincerely, J. Q. Educator Superintendent of Schools attachments |
Dear Commissioner Neeley, I have analyzed the leaver information for Elm Street Junior High and believe that one student should not have been counted as an official dropout in the statewide record reconciliation and assignment system. I have reason to believe that this student has been enrolled at the transfer district since the beginning of the school year even though a request for records was not received until February. Sincerely, J. Q. Educator Superintendent of Schools [no attachments] |
Dear Commissioner of Education, I have analyzed the dropout list for Elm Street High School and wish to appeal the status of 15 dropouts. Most of these students, I believe, are back in school as of May 2006. The remaining students are either gone from the state or have left the country. Please revise my 2005 rating in light of this information. Sincerely, J. Q. Educator Superintendent of Schools [no attachments] |
Once an appeal is received by the Division of Performance Reporting, the process for evaluating the information will be followed as outlined below:
- The details of the appeal are entered into a database for tracking purposes.
- Researchers evaluate the request using agency data sources to validate the statements made to the extent possible. The agency examines all relevant data, not just the results for the students specifically named in the correspondence.
- Researchers analyze the effect that granting a campus appeal may have on other campuses in the district (such as paired campuses), whether they are specifically named in the appeal or not. Similarly, the effect that granting a campus appeal may have on the district is evaluated, whether the district is named in the appeal or not. In single-campus districts, both the campus and the district are evaluated, whether the district submits the appeal as a campus or district appeal.
- Staff prepares a recommendation and forwards it to an external panel for review. This review panel will provide independent oversight to the appeals process.
- The review panel examines the appeal, supporting documentation, staff research, and the staff recommendation. The panel determines its recommendation.
- The panel's recommendation is forwarded to the commissioner.
- The commissioner makes a final decision.
- The superintendent is notified in writing of the commissioner's decision and the rationale upon which the decision was made. The decision of the commissioner is final and is not subject to further negotiation at this point. The commissioner will respond in writing to each appeal received.
- If an appeal is granted, the data upon which the appeal was based will not be modified. Accountability and AEIS reports, as well as all other publications reflecting accountability data, must report the data as they are submitted to the TEA. Accountability data are subject to scrutiny by the Office of the State Auditor.
When a rating is changed due to a granted appeal, the letter from the commissioner serves as notification of the official rating for the district or campus. Districts are free to publicize the changed rating at that time. The agency website and other state accountability products will be updated after the resolution of all appeals. This update will occur in October 2006 concurrent with the release of the Gold Performance Acknowledgments. Note that the update will reflect only the changed rating; the values shown on the report, such as percent met standard, are never modified. Between the time of receipt of the letter granting an appeal and the update of agency state accountability products, the agency sources will not reflect the changed campus or district rating.
Accountability 2005 | Accountability | Performance Reporting | TEA Home