_Section VI- Special Issues and Circumstances_ |
The vast majority of district and campus accountability ratings can be determined through the standard evaluation process detailed in Sections I - III - The Basics. However, there are special circumstances that require closer examination. Accommodating all Texas campuses and districts increases the complexity of the accountability system, but it also increases the fairness of the ratings ultimately assigned. This section describes pairing, special analysis, and the treatment of non-traditional campuses and their data in the accountability system.
All campuses serving grades 1-12 must receive an accountability rating. Beginning in 1994, campuses with no state assessment results due to grade span served were incorporated into the accountability system by having districts choose another campus within the same district with which to pair for accountability purposes. The campuses shared TAAS data. The pairing process is continued in the new accountability system. A new pairing feature, available for the first time in 2004, is that districts also have the option to pair a campus with the district and be evaluated on the district's results.
TEA determines which campuses need to be paired for any given accountability cycle after analyzing enrollment files submitted on PEIMS Submission 1. All districts with campuses with enrollment in grades higher than kindergarten, and solely in grades with no TAKS data, i.e., grades 1, 2, or 12, receive a request for pairing. Charters are not asked to pair any of their campuses; nor are registered alternative education campuses asked to pair.
For campuses that are paired, only TAKS data are shared. The paired campus is evaluated on any non-TAKS indicator data it may have. Similarly, the campus with which it is paired does not share any dropout, completion, or SDAA data it may have.
Comparable Improvement. In the past, districts were asked to pair campuses with a high grade of 3 in order to compute the TLI growth measures needed for Comparable Improvement (CI). The CI indicators do not exist in 2004, so no pairings associated with CI are required this year.
Required Improvement. In 2004, Required Improvement will be calculated with 2003 data based on the 2004 pairing relationships since no pairings were established in 2003. Thus if a campus is paired in 2004, that same pairing relationship is used to establish 2003 TAKS data for that campus. Similarly, if a campus is not paired in 2004, there will be no pairing relationship for it in 2003. Schools paired in 2004, but with their own 2003 data, will be evaluated on their 2003 data.
Districts have the opportunity to reaffirm prior year pairing decisions and to provide new information by completing special data entry screens on the TEA website. (See samples that follow.) In late March districts with campuses that needed to be paired received instructions on how to access this on-line application. Pairing decisions were due by April 30, 2004.
If a district fails to inform the state, pairing decisions are made by agency staff. In the case of campuses that have been paired in the past, staff will assume that prior year pairing relationships still apply. In the case of campuses identified as needing to be paired for the first time in the 2003-04 school year, pairing selections will be made based on the guidelines given in this section in conjunction with analysis of attendance and enrollment patterns using PEIMS data.
Screen 1 of 3
Screen 2 of 3
Screen 3 of 3
Districts and campuses with small numbers of students pose a special challenge to the accountability system. There are two types of small numbers situations. One is small numbers of students within a group, e.g., few African American test-takers in science. These are handled by applying the minimum size criteria described in previous sections of this manual. The second type is small numbers of total students, that is, few students tested in the All Students category.
Districts and campuses with small numbers of total students raise issues regarding the stability of the data. Special analysis is used to ensure that ratings based on small numbers of TAKS results are appropriate. As a result of special analysis, a rating can remain unchanged, be elevated, or be changed to Not Rated. If special analysis is applied, only All Students performance is examined.
A campus undergoes special analysis if:
- the campus is Academically Unacceptable due to TAKS only, with fewer than 30 All Students tested in one or more of the Academically Unacceptable subject(s); OR
- the campus is Academically Acceptable or Recognized due to TAKS only, and the evaluation is governed by the results of fewer than four All Students tested.
The following are examples of campuses that will NOT undergo special analysis:
- Campuses rated Exemplary.
- Campuses that are Not Rated.
- Campuses that are not small (30 or more testers in all subjects).
- Campuses that have few students tested in TAKS, but whose rating of Academically Unacceptable, Academically Acceptable, or Recognized is due to the other indicators.
Campuses that undergo special analysis fall into three treatment categories: those that receive confidence interval analysis, those that receive professional review based on analysis of available performance data, and those that receive a combination of the two.
The treatment applied depends on the initial rating assigned. Campuses in the special analysis pool that are rated Academically Unacceptable are divided into two groups: those with at least 10 students tested in all subjects and those with fewer than 10 students tested in at least one subject. The confidence interval methodology is applied to the group with 10 or more tested. The group with fewer than 10 tested receives professional review.
Campuses in the special analysis pool that are rated Academically Acceptable or Recognized receive professional review.
Confidence Intervals. For each subject, a new threshold will be computed that represents the statistical equivalent of the Academically Acceptable standard for each unique combination of percent Met Standard and number tested. The percent Met Standard for each Academically Unacceptable subject is compared to the confidence interval threshold. If the campus meets the confidence interval threshold for all its Academically Unacceptable subjects, the rating is raised to Academically Acceptable. If the campus does not meet the confidence interval threshold for at least one subject, it remains Academically Unacceptable.
Professional Review. Professional review involves producing a summary of the district or campus data, analyzing the data, and arriving at a consensus decision. The summary data report includes all available indicator data for both 2003 and 2004. Trends and aggregate data are reviewed. However, because of the small numbers of test takers involved, it can be difficult to assign a rating that is considered reliable and fair. Thus, professional review can result in a Not Rated label for some campuses not otherwise meeting the automatic criteria for Not Rated.
If a new campus (regular or charter) would receive a rating of Academically Unacceptable in its first year of operation, it will be labeled Not Rated: Other. New campuses are not rated Academically Unacceptable before they have prior-year data on which to calculate improvement. For 2004 ratings, a campus is determined to be new if students are reported in membership in the fall 2003 PEIMS submission 1 (for the 2003-04 school year), but the campus has no students in membership in the prior year (fall of 2002). Changes to campus numbers across years within a district or charter will be examined to identify inappropriate use of new campus numbers. See Section V - Responsibilities and Consequences for more information.
Based on fall PEIMS data for the 2003-04 school year, there were 190 charters serving approximately 61,000 students. Most charters have only one campus (147 of the 190); however, some operate multiple campuses. In this publication the term "charter" refers to the charter operator, not an individual charter school. The term "charter campus" refers to an individual school.
By statute, charters are subject to most of the same federal and state laws as other public schools, including reporting and accountability requirements. In the previous accountability system only the campuses operated by the charter received an accountability rating. Beginning with 2004, charters as well as the campuses they operate will be rated, meaning charters will be rated under district rating criteria based on aggregate performance of the campuses operated by the charter. This means charters are also subject to the additional performance requirements applied to districts (underreported student standards and the check for Academically Unacceptable campuses). Because they will be rated, charters will also be eligible for Gold Performance Acknowledgments for the first time.
In 2004, there are some differences between the treatment of charters and traditional districts. These are:
- A charter may be labeled Not Rated: Alternative Education. This will occur in cases where the charter operates one or more registered alternative education campuses. A traditional district will never receive this rating.
- A charter may be labeled Not Rated: Other. This will occur in cases where the charter is new and would otherwise be rated Academically Unacceptable. In most cases, labeling first-year charters Academically Unacceptable would have the affect of rating the campuses operated by the charter in the first year of operation.
- Charters are not asked to pair any of their campuses. Charters are unique in that they either have only one campus or they have multiple campuses with no feeder relationships; therefore, pairing charter campuses is inappropriate.
As with non-charter campuses, a charter campus that is a registered alternative education campus will be rated Not Rated: Alternative Education.
As previously stated, all campuses in the state serving grades 1-12 must receive a campus rating; however, the accountability system recognizes that some campuses offering an alternative program may need to be evaluated under different criteria than regular campuses.
In 2004, alternative education campuses (AEC) are either registered or not registered.
In 2004, registered AECs will be labeled Not Rated: Alternative Education. Campuses obtained their 2004 registration status by responding to a July 29, 2003 letter to administrators from the TEA Associate Commissioner for Quality, Compliance, and Accountability Reviews. In this letter, superintendents were informed that AECs that registered and were eligible in the 2002-03 school year would automatically be registered in 2003-04 unless the district notified the agency otherwise. Campuses not registered for 2002-03 were instructed to follow directions for new registrations. The on-line process opened July 30, 2003 and closed September 15, 2003. Under the guidelines set forth in 2002-03 and 2003-04, possible types of eligible AECs include: local district, charter school, community-based, and shared services arrangement (SSA). AECs that served students residing in residential facilities could have been any one of the four types listed above.
The performance results of students at registered AECs are included in the district's performance and are used in determining the district's rating and acknowledgments.
Campuses providing alternative education programs that either did not seek or were not approved to be registered alternative education campuses are evaluated using the standard evaluation process. These campuses will be rated Exemplary, Recognized, Academically Acceptable, Academically Unacceptable, Not Rated: Other, or Not Rated: Data Integrity Issues.
As with other campuses, the performance results of students at these alternative education programs are included in the district's performance and are used in determining the district's rating and acknowledgments.
Two types of alternative education programs not eligible to register as AECs are: Juvenile Justice Alternative Education Programs (JJAEPs) and Disciplinary Alternative Education Programs (DAEPs).
- JJAEPs. Statute prohibits the attribution of performance results to JJAEPs. For counties with a population of 125,000 or more, Texas Education Code §37.011(h) requires that a student enrolled at a JJAEP be reported as if the student were attending and being tested at his or her "sending" campus. Each district that sends students to a JJAEP is responsible for properly attributing all performance data according to the PEIMS Data Standards and the TAKS testing guidelines.
By statute, procedures for evaluating the educational performance of JJAEPs in large counties are the responsibility of the Texas Juvenile Probation Commission. In the state accountability system, campuses identified to be JJAEPs will be labeled Not Rated: Other. Any accountability data erroneously reported to a JJAEP campus will be subject to further investigation.- DAEPs. Statutory intent prohibits the attribution of performance results to a DAEP. Each district that sends students to a DAEP is responsible for properly attributing all performance data according to the PEIMS Data Standards and the TAKS testing guidelines.
All campuses identified to be DAEPs will be labeled Not Rated: Other. This is a change for 2004; in prior years DAEPs with performance data were rated. Accountability data erroneously reported to a DAEP campus will be subject to further investigation.In cases where JJAEP and/or DAEP data are appropriately attributed to a sending district and campus, the data will affect the accountability ratings of the sending district and campus.
In summary, the performance data for all campuses in a district (including any data erroneously reported to a JJAEP or DAEP) will be included in the district analysis for ratings and the GPA system, regardless of whether the alternative education campus was registered or not.
The development of indicators, standards, methods, and criteria for the evaluation of registered alternative education campuses will take place during 2004-05 and will be used to issue ratings for these campuses in 2005. The registration process and eligibility criteria will be examined as part of the development process. Development of the 2005 rating procedures for alternative education campuses is the responsibility of the Department of Accountability and Data Quality. See Section X-Preview of 2005 and Beyond for more information.
Campuses where all students are served in special education programs and none are tested on TAKS will be labeled Not Rated: Other, because they have no TAKS results on which to be evaluated. See Section III - The Basics: Determining a Rating for more information on the use of this rating label.
Accountability 2004 | Accountability | Performance Reporting | TEA Home