Allen v. District of Columbia, 812 F. Supp. 1239 (D.D.C. 1993)

812 F. Supp. 1239 (1993)

Joseph ALLEN, et al., Plaintiffs,
v.
DISTRICT OF COLUMBIA, Defendant.

Civ. A. No. 92-555 (CRR).

United States District Court, District of Columbia.

February 16, 1993.

Derrick A. Humphries and Leroy T. Jenkins, Jr., Washington, DC, for plaintiffs.

George C. Valentine and Benjamin J. Blustein, Asst. Corp. Counsel, for the District of Columbia, for defendant.

George H. Cohen, Jeremiah A. Collins, and Robert M. Weinberg of Bredhoff & Kaiser, Washington, DC, for intervenors.

CHARLES R. RICHEY, District Judge.

 
I. INTRODUCTION

The question in this case is whether a promotional examination for the positions of Captain, Lieutenant, and Sergeant in the District of Columbia Fire Department ("the Department") complied with the requirements of Title VII of the Civil Rights Act of 1964, as amended, 42 U.S.C. § 2000e et seq. and whether the test, as administered, constituted a violation of the same statute.

The test was developed and constructed by the Test Development Committee ("TDC"),^[1] an independent committee established pursuant to the agreement of the Plaintiffs and their colleagues who were and are a part of the class in Hammon v. Barry, 752 F. Supp. 1087 (D.D.C.1990). The parties in that litigation agreed to the creation and the specific membership of the TDC on June 24, 1986, over six years ago. See Hammon v. Barry, Civ. Action No. 84-903(CRR), Stipulation and Order Regarding Development of Promotion Procedures, June 24, 1986, p. 2-4. The TDC's ultimate cost, paid for by the city, exceeded $400,000.

Trial was held in this case on October 1 and 2, 1992. The Court has carefully considered the submissions of the parties, the testimony of witnesses, the exhibits, the arguments of counsel, the applicable law, and the entire record herein, and concludes that the Plaintiffs have not proved that the promotional test or its administration unlawfully discriminated against them on the basis of race. The promotional test was created by an impartial, independent committee and was scored by experienced senior officials of the Department in such a way that the race and identity of the candidates was kept completely secret to the graders. The examination was a legitimate, neutral employment practice, clearly in keeping with the agreement of the Department to administer a non-discriminatory test in a non-discriminatory manner. Consequently, the Court shall enter judgment on the Complaint for the Defendant. *1240 This Opinion shall constitute the Court's findings of fact and conclusions of law pursuant to Fed.R.Civ.P. 52(a).

 
II. BACKGROUND^[2]

In 1990, this Court approved a settlement agreement in Hammon v. Barry, a suit brought by African-American firefighters against the District of Columbia ("the District") charging the District with discriminatory employment practices. A copy of the settlement agreement, inter alia, is attached hereto and incorporated by reference as Exhibit "A". The settlement provided for a payment of $3.5 million in cash payable to African-American firefighters of the D.C. Fire Department who suffered discrimination on account of their race. The agreement was only entered into after long negotiations by the parties and with the intervention of the Court with the late Herbert R. Reed, former Corporation Counsel for the District of Columbia and Dean of the Howard University School of Law. The agreement also provided for the promotional examination that is the subject of this litigation. The examination was to be used to fill job vacancies for the positions of Sergeant, Lieutenant, and Captain arising after March 1, 1989, and before June 1, 1991. Hammon v. Barry, 752 F. Supp. 1087 (D.D.C.1990).

The examination was prepared by the TDC, which had sole responsibility for the test content, validity, scoring, and reporting. The examination, as originally developed by the TDC, consisted of three parts, the Job Knowledge Test, the Fire Scene Simulation Test, and the Supervisory Style Questionnaire. The TDC members drew on the information they had from promotional examinations administered in Saint Louis, Detroit, and Cleveland in creating the examination, to the extent that the firefighting techniques in those cities were similar to those necessary in the District of Columbia. The Job Knowledge Test was a multiple-choice test based on job manuals and other written materials. The second part, the Fire Scene Simulation Test ("FSS"), contained two different problems. Each problem contained a written description of a fire scene and contained a number of questions which asked the test candidates to describe the decisions they would make and the orders they would give in handling the fire. The third part of the examination, the Supervisory Style Questionnaire, measured the performance priorities of those individuals taking the test and compared them with the priorities of higher-ranking officers of the Department.

The examination was administered by the D.C. Office of Personnel on December 15, 1990. Following the examination, the TDC met with seven senior Department officials to review the test questions. Four of the officials were African-American, and three were white. The TDC and the officials reviewed each of the 108 multiple-choice questions in the Job Knowledge portion of the examination. The officers answered inquiries about the test questions to insure that the questions were job-related; the TDC threw out those questions which it deemed to be not sufficiently job-related. Of the 108 multiple-choice questions on the examination, 60 were retained for scoring purposes by the TDC. The TDC provided an answer key to the D.C. Office of Personnel, which graded the multiple-choice section by mechanical scanner.

The same group of TDC members and senior officials developed model answers for the FSS portion of the examination. Using the model answers as a guide, groups of chiefs graded the FSS answers. For security purposes, a "double-blind" procedure was used for grading the FSS portion of the examination. The candidates placed an identification number on their test booklets; then, employees of the D.C. Office of Personnel inserted a second conversion number on the FSS booklets, which replaced the original identification number. This system insured that the persons scoring the test booklets had no way of knowing the name or the race of the candidate *1241 whose test booklet they were scoring. Of those officers who participated in the grading, twenty were white and four were African-American. McCaffrey Test., Oct. 1, 1992.

During the grading of the FSS portion, the graders raised questions as to how close a candidate's answer had to match the model answer in order to receive credit. On a number of occasions, the graders consulted with the officers who prepared the model answers as to whether particular phrases or words had to be used. McCaffrey Test., Oct. 1, 1992.

The third part of the examination, the Supervisory Style Questionnaire, was not used in scoring the December 1990 examination because some of the candidates had received copies of the Questionnaire before the examination. The TDC determined that, because some of the candidates had retained copies of the Questionnaire during the study period before the examination and used them in study groups, the use of the Questionnaire in scoring the examination would not have been fair to those candidates who did not have advance notice of the Questionnaire.

The scores on the two other portions of the examination, the Job Knowledge test and the FSS test, were weighted approximately equally, Barrett Test., Oct. 1, 1992, and used to generate a final, rank-ordered listing for Captain, Lieutenant, and Sergeant candidates. The D.C. Office of Personnel took these scores, factored in points for service and education (as required by the Hammon Settlement Agreement) and compiled a register of candidates from which promotions were made by the Department in February 1991. The Plaintiffs were not on those lists. The Plaintiffs sought relief from the Equal Opportunity Employment Commission ("EEOC"), which dismissed their charges on December 5, 1991, because of this Court's continuing jurisdiction over the original Hammon Settlement Agreement.

Plaintiffs originally brought this suit, pro se, on March 5, 1992, against the District and against other individuals connected with the Hammon litigation. Plaintiffs sought relief under Title VII of the Civil Rights Act of 1964, as amended, 42 U.S.C. § 2000e et seq., 42 U.S.C. § 1981, and the Fifth Amendment. This Court, during the very early stage of this litigation, dismissed all Defendants except for the District because of the constraints imposed by the doctrines of qualified and judicial immunity. See, e.g., Simons v. Bellinger, 643 F.2d 774 (D.C.Cir.1980); Briggs v. Goodwin, 569 F.2d 10 (D.C.Cir. 1977), cert. denied, 437 U.S. 904, 98 S. Ct. 3089, 57 L. Ed. 2d 1133 (1978). The other Defendants were not liable in the performance of their discretionary functions because the Plaintiffs failed to plead specific facts of any unconstitutional actions or motives on the part of the Defendants. Siegert v. Gilley, ___ U.S. ___, ___, 111 S. Ct. 1789, 1793, 114 L. Ed. 2d 277 (1991); Whitacre v. Davey, 890 F.2d 1168 (D.C.Cir. 1989), cert. denied, 497 U.S. 1038, 110 S. Ct. 3301, 111 L. Ed. 2d 810 (1990). The "bare allegations of malice" were not sufficient to inflict the costs of trial or discovery on the other Defendants in this case. Harlow v. Fitzgerald, 457 U.S. 800, 817-18, 102 S. Ct. 2727, 2738, 73 L. Ed. 2d 396 (1982). The Plaintiffs' allegations failed to state a violation of clearly established law, such that the unlawfulness of the officials' challenged conduct would have been apparent in the light of pre-existing law. Anderson v. Creighton, 483 U.S. 635, 640, 107 S. Ct. 3034, 3039, 97 L. Ed. 2d 523 (1987); Mitchell v. Forsyth, 472 U.S. 511, 526, 105 S. Ct. 2806, 2815, 86 L. Ed. 2d 411 (1985). Consequently, the officials were properly dismissed before the commencement of discovery. Mitchell, at 526, 105 S. Ct. at 2815.

Further, the Court dismissed the claims against the District based on § 1983 and the Fifth Amendment because the Plaintiffs failed to assert that the alleged constitutional violation resulted from a specific "statute, ordinance, regulation, custom, or usage" of the District. In other words, the Plaintiffs could point to no District policy, either official or unofficial, that "caused" the TDC to create an allegedly invalid promotional examination. See Monell v. Department of Social Servs., 436 U.S. 658, 692, 98 S. Ct. 2018, 2036, 56 L. Ed. 2d 611 *1242 (1978). Absent any claim that governmental policy was "the moving force of the constitutional violation," the asserted discrimination contained within the examination could not be attributed to the District. Id. at 694, 98 S. Ct. at 2037; accord Polk County v. Dodson, 454 U.S. 312, 326, 102 S. Ct. 445, 454, 70 L. Ed. 2d 509 (1981); Morgan v. District of Columbia, 824 F.2d 1049, 1058 (D.C.Cir.1987); Miller v. Barry, 698 F.2d 1259 (D.C.Cir.1983).

Finally, the Court dismissed the Plaintiff's claim under § 1981 because the promotions at issue would not have created a "new and distinct relationship" between the firefighters and the Department. Patterson v. McLean Credit Union, 491 U.S. 164, 109 S. Ct. 2363, 105 L. Ed. 2d 132 (1989). The Court also permitted the intervention of Local 36 of the International Association of Fire Fighters, whose members could have also been affected by the outcome of this litigation. See Order, July 2, 1992. Trial on the merits of the Title VII claim began on October 1, 1992 and was concluded on October 2, 1992.

 
III. DISCUSSION 
A. BECAUSE THE PROMOTIONAL EXAMINATION, AS CREATED BY THE TDC, WAS A VALID MEASURE OF THE ABILITIES AND PROBABLE FUTURE SUCCESS OF THOSE INDIVIDUALS TAKING THE TEST, THE EXAM SERVES THE LEGITIMATE EMPLOYMENT GOALS OF THE DEPARTMENT.

The Plaintiffs allege that the promotional examination had a discriminatory impact on African-American firefighters in violation of Title VII of the Civil Rights Act of 1964, as amended, 42 U.S.C. § 2000e et seq. They question the validity of the examination as prepared and as administered. The Defendant denies that any discrimination occurred in either respect.

The parties agree that the legal standard to be applied in this case is set out in Ward's Cove Packing Co. v. Atonio, 490 U.S. 642, 109 S. Ct. 2115, 104 L. Ed. 2d 733 (1989).^[3]Ward's Cove addresses the issue of what constitutes a showing of "disparate impact" sufficient to cause Title VII liability. In order to establish a prima facie case of disparate impact, the Plaintiffs must prove that a facially neutral policy or test has a significant discriminatory effect on the African-American firefighters seeking promotion. See id. at 655, 109 S. Ct. at 2124. The Defendant must then put forward evidence of a business justification for the employment practice. The Plaintiffs then bear the burden of persuasion that the adverse employment action or practice was based on pretext. Id. at 659, 109 S. Ct. at 2126.

The Plaintiffs' first requirement is to show a specific employment practice resulted in the promotion of the candidates "in a racial pattern significantly different from that of the pool of applicants." Albemarle Paper Co. v. Moody, 422 U.S. 405, 425, 95 S. Ct. 2362, 2375, 45 L. Ed. 2d 280 (1975). Although the Plaintiffs did not present the Court with specific numbers, the Plaintiffs showed, through the Report of the TDC and the trial statements of Dr. Barrett, that the promotional examination produced disparity in the number of promotions awarded to African-Americans. Barrett Test., Oct. 1, 1992. The Defendant conceded that African-Americans, as a group, did less well on the examination than did whites in its Motion for Judgment as a Matter of Law.

However, the Plaintiffs have failed to prove that the promotional examination was a pretext for discrimination. In fact, the examination served the legitimate employment goal of helping select qualified candidates for promotion in a neutral fashion. It is undisputed that the examination was part and parcel of the Settlement Agreement in Hammon v. Barry, 752 F. Supp. 1087 (D.D.C.1990). The examination was designed to serve as a "fair, nondiscriminatory, job-related promotional *1243 test" to be used in selecting individuals for promotion. Id. at 1097. Furthermore, the examination was designed by the TDC, an independent group specifically formed for the task. The Defendant has easily met its burden of production in showing the legitimate nature of the challenged examination.

Plaintiffs claim that the use of the examination does not constitute a legitimate business practice. The Plaintiffs have failed to convince the Court, however, that any objections to the substance of the examination render the District's reliance on the examination results improper. The examination still, "in a significant way, serves the legitimate employment goals" of the District. Ward's Cove, 490 U.S. at 659, 109 S. Ct. at 2125-26.

The validity of the examination is supported by both the Special Master and the TDC. On February 4, 1991, the Special Master in the Hammon case issued a report on the validity of the December 15, 1990, examination. A copy of his report is attached hereto and incorporated by reference as Exhibit "B". The Special Master concluded that both the Job Knowledge and FSS portions of the examination were valid. Report of the Special Master at 12-13.

On February 25, 1991, the TDC submitted its report on the December 1990 examination. In the report, the TDC reviewed the development and validity of the examination. The TDC concluded that the examinations given to the Captain, Lieutenant, and Sergeant candidates possessed "content validity" because of the neutral process by which the battalion chiefs evaluated the multiple-choice questions contained in the Job Knowledge portion of the exam and also prepared an answer key for the problems contained in the FSS portion of the exam. TDC Report, Executive Summary at 1-2. In other words, the information sought by the examination was a fair measure of the knowledge relevant to being a successful firefighter because the grading of both parts of the examination was supervised by experienced battalion chiefs.

The TDC also evaluated the examination by another standard, called "criterion-related" validity. The "criterion-related" validity of the examination was measured by comparing the exam scores of selected individuals with personal evaluations of these same individuals provided by battalion fire chiefs. The comparison was designed to measure whether candidates who performed well on the job would score well on the exam. The TDC concluded that the multiple choice questions on the Job Knowledge portion of the examinations given to the Captain, Lieutenant, and Sergeant candidates possessed moderate "criterion-related" validity as to all of the exams. The FSS portion had moderate "criterion-related" validity on the Lieutenant exams, but not on the Captain and Sergeant exams. Barrett Test., Oct. 1, 1992; cf. TDC Report, Executive Summary at 4.

The Plaintiffs assert that, because the FSS portion lacked "criterion-related" validity on the Captain and Sergeant exams, the entire examination must be declared invalid. However, the Plaintiffs fail to give adequate weight to two considerations. First, the TDC found that the FSS portion of the examination, together with the Job Knowledge portion, possessed content validity. Content validity, as Dr. Barrett explained, is a proper method of determining whether the examination was a fair measure of the information necessary for success as a firefighter. In this case, the content validity of the examination was insured by using a racially mixed group of fire officials to evaluate the multiple choice questions and to develop model answers for the FSS portion of the examination. The Court places more weight on the results of this content validity measurement than the "criterion-related" validity measurement because the former method relies on a generalized, neutral evaluation of what information a successful candidate should know; the latter method depends more on subjective evaluations of individual firefighters. See Police Officers for Equal Rights v. City of Columbus, 916 F.2d 1092 (6th Cir. 1990). Cf. 29 C.F.R. § 1607.5 (1991) (accepting both criterion-related and content validity studies as satisfying EEOC guidelines).

*1244 Second, even if the Court were inclined to give more weight to the "criterion-related" validity than the content validity, the "criterion-related" method calls into question only part of the exam. The FSS made up only one half of the examination; the other half, the multiple-choice Job Knowledge test, had moderate "criterion-related" validity for all three candidate classes Sergeant, Lieutenant, and Captain. Thus, the impact of the FSS on the overall examination was lessened. The Court is not convinced that the results of the examination must be considered invalid merely because, under one measure of validity, a portion of the exam lacked only moderate validity as to some of the candidates when the entire examination satisfied another, more objective measure of validity.

The Plaintiffs assert that the content validity of the FSS portion must be called into question because the battalion chiefs who graded the examinations did not strictly adhere to the model answers provided. As Assistant Fire Chief McCaffrey stated in his testimony, during the grading period, the graders raised questions as to how closely the test answers had to match the model answers in order to receive credit. After consultation with the committee which developed the model answers, the graders gave credit for certain answers which, though not identical with the model answers, conveyed the same intent as the model answers.^[4]

The Court does not believe that the slight modifications in the model answers brought out at trial render the FSS portion of the exam invalid. The Plaintiffs failed to show that these modifications would do anything but increase the number of fundamentally correct answers given credit by the graders. Thus, the Plaintiffs have failed to show that the examination is not a valid method to achieve legitimate employment goals and they have failed to meet their burden with respect to their claim that the examination itself was invalid and served only as a pretext.

 
B. BECAUSE THE PLAINTIFFS FAILED TO COME FORWARD WITH ANY EVIDENCE OF CHEATING ON THE EXAM, THE MINOR IRREGULARITIES OCCURRING DURING THE ADMINISTRATION OF THE TEST DO NOT RENDER THE TEST INVALID.

The Plaintiffs, having failed to show that the content of the examination rendered it invalid, next claim that the examination results are invalid because of what they perceive as faulty administration of the examination. The Plaintiffs allege that the entire examination is invalid because of certain minor irregularities that occurred during the course of the examination. The Court finds, however, that the administration problems were not so severe as to invalidate the results of the examination. It is true, for example, that individuals were permitted to go to the restroom unescorted. Drumming Aff. at 2. Some individuals were seen talking in the halls and restrooms during the course of the examination. Wright Aff. at 2. Monitors failed to verify the identity of all those taking the test at all times during the examination. Spaulding Aff. at 2. The Court recognizes that the irregularities that occurred during the examination might have created an opportunity for cheating. However, no witness had personal knowledge of anyone cheating on the examination. The Court cannot draw the conclusion that cheating actually did occur when the Plaintiffs have put forward no evidence to that effect. Furthermore, it is undisputed that the alleged irregularities were not directed towards nor unduly affected African-Americans as compared with candidates of another race.

A few other minor problems arose during the test administration. Some of the *1245 monitors of the examination were allegedly confused as to the examination instructions. Chism Aff. at 2. Some monitors allegedly talked or ate during the examination. Redding Aff. at 1-2. In some rooms, copies of the FSS portion of the exam were allegedly passed out early and immediately re-collected when the mistake was noticed a few minutes later. Rudder Aff. at 1-2. As previously mentioned, these irregularities, to the extent they affected the test candidates at all, affected them equally, regardless of race.

Dr. Barrett, the Defendant's opinion witness, testified that he believed that the test as administered retained its validity. Dr. Theodore Rosen, one of the Plaintiffs' opinion witnesses, concluded that the irregularities had a negative impact. Rosen Decl. at 5. However, the Court agrees with the Defendant that any mistakes which might have occurred were minor and did not affect the results of the examination. This conclusion was shared by the Special Master. Report of the Special Master at 10. Consequently, the Plaintiffs have failed to show that the examination, either as constructed or as applied, is pretextual; the examination was an employment practice based solely on legitimate neutral considerations and is consistent with the requirements of Title VII. See Ward's Cove, 490 U.S. at 659, 109 S. Ct. at 2126.

 
C. BECAUSE THE PLAINTIFFS HAVE FAILED TO PUT FORWARD ANY EVIDENCE THAT OTHER SELECTION DEVICES WOULD EQUALLY SERVE THE DISTRICT'S INTERESTS WITH LESS OF A RACIAL IMPACT, THE PROMOTIONAL TESTS USED IN THIS CASE MUST BE UPHELD.

On October 2, 1992, at the very end of the trial, during the cross-examination of Dr. Barrett, the Plaintiffs asserted the claim that another selection device rather than the test developed by the TDC as agreed upon by the parties would have had a less discriminatory impact on African-American firefighters. While it is true that the Plaintiffs may prevail on their Title VII claim if they can show that "other tests or selection devices, without a similarly undesirable racial effect, would also serve ..." the District's needs, id., the Plaintiffs failed to show that less discriminatory alternatives were available.

The Plaintiffs' entire argument is based on the cross-examination of Dr. Barrett. Dr. Barrett did say that less discriminatory alternatives might exist, but neither he nor the Plaintiffs gave details as to what those alternatives might be. Several options were mentioned: assessment centers, training programs, peer review, etc. But the Plaintiffs failed to put forward evidence that any of these alternative methods for selecting candidates for promotion could reasonably have been used in this case. Further, Plaintiffs failed to show that these alternatives would have been as likely as the promotional examination to select candidates with the appropriate qualifications. Cf. Bridgeport Guardians, Inc. v. City of Bridgeport, 933 F.2d 1140, 1148 (2nd Cir.), cert. denied, ___ U.S. ___, 112 S. Ct. 337, 116 L. Ed. 2d 277 (1991) (finding that the plaintiffs in that case had shown that an alternative method of officer selection with a less discriminatory effect could have been utilized).

The Court also notes that all of the parties in the Hammon litigation had agreed to the development of the promotional examination by the TDC. The TDC and the parties worked diligently after the approval of the settlement agreement to ensure that the promotional examination would be ready to administer on December 15, 1990. Absent any showing that an alternative method was both possible and practical, the Plaintiffs' claim that another method of promotion should have been used in lieu of the examination must fail.

 
IV. CONCLUSION

For the reasons expressed herein, the Court finds that the Plaintiff has not proved its case under Title VII. The Plaintiff failed to show that the examination was a pretext for discriminatory conduct; rather, the examination furthered the legitimate business concerns of the Defendant. *1246 Furthermore, the Plaintiff did not show that alternative methods of promotion exist that would have a less discriminatory effect. Accordingly, the Court shall enter judgment for the Defendant and shall dismiss the above-captioned case from the dockets of this Court. Also, because the Plaintiffs are individuals of limited financial means, a great disparity exists between the financial resources of the Plaintiffs and the Defendant, and the litigation raised issues of public concern, the Court, in its discretion, will not assess costs against the Plaintiffs in this case. See, e.g., Wrighten v. Metropolitan Hospitals, Inc., 726 F.2d 1346 (9th Cir. 1984); Badillo v. Central Steel & Wire Co., 717 F.2d 1160, 1165 (7th Cir.1983). The Court shall enter an Order of even date herewith consistent with the foregoing Opinion.

 
JUDGMENT

Upon consideration of the claims of the parties at the October 1-2, 1992, trial of the above-captioned case, the record herein, and the applicable law, and for the reasons expressed in this Court's Opinion of even date herewith, it is, by this Court, this 16th day of February, 1993,

ORDERED that the Plaintiffs' request for relief shall be, and hereby is, DENIED; and judgment in the above-captioned case shall be, and hereby is, entered for the DEFENDANT; and it is

FURTHER ORDERED that the above-captioned case shall be, and hereby is, DISMISSED from the dockets of this Court, without costs to any party.

 
Exhibit "A" 
Filed Feb. 16, 1993 
SPECIAL MASTER'S PROPOSED FINAL AGREEMENT AMONG HAMMON PLAINTIFFS, BYRNE PLAINTIFFS, AND LOCAL 36 AND THE CITY

This agreement has been drafted by the Special Master, who has for purposes of encouraging discussions, circulated for private consideration other drafts, but who has asked that all prior drafts be considered privileged, preliminary discussion material and not circulated beyond counsel and principal representatives of each party. In an effort to promote settlement and to clear the air with respect to what the Special Master believes would be a fair and equitable but by no means perfect resolution of all claims and disputes, the Special Master has prepared a proposed agreement. It may be shown to anyone and is not intended to be confidential.

It should be clear that no party approved this proposed settlement agreement prior to its circulation. The proposed agreement represents a way of resolving all issues at a cost to the City which the Special Master believes is reasonable and defensible; provides compensation for alleged past discrimination, some of which might otherwise be barred by a statute of limitations in the interests of achieving substantial justice; and assures that no individual is favored or disadvantaged with respect to job opportunities in the Fire Department on the basis of race.

The views of no one party predominate. Some provisions of this agreement are included despite the Special Master's knowledge that they are objectionable to one or even two parties. But, the Special Master is persuaded that the agreement is fair overall, that it provides all parties with knowledge as to what amount of money is to be paid and what other remedies are to be provided, it will provide much needed promotions sooner than any other plan, and the public should find it responsive to its needs as well as the needs of all litigants.

This proposed agreement is divided into two parts. The first relates to hiring and all related claims. The second relates to promotions. If the Hammon plaintiffs, the Byrne plaintiffs and the City together agree on Part B and sign it, And, if the Hammon plaintiffs and the City agree on Part A and sign it, the Special Master will ask the Court to approve the agreement, notwithstanding any opposition from the Byrne plaintiffs or from any other source to the monetary amount provided or any other remedy set forth in Part A. Neither Part A nor Part B will be effective unless both parties are ratified by the parties thereto.

 
*1247 PART A 
SETTLEMENT AGREEMENT AS TO HIRING AND RELATED ISSUES BETWEEN THE HAMMON PLAINTIFFS AND THE CITY 
Summary of Agreement

1. This agreement sets forth all essential terms of the settlement of hiring and related issues (defined as all issues other than future promotions). It does not set forth in detail all procedures and details, which shall be provided in a more comprehensive decree which the Hammon plaintiffs and the City shall develop with the Special Master for submission to the Court.

2. This agreement between the Hammon plaintiffs and the City is executed simultaneously with an agreement on future promotions (Part B) among the Hammon plaintiffs, the Byrne plaintiffs, and the City.

3. This agreement and the simultaneous agreement on future promotions, resolve all claims and binds all members of the Hammon class, and no member of the Class may sue the City or any department or representative, or any party to this settlement with respect to any matter raised in this litigation. The agreement resolves all disputes as to matters arising as of the date it is executed, but does not resolve disputes resulting from future events, tests, decisions, etc. that occur after the date of this agreement.

4. This agreement provides substantial monetary relief to the Hammon class. The City shall pay $3,500,000, to the Hammon class, independent of attorney's fees which may be sought subsequently, and this entire sum shall be distributed to the Hammon class as provided in this agreement.

5. The monetary compensation provided herein has been allocated, to the greatest extent practicable, in accordance with the strength of the individual claims of class members.

6. The amount of compensation $3,500,000 shall be placed in a special fund prior to October 15, 1990, and shall be distributed as soon as possible thereafter in accordance with this agreement, provided that the promotions set forth in Part B, paragraph 5 have been made.

7. The Special Master will work with the parties to provide a detailed decree for consideration by the Court as early as possible.

 
Pre-1980 Compensation

8. Of the $3,500,000, $2,400,000 is allocated to members of the Hammon class who were also members of the Fire Department on January 1, 1980. Each member of the class shall receive compensation for each year from 1962-1979 in which he or she served as a member of the Department. The compensation will be allocated as follows: (a) 70% of the total amount is allocated to the years 1962-1968, (b) 20% of the total amount is allocated to the years 1969-1972, and (c) 10% of the total amount is allocated to the years 1973-1979.

9. Under paragraph 8, $1,680,000 will be allocated to the years 1962-1968, $480,000 to the years 1969-1972, and $240,000 to the years 1973-1979.

10. Allocation of these sums will be made as follows: The amount allocated to each of the three periods will be divided by the total number of firefighter years of the eligible member of the Hammon class. For example, if 50 eligible firefighters each worked all seven years from 1962-1967, and if 50 other eligible firefighters worked five of the years from 1962-1967, the total number of workyears would be 600 (50 × 7 = 350, + 50 × 5 = 250). The $1,680,000 would be divided by 600, and the figure of $2800 per workyear would used as the basis for awarding this portion of the pre-1980 monetary award. An identical process would be used for the periods, 1969-1972, and 1973-1979. The total award for any firefighter eligible for pre-1980 damages would be based on the total of the work year figures for the three periods.

11. In addition to the $2,400,000, an additional sum may be available to be paid as *1248 additional damages for the years 1962-1968, as provided in paragraphs 15 and 17, infra. If any such sum is available, it will be divided by the total work years for this period, and a resulting award for eligible firefighters will be made as a supplementary payment.

 
Post-1980 Claims

12. The total amount available for compensation for post-1980 claims is $1,000,000. This amount will be distributed as stated in paragraphs 13, 14 and 15, infra.

13. Each member of the Hammon class who completed the 1980 hiring examination and who was hired by the Fire Department will receive the following: $150 per month for each month between July 1, 1982 and the actual date of employment as a firefighter.

14. Each member of the Hammon class who completed the 1980 hiring examination and was never hired by the Fire Department will receive the following: $100 per month for each month between July 1, 1982, and the date when the City notified the class member that a position as a firefighter was available. The total paid to these class members shall not exceed $500,000.

15. Should the total amount of the payments made pursuant to paragraphs 13 and 14 be less than $1,000,000, the remainder shall be used to pay supplemental damages for pre-1980 claims for the years 1962-1968, as provided in paragraph 11, supra.

16. The Special Order issued by the Fire Department which provided an EOD date of April 12, 1981 for all firefighters hired from the 1980 list, including all members of the Hammon class, shall remain in full force and effect in the future.

 
Litigation Contributions

17. A total of $100,000 shall be set aside to compensate individual members of the Hammon class for services rendered in this litigation on their own time. Any member of the Hammon class may submit a claim to the Special Master indicating the hours worked, the services rendered, and the amount claimed. The Special Master shall have discretion to award an appropriate amount to any such claimant, and to distribute the $100,000 equitably among claimants. Should the total amount of the distributions made pursuant to this paragraph be less than $100,000, the remainder shall be used to pay supplemental damages for pre-1980 claims for the years 1962-1968, as provided in paragraph 11, supra.

18. The Hammon plaintiffs will proposes a schedule of payments and rationales, which the Special Master will approve under a reasonableness standard.

 
Privacy Rights

19. The Special Master will approve all payments made pursuant to the provisions set forth above, and the total amounts paid are, by the nature of this document, public. Each member of the Hammon class shall have the right to protect the fact that he or she received payment and the amount of any payment from public disclosure.

 
Timing

20. No promotions pursuant to Part B of the settlement agreement may be made until the $3,500,000 has been set aside in a fund, which will be more specifically described in the comprehensive decree to be drafted under paragraph 1, supra.

 
Finality

21. This agreement resolves all claims other than future promotional claims of the members of the Hammon class. Parts A and B together resolve all claims and bind all class members with respect all claims of discrimination as of the date of this agreement.

/s/ Joan A. Burt Joan Burt, Esq. Counsel for Hammon Plaintiffs Joan Burt, Esq. /s/ Herbert O. Reid, Sr. *1249 Corporation Counsel For the District of Columbia
PART B 
SETTLEMENT AGREEMENT AS TO PROMOTIONS AMONG THE BYRNE PLAINTIFFS AND LOCAL 36, THE HAMMON PLAINTIFFS AND THE CITY 
Summary of Agreement

1. This agreement sets forth all essential terms of the settlement of all issues concerning promotions which are in dispute. It will be formalized in a decree to be submitted to the Court.

2. This agreement among the Hammon plaintiffs, the Byrne plaintiffs and Local 36, and the City is executed simultaneously with Part A, an agreement between the Hammon class and the City on all other claims raised by members of the Hammon class. Together Parts A and B resolve all claims and bind all members of the Hammon and Byrne classes, and no member of either class may hereafter sue the City, any department or representative, or any party to the settlement agreement, with respect to any matter raised in this litigation.

3. This agreement resolves all disputes as to matters arising as of the date it is executed, but does not resolve disputes resulting from future events, tests, decisions, etc. that occur after the date of this agreement. In addition, this agreement does not resolve issues regarding attorney's fees and costs, and the parties reserve all rights on these issues.

4. One fundamental goal of this agreement is to enable firefighters who have waited for promotion longer than any party to this case would have wished to be promoted without further delay and to assure that future promotional opportunities will be available on a nondiscriminatory basis as soon as possible, and that equal and expedited opportunities for promotion will exist.

 
Promotions to Vacancies as of March 1, 1989

5. Promotions to Sergeant, Lieutenant and Captain will be made by using the lists attached to the October 2, 1989 Byrne/Hammon Joint Proposal to fill vacancies as of March 1, 1989. All those promoted from these lists shall be eligible to compete for promotion to the next highest rank in the examinations referred to in paragraph 7, infra.

6. A fourth platoon will be instituted by June 1, 1991, with the result that 24 new Sergeant positions, 56 new Lieutenant positions and 9 new Captain positions will be created.

 
Promotional Tests

7. To fill vacancies arising after March 1, 1989 and before June 1, 1991, the City will develop a fair promotional examination for promotions to Sergeant, Lieutenant and Captain that shall be administered and the resulting promotional registers announced on or before February 1, 1991. The period in which materials will be available for this examination may be as short as two months. The Special Master shall work with the City to find a way to meet this deadline, and shall have full authority to compel any of the parties and their consultants to do whatever is reasonably necessary to assure that the examination and announcement of the resulting promotional registers take place no later than February 1, 1991.

8. To fill vacancies arising after June 1, 1991 and for two years thereafter, the City will develop a fair promotional examination for promotions to Sergeant, Lieutenant and Captain that shall be administered and the resulting promotion registers announced on or before August 1, 1991. The period in which materials will be available for this examination may be as short as two months. The Special Master shall work with the City to find a way to meet this deadline, and shall have full authority to *1250 compel any of the parties and their consultants do whatever is reasonably necessary to assure that the examination and announcement of the resulting promotional registers take place no later than August 1, 1991.

9. The development of these examinations is considered to be of the utmost importance, and the parties will cooperate in all reasonable ways to assist in meeting these deadlines and assuring the opportunities for nondiscriminatory promotions are present in the Fire Department.

10. The Hammon and Byrne plaintiffs retain the right to challenge the tests referred to in paragraphs 7 and 8, supra, and any subsequent examinations, the process in which they are developed, and the administration or use of tests.

 
Announcement of Promotions

11. As soon as possible after this agreement is ratified by all parties, members of the Fire Department will be informed of the fact that the City plans to offer the promotional tests set forth in paragraphs 7 and 8, supra.

12. Prior to October 1, 1990, the Special Master shall indicate to the parties the plan he has found to be adequate to provide the tests set forth in paragraphs 7 and 8, supra.

 
Timing

13. Promotions pursuant to paragraph 5, supra, shall be made on or before 30 days after the Court's approval of this settlement agreement, but no promotions pursuant to this settlement agreement may be made until the amount specified in Part A has been set aside in a fund, as provided therein.

/s/ Joan A. Burt Joan Burt, Esq. Counsel for the Hammon Plaintiffs _____________ For the City /s/ Herbert O. Reid, Sr. Herbert O. Reid, Sr. Corporation Counsel For the City /s/ George H. Cohen George Cohen, Esq. Counsel for the Byrne Plaintiffs August 20, 1990

Schedule A  Sergeant
 1. Herlihy
 2. Wilk
 3. Clark
 4. Grace
 5. Foley
 6. Thuman
 7. Sellitto
 8. Blair
 9. B. Washington
10. E.L. Smith
11. Barchers
12. Reid
13. Settle
14. Kelly
15. Benson
16. Norris
17. Strawderman
18. Lokey
19. Talbert
20. McNally
21. Deily
22. Grambo
23. Wood
24. Vermeire
25. Wilson
26. Rudder
27. McCartney
28. Locksley
29. Byrne
30. Jamison
31. Donahue
32. Miller
33. Lopford
34. Tyson
35. Quirk
36. Yocum
37. Moore
38. Gray
39. Bloom
40. Wallace
41. Deaton
42. Dutton
43. Lockman
44. McGurk
45. Heckendorn
46. Flynn
47. Alston
48. Faison
49. Thompson
50. Barnard
51. Kefauver
52. D.R. Smith
53. Tanis
54. C.R. Johnson
55. Knott
56. Gross
57. Gerry Jackson
58. Gallagher
59. McCray
60. Schellenberg
61. Sims
62. Neal
63. Gaskins
64. Weissmueller
65. Mattox
66. Lee
67. R.H. Clark
68. Shipp
69. Fitzgerald
70. Graves
71. Taylor
72. Subacz
73. J.R. Martin
74. A. Crochran
75. T. Burrell
76. J. Monahan

 

*1251
Schedule B  Lieutenant
 1. Funk
 2. Herr
 3. Harney
 4. Goldsmith
 5. Settle
 6. Roberts
 7. Godfrey
 8. Schaefer
 9. Wood
10. Pugh
11. Belcher
12. Wooten
13. Skinner
14. Winokur
15. Pestone
16. Biggs
17. McDonald
18. Mould
19. Anderson
20. Deck
21. Bingham
22. Coughlin
23. Chroniger
24. Whitley
25. James
26. Zollares
27. G. Johnson
28. Haller
29. Proctor
30. E.A. Jackson
31. Beacham
32. McDuffie
33. Dwyer
34. Flowers
35. Dean
36. Diggs
37. Sullivan
38. Jardine
39. Spaulding
40. Williams
41. Wiggleswort
42. Bruce
43. Tyndell
44. High
45. Grant
46. Hall
47. Drumming
48. Warner
49. Waters
50. Stewart
51. Dutton
52. Cochran
53. Handley
54. Lewis
55. Reese
56. T.C. Johnson
57. Bartek
58. P.C. Johnson
59. Robinson
60. Martin
61. M.L. Smith
62. Dunn
63. L. Buck
64. G. Buck
65. Neville
66. Gross
67. Rupard
68. Smith
Schedule C  Captains
 1. Bullock
 2. Galloway
 3. Grimes
 4. Watts
 5. Partain
 6. Arthur
 7. Gilbert
 8. Callaway
 9. Madison
10. Crapo
11. Edwards
12. Chipouras
13. Lancaster
14. Stancovich
15. Ford
16. Garrott
17. Francisco
18. Gould
19. Danner
20. Askins
21. Holmes
22. N. Peterson
23. Gourlay
24. Walsh
25. Moton
26. Sheffield
27. Abell
28. Brown
29. Mansfield
30. Carter
31. D'Amico
32. Harrison
33. Forman
34. Lucas
35. O'Leary
36. Kavelak

 
Exhibit "B" 
United States District Court District of Columbia 
Marvin K. Hammon, et al., Plaintiffs 
v. 
Marion S. Barry, et al., Defendants 
Kevin M. Byrne, et al., Plaintiffs 
v. 
Theodore R. Coleman, et al., Defendants. 
Filed February 16, 1993. 
C.A. 84-0903 (CRR) 
C.A. 85-0782 (CRR) 
SPECIAL MASTER'S REPORT ON DECEMBER 15, 1990 PROMOTIONAL TESTS 
I. Summary of Conclusions

On the basis of his own observations at the test site on December 15, 1990, an examination of special reports and letters written to the Fire Department or to the Special Master, and two informal hearings held on January 8 and 9, 1991, the Special Master has reached the following conclusions with respect to the promotional tests for sergeant, lieutenant and captain:

 

1. There were problems with both the substance and the administration of the promotional examinations. Many of these problems could have been avoided by more careful work on the part of the test drafters and more intensive supervision at the test site.

 

2. Because the Supervisory Style Questionnaire prepared by Dr. Barrett was available to some firefighters, but not all, during the period for exam preparation, and because some firefighters were able to discuss the questionnaire in groups and arrive at collective responses, the questionnaire cannot be used in arriving at test scores for the examinations.

 

*1252 3. Although the multiple choice portion of the test was adversely affected by inclusion of questions which did not relate to material on the list of study materials and some questions which were otherwise problematic, the exclusion of these questions resulted in a job knowledge test which was fair and job-related.

 

4. Although there were various mistakes in the fire scene scenarios, the grading key produced by the Battalion Chiefs who created the key along with Dr. Holmes was sufficiently sensitive to all problems that the fire scene scenarios were a useful part of the examination process and produced tests results that are valid and job related.^[1]

 

5. Despite some of the procedural irregularities in the handing out of the various parts of the examinations and in the directions given, there is no evidence that any irregularity was sufficiently prejudicial to call into question the test results.

 

6. By combining the scores on the multiple choice and fire scene scenarios and excluding the supervisory style questionnaire, and by adding the various additional points provided for in promotional regulations, the Fire Department can arrive at valid promotional registers on or about February 1, 1991, as required by the final order and decree of the District Court.^[2]

 
II. The Basic Problems with the Tests  
A. The Substance of the Tests

The Test Development Committee (TDC) was composed of three experts, Drs. Barrett, Holmes and O'Leary. They were asked to produce a job-related, nondiscriminatory examination that could be administered according to the time frame set forth in the parties' August settlement and ultimately approved by the District Court. The TDC was to determine how best to test the District's firefighters in light of the time deadlines that were in force and the job analyses which it had performed. Ultimately, the TDC decided to offer a test that had three parts.

Dr. O'Leary had principal responsibility for the multiple choice or job knowledge part of the test; Dr. Holmes prepared the fire scene scenarios; and Dr. Barrett created the supervisory style questionnaire. These three parts were created separately, but the District's expectation and understanding was that the TDC agreed on all parts and warranted the quality of the product that was produced.

There is no doubt that there were problems with each part of the test. These are considered seriatim.

With respect to the multiple choice or job knowledge portion, it appears that more than a few questions were based on material that was not on the list of study materials posted by the Fire Department. It appears that the discrepancy between the list and the actual test questions is explained by a change which was made between a draft of the study materials list and the actual final version of the list. But, whatever the reason, questions appearing in the multiple choice portion were invalid because they were based on material not designated in advance on the list. Some other questions arguably were unimportant to District firefighters or raised other problems. Two questions were repeated, and one had no answers. There were also minor typographical errors.

Following the administration of the multiple choice questions, a group of Battalion Chiefs (the same group who created the grading key on the fire scene scenarios) reviewed these questions with Drs. Barrett *1253 and O'Leary. All possible objections were resolved by eliminating test questions. The end result was that sixty questions remained, which were deemed valid and useful by the Chiefs. These questions also are deemed job-related and valid by the TDC's computer analysis.

Thus, the Special Master concludes that the defects in the multiple choice questions were remedied by the post-examination excision of troublesome questions, which was carefully done with full input from a representative group of Battalion Chiefs. This portion of the test is, therefore, valid and should be used in arriving at a final score for test-takers.

It is unfortunate that some firefighters purchased materials which were on an early version of the study list, only to find that they were eliminated on the final version. And, it is also unfortunate that all firefighters had to answer questions based on materials not included on the study list. But, the questions which remained after the Chiefs and the TDC carefully examined the multiple choice part of the test successfully achieved the job knowledge testing.

Hindsight suggests that a number of problems could have been corrected if the TDC had more carefully proofread the multiple choice questions and if there had been closer attention paid to the final list of study materials. The TDC could have and should have done a better job before the multiple choice test was given. But, it also should be noted that there was enormous concern on the part of firefighters, both black and white, about possible leaks with respect to the tests. The TDC wisely determined to take extra precautions against possible advance disclosures of test questions. As part of these precautions, the TDC provided a single copy of the multiple choice questions to the Special Master, who hand-carried them to DCOP. No individual in the Fire Department had advance notice, and none was in a position, therefore, to catch any errors or to notice any variance between the study list and the test questions.

The Special Master met with the Battalion Chiefs who screened the questions after the test was given along with Drs. Barrett and O'Leary. The Chiefs concluded that, once questions were eliminated as not being related to the study material, as not being relevant to a District firefighter, or for other reasons, the remaining questions were sound and resulted in a useful test of which they approved. The Special Master relies heavily on the Chiefs' judgment. Each of the Chiefs took the multiple choice test, each discussed the questions, and each participated in the decision to keep or to eliminate a question. That the Chiefs believe the questions that remained after their discussions with the TDC constitute a good test is a substantial factor in the Special Master's conclusion that the problems with the multiple choice part of the test were substantially ameliorated by the post-test procedure.

No post-test procedure could salvage the supervisory style questionnaire, however. Dr. Barrett pre-tested this questionnaire on various firefighters who, he believed, would not have received an advantage from having taken the test once prior to December 15, 1990, since no firefighter who participated in the pre-test could know what answers were correct or more highly weighted. Although Dr. Barrett's personnel made efforts to assure that copies of the questionnaire were retrieved from those taking the pretest, these efforts apparently were neither uniform nor totally successful. Some firefighters obtained copies of the questionnaire which they retained throughout the study period. How this occurred is not exactly clear, but the informal hearings held by the Special Master revealed that some firefighters requested and obtained the questionnaires from persons who were not identified and who may have been working for Dr. Barrett or for the District.

The informal hearings revealed, as did some of the written correspondence, that some firefighters used the questionnaire in study groups while preparing to take the test. Thus, some firefighters had the benefit of group input in answering questions about which tasks in the department were most and least important. Other firefighters *1254 saw the questionnaire for the first time on December 15, 1990.

Under these circumstances, there is a perception of unfairness in any use of the supervisory style questionnaires. Since some firefighters had weeks to prepare answers and the benefit of group discussion and others had no advance opportunity to prepare it appears that some firefighters had an unfair advantage. The test results, as best the Special Master can determine at this point, bear out the possible advantages of preparation. The scores of those who were pretested differed from their scores on the actual test by more than a random amount.

Although fault is hard to assign for the leak of the questionnaire, the end result is that use of the questionnaire in the determination of test results cannot be deemed fair. This conclusion is now concurred in by the TDC.

Finally, there were problems with the fire scene scenarios. The problems were of several types: 1) Scenarios portrayed equipment which the District does not use e.g., a four inch hose. 2) These scenarios assumed that equipment and personnel were being sent to a scene when standard operating procedures would have required additional equipment and personnel e.g., in the second scenario on the lieutenant's examination, a double local alarm dispatched a team to the scene that did not include a Battalion Fire Chief; the same scenario portrayed a building in which a hand-line was shown operating inside a building with a construction that would have made entry questionable; the first scenario in the captain's examination had a box alarm dispatch with no Battalion Chief, no squad and two engine companies short; that same scene indicated that only 1 tone rang in the firehouse over the radio; the second scenario in the captain's examination dispatched a team that was short an engine company, a Battalion Chief and a squad. 3) In the first scenario of the lieutenant's examination, one of the street names changed by mistake twice. 4) The instructions said that firefighters were "dispatchers," when it was clear they were dispatched. And, 5) a number of grammatical and typographical errors were apparent.

The Special Master met with the Battalion Chiefs who constructed the grading key together with Dr. Holmes in order to consider each of these problems. These Chiefs, unanimously and with enthusiasm, agreed that they had discovered all of the errors called to their attention by the Master and additional errors and had considered them in constructing the grading key. Their unanimous view was that the scenarios were a fair test of a firefighter's ability to address common situations and to deal with uncertainties that may arise. They constructed their grading keys so that answers that relied on standard operating procedures of the District would score high.

The Chiefs concluded that a test-taker could respond to each scenario without being unduly distracted, disadvantaged or prejudiced by the mistaken assumptions concerning the number of engines or the personnel dispatched to the scene. With respect to one scenario in the captain's examination, the Chiefs constructed two grading keys to recognize that there were two ways of approaching the scenario and that both should receive appropriate scores.

Although the Chiefs believed that the mistakes in the scenarios could have been corrected had they been reviewed by District personnel, they also indicated their strong approval of the preparation of the scenarios by an outsider to avoid security problems. The absence of review increased the number of mistakes but enhanced security, and the Chiefs unanimously expressed their belief that the enhanced security was extremely desirable and difficult to overestimate in importance.

As to each mistake, the Chiefs considered whether it might influence an exam taker and how it might, and all problems with the examinations were taken account of in the grading key. The Chiefs' unanimous view was that the scenarios were a good measure of the ability of firefighters to assume positions as officers and that, on *1255 balance, they worked well notwithstanding the distractions caused by the errors.

Several Chiefs indicated awareness that supervisory style questionnaires had been seen in the firehouses before the examination, thus confirming the problem of security identified above. They indicated that a combination of the multiple choice part and the fire scene scenario part of the examination ought to produce a valid examination. This conclusion carries great weight with the Special Master.

 
B. The Administration of the Tests

There were a number of problems with the administration of the tests. Some of these can best be described as inconsistencies in the ways monitors in various rooms handled the tests. With the elimination of the supervisory style questionnaire, the Special Master has concluded that none of the administrative problems was sufficiently serious to undermine the overall fairness of the tests. Essentially, the multiple choice questions were not adversely affected by any administrative actions, and the manner in which the grading key was arrived at minimizes significantly any problems in the administration of the supervisory style questionnaires.

Although it appears that the monitoring could have been improved significantly, no serious allegations of actual cheating have been made. The absence of such allegations is significant, particularly in view of the opportunity provided all firefighters to call any and all problems to the Special Master's attention.

The Battalion Chiefs anticipated some of the possible confusion that might have arisen with respect to the fire scene scenarios and the manner in which answers were to be recorded. Their foresight was not only commendable, but also of importance in eliminating the possible problems in the administration as well as the substance of the scenarios.

 
III. Conclusion

Were it not for the work performed by the Battalion Fire Chiefs, the tests prepared by the TDC might well have proved to be inadequate. The deficiencies that have been noted are deplorable and particularly difficult to understand in view of the fact that the TDC was appointed in 1986 and has had years to inform itself of the District's procedures and to refine tests.

The goal of security did deprive the TDC of the input of the Fire Department once the tests were drafted. And the time deadlines fixed by the settlement and the Court's approval may have required somewhat faster work than is typical. But, the Special Master joins the various firefighters who have stated that many of the errors could and should have been avoided.

Fortunately, the Battalion Fire Chiefs worked carefully on the multiple choice questions and saw that all objectionable questions were deleted from the scoring. This left the remaining questions as a fair, objective test of job knowledge. And, perhaps more fortunately, the Chiefs discovered all of the problems with the fire scene scenarios and worked around them in preparing grading keys that were sensitive the problems and sufficient to assure that test takers would be rewarded for answering questions as their superiors in the District would expect them to be answered pursuant to standard operating procedures.

Because the Battalion Chiefs saved the day, the Special Master has concluded that the multiple choice and fire scene scenarios constitute a valid examination. That it is a valid examination is the judgment of the TDC, and that judgment is supported by the Chiefs who were so integral a part of the grading process. A rank order list will be provided to the District by the TDC. Points will be added to the scores provided by the TDC, tiebreakers will be employed, and the District will announce promotional lists for sergeant, lieutenant and captain on or about February 1, 1991, as provided for in the settlement and in the order and Decree of the District Court.^[3]

*1256 No one need be satisfied that the substance of the tests or the administration was perfect; it was not. No one need be persuaded that the errors in substance or administration were unavoidable; many could have been avoided.

But, the Special Master has considered every alleged defect in the tests and administration, has determined that the supervisory style questionnaire may not be used because of a security breach (a judgment with which the TDC concurs), and that the remaining portions of the test are substantial and sufficient, particularly so in the eyes of the leaders of the Fire Department who carefully examined each aspect of the tests, and constitute a valid promotional examination.

The TDC has determined that the tests are job-related, the tests were constructed in a nondiscriminatory manner by experts agreed to by the parties, and the results should now be implemented so that promotions can be made. The lessons of these examinations should not be forgotten, however, and every effort should be made to assure that the substance and procedures are improved next time around. Indeed, the Special Master strongly believes that precautions must be taken prior to the administration of another set of examinations to assure that the quality of the tests and the manner in which they are administered are beyond reproach.

DATED: January 29, 1991 /s/ Stephen A. Saltzburg Stephen A. Saltzburg Special Master NOTES

[1] The TDC was made up of three individuals, Dr. Richard S. Barrett, of New York, Dr. Lawrence O'Leary, of Saint Louis, and Dr. Carl Holmes, of Oklahoma.

[2] Most of the background facts are undisputed by the parties. See generally Plaintiffs' Proposed Findings of Fact and Conclusions of Law; Defendant's Proposed Findings of Fact and Conclusions of Law.

[3] The Civil Rights Act of 1991 changed some of the burdens of proof set out in Ward's Cove. However, the events giving rise to this suit took place prior to the enactment of the Civil Rights Act of 1991. The parties agree that the Ward's Cove standards are applicable.

[4] One example of this practice was brought out at trial. On one FSS problem, where the model answer contained the language "stop at the hydrant," the language "hit the hydrant" was given credit. Assistant Chief McCaffrey explained that the committee believed that the essential concept was to get water onto the fire. No other examples were presented by either side at trial.

[1] A group of Battalion Chiefs was selected by the Fire Department to work with Dr. Holmes on the grading of the scenarios. The group was racially mixed and, as the Special Master observed, dedicated to a fair examination process. References hereinafter to the Battalion Chiefs or the Chiefs are to this group of Chiefs. Not all Battalion Chiefs participated in the grading process.

[2] It should be emphasized that no one in the Fire Department of the District Government nor the Special Master has any idea of how individuals will be ranked when the lists are announced. Thus, there is no way of knowing who will be favored or disfavored in the grading process. The sole concern is that the test be valid, which the Special Master concludes that it is.

[3] The Special Master has recommended to DCOP that firefighters be permitted to examine their answers to the fire scene scenarios and to see the grading keys. Since the grading key was formulated by a group of Battalion Chiefs who are familiar with the District's operating procedures and who also know the qualities and knowledge that are desirable in officers, there is no reason to question the grading key. The sole purpose for making the booklets available is to provide feedback to firefighters on their test performance and to permit them to know how the Chiefs would have responded to the various scenarios. The Special Master has also suggested that DCOP indicate the points that will be added, if any, for each candidate for promotion so that the candidate has an opportunity to point out any administrative errors that might have been made. The TDC will provide DCOP with a ranking of candidates on a 100 point scale which will be based upon scores on the multiple choice and the fire scene scenario portions of the examination, which scores will be converted to establish the 100 point scale in accordance with what the TDC believes to be sound testing principles.