In the Matter of Police Sergeant City of Paterson

SYLLABUS

(This syllabus is not part of the opinion of the Court. It has been prepared by the Office of the Clerk for the convenience of the reader. It has been neither reviewed nor approved by the Supreme Court. Please note that, in the interests of brevity, portions of any opinion may not have been summarized).
 

In the Matter of Police Sergeant (PM3776V) City of Paterson (A-138-01)
 
 
Argued February 3, 2003 -- Decided April 16, 2003

ZAZZALI, J., writing for a unanimous Court.

This appeal involves a civil service examination administered by the Department of Personnel (DOP) for promotional consideration for nine sergeant positions in the City of Paterson Police Department. The Court considers whether the DOP s practice of administering identical exams to original and make-up candidates in the same testing cycle is a per se violation of a candidate s state constitutional right to a fair and competitive civil service examination. The Court further considers whether the make-up exams, as administered, violate the State Constitution when there is evidence that unknown persons disseminated test questions and answers throughout the Paterson Police Department immediately following the original exam and before the make-up exam, but no evidence that any make-up candidates had advance knowledge of the test s content.

In October 1997, the DOP administered a statewide civil service promotional exam for the position of police sergeant. A total of 182 candidates from the City of Paterson Police Department took the exam. One hundred thirty-five of those candidates were successful. Petitioners, consisting of numerous candidates for the position, claimed that immediately following the exam, candidates congregated outside of the testing area and discussed the content of the exam with one another. In addition, there was evidence that questions appearing on the exam, along with answers to them, were later typed out by some of the candidates who sat for the exam and distributed them within the police department. Furthermore, petitioners claimed that various unidentified members of the police department discussed the October exam, making its content common knowledge throughout the Department prior to the administration of any make-up exams.

Shortly after the exam, petitioners filed a petition with the DOP requesting that it prohibit any candidate from taking a make-up exam for the October 1997 test, claiming that they had been disadvantaged by the dissemination of information contained on the exam. In support of their request, petitioners provided the DOP with copies of a substantial portion of the questions and answers from the October 1997 exam as proof that exam security had been breached. The make-up exam nevertheless was administered to three candidates on different occasions after the candidates signed a Security Pledge certifying that they had no knowledge of the contents of the original exam.

In total, eight officers in the Paterson Police Department were promoted to sergeant: five out of the 135 candidates who passed the original exam and three out of the three candidates who took a make-up exam. In July 1998, petitioners filed a petition with the DOP and the Merit System Board (Board) asserting that the DOP s practice of administering identical exams to original and make-up candidates violated their right to a fair and competitive civil service examination under the New Jersey Constitution. Petitioners also sought the removal of two of the three successful make-up candidates (Durkin and Catania), alleging that they had advance knowledge of the exam s content. Again, petitioners provided copies of questions and answers to the October 1997 exam, which they found at roll call in the Paterson Police Department shortly after the original exam but before the make-up exams.

The Board rejected the petitioners constitutional challenges to the DOP s practice of administering identical exams to original and make-up candidates. Rather, the Board viewed the practice as the only mechanism it had to provide candidates with a fair and competitive examination and to provide the statistically valid means to compare the performance of the original candidates and the make-up candidates fairly and accurately. The Board referred the petitioners specific allegations against Durkin and Catania to the Office of Administrative Law for a hearing to determine whether they had knowledge of the exam s content prior to their make-up exams.

In July 2000, an administrative law judge granted Catania s and Durkin s motion for summary judgment, concluding that there was no evidence that they had advance knowledge of the contents of the exam. Petitioners appealed the ALJ s decision, and the Board affirmed in October 2000.

In an unreported per curiam opinion, the Appellate Division affirmed the Board s decision, rejecting petitioners general challenge to the DOP s practice of using identical questions on original and make-up exams and affirming the Board s finding that there was no evidence to support the conclusion that the exam, as held, was unfair.

The Supreme Court granted the petitioners petition for certification.

HELD : Although the Department of Personnel s practice of administering identical exams to original and make-up candidates in the same testing cycle was not a per se violation of a candidate s constitutional right to a fair and competitive civil service examination, in light of the undisputed evidence of a breach in security following the original exam but before the administration of the make-up exam and the DOP s failure to make adjustments to the make-up exams to address that breach, the make-up exams as administered in Paterson, violated the New Jersey State Constitution.

1. The New Jersey Constitution and the New Jersey Civil Service Act require appointments and promotions in the civil service of the State to be based on considerations of merit and relative knowledge, skill, and ability. To implement those constitutional and statutory requirements, the Legislature has vested the DOP, the Board, and the DOP Commissioner with broad power to devise a fair, secure, merit-based testing process by which candidates are selected for employment and promotion. Among other things, the Commissioner must provide for the security of the examination process and appropriate sanctions for breach of security. (pp. 9-10)

2. In the past, the Court has sanctioned the DOP s practice of reusing a substantial number of exam questions from test to test, in identical or similar form, recognizing that the DOP s decision to reuse exam questions was clearly within the range of responsibilities that the Legislature has delegated to the DOP to implement the most effective and efficient procedure to assure public hiring and promotion based on merit. (pp. 10-12)

3. Noting its limited role in reviewing the DOP s administration of the civil-service system and its determination regarding civil-service testing processes, the Court declines to bar outright the DOP s practice of administering identical exams in the same testing cycle. (pp. 12-14)

4. To effectuate the goal of devising a fair, secure, merit-based testing process by which candidates are selected for employment and promotion, the DOP Commissioner must provide for the security of the examination process and appropriate sanctions for breach of security. To that end, the Board specifically prohibits copying, recording, or transcribing any examination question or answer, and/or the removal from any exam room of any question sheet, answer sheet, notes, or other papers or materials related to the content of an examination. (pp. 15-18)

5. Other jurisdictions also have recognized the importance of a fair, secure, and competitive civil service examination process. (pp. 18-19)

6. Although there is no evidence indicating that any of the three make-up candidates had access to exam materials prior to sitting for their make-up exams, it is undisputed that exam security was breached prior to the administration of the make-up exams. Once the DOP discovered that breach in exam security, the make-up exams should have been cancelled and an appropriate remedy fashioned to ensure all candidates a fair and competitive exam. Its failure to do so undermined the integrity of the examination process and impaired the candidates ability to demonstrate their relative merit and fitness. Thus, the make-up exams, as administered in Paterson, violated article VII, section 1, paragraph 2 of the State Constitution. (pp. 19-20)

7. Although security can and must be improved, it may be that no procedural safeguards can prevent the type of irregularities that occurred in this appeal, where identical exams were given to both original and make-up candidates. Therefore, going forward, the DOP and the Board should administer make-up exams that contain substantially different or entirely different questions from those used in the original examination. Moreover, exam participants must honor their legal and ethical obligations. (pp. 21-23)

8. Since the Court is voiding the results of the make-up exam and thereby prohibiting any permanent appointments to police sergeant from those candidates who sat for the make-up exams, three police sergeant positions, currently held by Officers Catania, Durkin, and Mejia, are open. Until a new annual exam is given, the Court leaves to the discretion of the DOP the question of who, if anyone, should fill the open sergeant positions, as well as the procedures for those appointments. This remedy is confined to the October 1997 sergeant s exam administered to the members of the Paterson Police Department. (pp. 23-24)

Judgment of the Appellate Division is AFFIRMED in part in respect of its holding that identical exams for original and make-up candidates is not per se unconstitutional, REVERSED in part in respect of its holding that the make-up exams as administered were fair and competitive, and REMANDED for appropriate relief consistent with the Court s decision.

CHIEF JUSTICE PORITZ and JUSTICES COLEMAN, LONG, VERNIERO, LaVECCHIA, and ALBIN join in JUSTICE ZAZZALI s opinion.

SUPREME COURT OF NEW JERSEY
A- 138 September Term 2001
 
 
IN THE MATTER OF
POLICE SERGEANT (PM3776V)
CITY OF PATERSON.

Argued February 3, 2003 Decided April 16, 2003

On certification to the Superior Court, Appellate Division.

Joseph S. Murphy argued the cause for appellants, Troy Oswald, Ronald Altman, Mason Maher, Steve Oliver and John Robb.

George N. Cohen, Deputy Attorney General, argued the cause for respondent, Merit System Board (David Samson, Attorney General of New Jersey, attorney; Michael J. Haas, Assistant Attorney General, of counsel; Todd A. Wigder, Deputy Attorney General, on the brief).

The opinion of the Court was delivered by
ZAZZALI, J.
 
This appeal involves a civil service examination administered by the New Jersey Department of Personnel (DOP) for promotional consideration for nine sergeant positions in the City of Paterson Police Department. We must determine whether the DOP s practice of administering identical exams to original and make-up candidates in the same testing cycle is a per se violation of a candidate s right to a fair and competitive civil service examination under article VII, section 1, paragraph 2 of the New Jersey Constitution. We also must determine whether the make-up exams as administered violate the State Constitution when there is evidence that unknown persons disseminated test questions and answers throughout the Paterson Police Department immediately following the original exam and before the make-up exams, but no evidence that any make-up candidates had advance knowledge of the test s content.
The DOP, an administrative law judge and the Merit System Board (Board) each concluded that the make-up exams as conducted did not violate the State Constitution. The Appellate Division affirmed. It also held that the DOP s practice of administering identical exams to original and make-up candidates in the same testing cycle is not per se unconstitutional.
Although we have serious reservations regarding the DOP s practice of administering identical exams to original and make-up candidates in the same testing cycle, we agree with the Appellate Division that that practice is not per se unconstitutional. We disagree, however, with the Appellate Division s finding that the make-up exams as administered did not violate petitioners right to a fair and competitive examination under our State Constitution. Allowing candidates to take a make-up exam, despite direct evidence that virtually all of the questions from the original exam had been disseminated throughout the police department, undermined the integrity of the examination process and impaired candidates ability to demonstrate their relative merit and fitness. We therefore affirm in part, reverse in part, and remand for appropriate remedial relief.

I.
 
 
In October 1997, the DOP administered a statewide civil service promotional examination for the position of police sergeant. A total of 182 candidates from the City of Paterson Police Department, including petitioners Troy Oswald, Ronald Altman, Mason Maher, Steve Olivo, and John Robb, III, took the examination. Of that number, 135 candidates passed the exam. The examination consisted of seventy-one evenly weighted, multiple-choice questions. The successful candidates names were placed on the eligible list for nine available sergeant positions in the Paterson Police Department. The DOP ranked the candidates in descending order according to a final average score that consisted of two weighted parts: the test score, which comprised 80% of the weighted average, and seniority, which comprised 20%.
Petitioners claim that immediately following the examination candidates congregated outside of the testing area and discussed the content of the exam with one another. They also claim that candidates wrote down questions and answers that appeared on the exam and reported that information to their respective study groups. Within a few days after the exam, unknown persons had typed out sixty-five of seventy-one questions on a sheet of paper and disseminated copies throughout the Paterson Police Department. Further, various unidentified members of the Paterson Police Department discussed the October 1997 sergeant s exam, making its content common knowledge throughout the Department prior to the administration of any make-up exams.
Shortly after the exam, petitioners, joined by 101 other candidates, filed a petition with the DOP requesting that it prohibit any candidate from taking a make-up exam for the October 1997 test. The signatories attested that they did not release any information on the examination but that other departments taking [the] same examination . . . have released the questions, and therefore disadvantaged the original candidates relative to potential make-up candidates.
Officers Marco Catania, James Durkin, and Alvaro Mejia requested make-up administrations of the October 1997 sergeant s exam for medical reasons. Each candidate submitted a doctor s certification, which indicated that each suffered from physical conditions that precluded participation in the original administration of the exam. The DOP evaluated the officers claims and subsequently granted their requests. The DOP conducted separate administrations of the make-up exam for each of the three candidates on different dates between November 1997 and June 1998. Consistent with DOP procedure for administering make-up exams, it provided the candidates with exams identical to the October 1997 test. See N.J.A.C. 4A:4-2.9(f). Each candidate signed a Security Pledge attesting that he had no knowledge of the content of the examination as a result of information gained from or furnished by other candidates who participated in the original examination. See N.J.A.C. 4A:4-2.9(h). The DOP then inserted the make-up candidates scores into the original distribution of candidate scores and ranked them accordingly on the eligible list. See N.J.A.C. 4A:4-2.9(g).
In November 1997, Catania took a make-up exam identical to the October 1997 sergeant s exam and placed second on the eligible list. In February 1998, William Dolan, Director of Public Safety in Paterson, advised the DOP that conversations and rumors were circulating in the Paterson Police Department that Officer Catania might have had access to exam materials prior to taking his make-up exam. In response, the DOP assured Dolan that test security is an extremely important issue, and that all make-up requests are thoroughly reviewed by the Department. It acknowledged that although it takes adequate measures to ensure test security, [u]nfortunately, there appear to be individuals in the public safety ranks who, to their own detriment, provide test content to those taking the make-up examinations. The DOP also stated that it could not take any action unless it had specific information that a candidate had obtained test materials or had access to test questions prior to the examination.
In June 1998, petitioners requested that the DOP postpone Officer Durkin s make-up exam. Petitioners provided the DOP with copies of a substantial portion of the questions and answers from the October 1997 sergeant s exam as proof that exam security had been breached. The DOP denied petitioners request, indicating that the record did not establish conclusively that Durkin or any other make-up candidate had advance knowledge of the exam s content. Durkin took a make-up exam in June 1998 and placed eighth on the eligible list.
In total, eight officers in the Paterson Police Department were promoted to sergeant: five out of the 135 candidates who passed the original exam and three out of the three candidates who took a make-up exam. See footnote 1 Petitioners Troy Oswald, Steven Olivo and John Robb, III were among the top eight candidates promoted to sergeant in the Paterson Police Department.
In July 1998, petitioners filed a petition with the DOP and the Board. They asserted that the DOP s practice of administering identical examinations to original and make-up candidates violated their right to a fair and competitive civil service examination under article VII, section 1, paragraph 2 of the New Jersey Constitution. Petitioners also argued for the removal of Officers Catania and Durkin from the eligible list, alleging that they had advance knowledge of the exam s content prior to taking a make-up exam. In support of those assertions, petitioners provided the Board with copies of questions and answers to the October 1997 sergeant s exam, which they found at roll call in the Paterson Police Department shortly after the original exam and before any make-up exams.
The Board rejected petitioners constitutional challenge to the DOP s practice of administering identical exams to original and make-up candidates. It observed that:
[P]rior to 1988 it was standard practice to utilize different tests for police and fire make-up examinations. . . . However, in July 1988, the Merit System Board issued a decision, In the Matter of Ronald W. Sutton, Police Sergeant (PM1211H), Atlantic City, which resulted in the discontinuance of use of different tests. It was determined that when two groups are not formed by random selection methods, no equating is possible unless there is an anchor test available to correct for differences between the groups. Since the Department of Personnel develops and administers an essentially new examination for each testing cycle, no such anchor test is available. Under these circumstances, and in the absence of statistical evidence to the contrary, petitioners have not established a basis to use a different examination for make-up candidates.
 
The Board referred petitioners specific allegations against Officers Catania and Durkin to the Office of Administrative Law for a hearing to determine whether they had knowledge of the exam s content prior to their make-up exams.
In July 2000, an administrative law judge granted Catania s and Durkin s motion for summary judgment pursuant to N.J.A.C. 1:1-12.5(b). The ALJ concluded that there was no residuum of competent evidence that Catania or Durkin was present after the original examination when the examinees were discussing the test content or present at the police station when copies of the examination were circulating. It also concluded that there was no evidence that Catania or Durkin participated in study groups in which the instructors had knowledge of the test content or attempted to discover the content of the exam prior to taking the make-up exam. It held that without this linking evidence, [Catania s and Durkin s] advance knowledge of test content is no more than speculation, suspicion and surmise. The ALJ also held that even if knowledge of the test content was imputed to Catania or Durkin, on the strength of the record as it exists that inference does not arise to preponderating evidence, which would be the burden of proof the challengers would have to shoulder at a plenary hearing. Petitioners appealed the ALJ s decision, and the Board affirmed in October 2000.
In an unreported per curiam opinion, the Appellate Division affirmed the Board s decision. The panel first rejected petitioners general challenge to the DOP s practice of using identical questions on original and make-up exams. It then affirmed the Board s finding that there was no competent proof that the examination as held . . . was unfair. We granted petitioners petition for certification. 174 N.J. 41 (2002).
II.
 
 
We first address petitioners broad constitutional challenge to the DOP s practice of administering identical examinations to original and make-up candidates.
A
 

The New Jersey Constitution provides that [a]ppointments and promotions in the civil service of the State . . . shall be made according to merit and fitness to be ascertained, as far as practicable, by examination, which, as far as practicable, shall be competitive . . . . N.J. Const. art. VII, 1, 2. Consonant with paragraph 2, the Legislature, under the New Jersey Civil Service Act, N.J.S.A. 11A:1-1 to 12-6 (the Civil Service Act), has declared that the selection and advancement of state employees should be dependent on considerations of merit, N.J.S.A. 11A:1-2c, and determined on the basis of relative knowledge, skill, and ability, N.J.S.A. 11A:1-2a. To implement those constitutional and statutory requirements, the Legislature vests the DOP, the Board, and the DOP Commissioner with broad power to devise a fair, secure, merit-based testing process by which candidates are selected for employment and promotion. Brady v. Department of Personnel, 149 N.J. 244, 254 (1997); see N.J.S.A. 11A:4-1. The Commissioner must provide for the announcement and administration of examinations which shall test fairly the knowledge, skills and abilities required to satisfactorily perform the duties of a title or group of titles, N.J.S.A. 11A:4-1a, the rating of examinations, N.J.S.A. 11A:4-1b, and, importantly, the security of the examination process and appropriate sanctions for breach of security[.] N.J.S.A. 11A:4-1c.
Recently, in Brady, supra, we sanctioned the DOP s practice of reusing a substantial number of exam questions from test to test, in identical or similar form. 149 N.J. at 255. In that case, the plaintiff, a police sergeant who took a civil service promotional test, challenged the results of his examination. 149 N.J. at 255, 257. In response to the plaintiff s request to review test materials, the DOP provided the plaintiff only limited access to actual test questions and answers pursuant to N.J.A.C. 4A:4-6.4, which affords the DOP substantial discretion to determine the appropriate level of access to exam materials. Brady, supra, 149 N.J. at 255. The DOP asserted that because it reused a significant number of questions for different test administrations, limited access to exam content was necessary to preserve the integrity of the civil service process and the security and confidentiality of civil service examinations. Id. at 249, 255.
The plaintiff argued that the DOP abused its discretion by reusing exam questions from test to test, either in identical or similar form for different test administrations. Id. at 259. We disagreed, observing that:
[The DOP s] reasons for this practice are the expense involved in formulating entirely new questions for each test, the consistency that recycling questions achieves among exams given at different times, and the ability to evaluate the effectiveness of questions that results from providing the questions to a large pool of examinees. Given the massive number of civil-service examinees and the hundreds of tests administered each year, the DOP has concluded that its reuse of test questions would render it impossible to ensure the integrity of the testing process if candidates were to have full access to exam materials. Moreover, according to the DOP, full access would lead to significantly increased costs in propounding test questions and would impair the DOP s ability to contract with private testing firms to provide this service.
 
[Id. at 255.]
 
 
In rejecting the plaintiff s challenge, we acknowledged the judiciary s limited role in reviewing the DOP s determinations regarding civil-service testing processes. Id. at 259. We held that the DOP s decision to reuse exam questions was clearly within the range of responsibilities that the Legislature has delegated to the DOP to implement the most effective and efficient procedure to assure public hiring and promotion based on merit. Ibid. (citing Brotspies v. Department of Civil Serv., 66 N.J. Super. 492, 495-96 (App. Div. 1961)).
B
 
 
Petitioners do not challenge the constitutionality of make-up exams in general or the DOP s practice of reusing a substantial number of exam questions from test to test, in identical or similar form. Instead, they question the DOP s practice of administering identical exams to original and make-up candidates in the same testing cycle. They argue that that practice is per se unconstitutional because inadequate procedural safeguards are in place to address the substantial likelihood that make-up candidates will obtain advance knowledge of the test s content.
Although this appeal presents an issue discrete from that addressed in Brady, that opinion provides guidance for our determination. In Brady, supra, we emphasized that the DOP has broad authority to implement and administer the most effective and efficient procedure to assure public hiring and promotion based on merit. 149 N.J. at 259. Moreover, we noted that the judiciary is not disposed to engage in a critical supervisory examination of the composition of the questions to be propounded to candidates in promotional tests . . . or to exercise an appellate review of the scoring of the answers . . . . Id. at 260-61 (quoting Artaserse v. Department of Civil Serv., 37 N.J. Super. 98, 105 (App. Div. 1955)). Rather,
[t]he preparation and administration of civil service examinations is an administrative function delegated most liberally to the authorized examiners of the Department . . . by the Legislature. The fulfillment of that function is a matter requiring special expertise, involving as it does the determination of what job knowledge, skills and abilities are necessary or desirable in a candidate for a particular position, and the highly technical problem of devising suitable examination questions which will demonstrate as accurately as possible whether an applicant possesses those requirements sufficiently to qualify for the position. In view of the above, the courts cannot intervene to nullify a civil service examination unless it is clearly shown that the Department has abused its discretion.
 
[Id. at 257 (quoting Zicherman v. Department of Civil Serv., 40 N.J. 347, 350-51 (1963) (quotations and citations omitted)).]

In essence, the Board maintains that only by administering identical exams to all candidates during a single testing cycle can it provide candidates with a fair and competitive examination. It argues that that methodology is the only practicable, statistically valid means to compare the performance of original and make-candidates fairly and accurately. See N.J.A.C. 4A:4-2.9(f)( Make-up examinations will be administered to the extent possible under the same conditions as the original examination. )(emphasis added).
Consistent with N.J.A.C. 4A:4-2.9(f), the Board argues that because the DOP develops and administers an essentially new examination for each testing cycle, no anchor test is available to compare make-up results with results from the original test. The Appellate Division below observed that [t]his determination of the Board, based on an interpretation of its own enabling legislation and implementing regulations, and grounded in its administrative expertise, is entitled to great deference. As noted, this Court has a limited role in reviewing the DOP s administration of the civil-service system and its determination regarding civil-service testing processes. Brady, supra, 149 N.J. at 256. Thus, for the present, and subject to our additional comments below, we decline to bar outright the DOP s practice of administering identical exams to original and make-up candidates in the same testing cycle.

III.
 
 
Although we decline to hold that the DOP s practice of providing identical exams to original and make-up candidates in the same testing cycle is per se unconstitutional, we must determine whether the make-up exams as administered in Paterson violated petitioners right to a fair and competitive examination under our State Constitution.
A
 
As noted, the DOP has the authority to devise a fair, secure, merit-based testing process by which candidates are selected for employment and promotion. Brady, supra, 149 N.J. at 254. To effectuate that goal, the Commissioner must provide for [t]he security of the examination process and appropriate sanctions for breach of security. N.J.S.A. 11A:4-1c. In accordance with N.J.S.A. 11A:4-1c, the Board has promulgated several regulations to ensure that all applicants receive equal opportunity to demonstrate their relative merit and fitness. N.J.A.C. 4A:4-2.10(a). The Board specifically prohibits the following:
Securing, by unauthorized persons, of questions or materials, unless the same are available to all applicants;

. . . .

Use or attempted use of any unauthorized aids, information or assistance, including copying or attempting to copy from, or helping or attempting to help another applicant in any part of an examination or performance of work assigned;

. . . .

Copying, recording or transcribing any examination question or answer, and/or the removal from any examination room of any question sheet, answer sheet, scrap paper, notes or other papers or materials related to the content of an examination.
 

[N.J.A.C. 4A:4-2.10(b)1, 5, 7.]
 
 
Any candidate caught engaging in such impermissible activities shall be disqualified from the examination and may be rejected from future examinations and subject to punishment as provided by law. N.J.A.C. 4A:4-2.10(c). Moreover, as a precondition to taking a make-up exam, all make-up candidates must sign a statement that they have no knowledge of the content of the examination as a result of information gained from or furnished by other candidates who participated in the original examination. N.J.A.C. 4A:4-2.9(h).
Petitioners argue that the Board and the DOP did not take adequate measures to prevent the make-up candidates from securing advance knowledge of test materials. They rely on Rox v. Department of Civil Service, in which the Appellate Division, in dicta, stated that it was important that the State Civil Service Commission (replaced by the Merit System Board) adopt adequate and meaningful precautionary measures to assure exam security. 141 N.J. Super. 462, 469-70 (1963). The Rox court invalidated an oral examination after it determined that the subjective nature of the exam rendered it noncompetitive and illegal. Id. at 467.
In Rox, the plaintiffs raised the issue of exam security, asserting that the Commission had failed to adopt adequate and meaningful precautionary measures to ensure the security of the exam. Id. at 469. The Commission administered an oral examination in Newark that had been given in identical form in neighboring municipalities on prior dates. Ibid. In the court s view, that circumstance creat[ed] a strong likelihood that some of the Newark candidates, through communication with candidates in other testing municipalities, had knowledge of the questions. Ibid. The Commission argued that security was maintained by denying the candidates information in respect of exam results until the conclusion of all examinations. Ibid. The court found those measures inadequate, stating that [a]lthough the advantage would be greater if the correct answers were also known, a distinct advantage inures to the individual having knowledge of the questions before being orally examined. Id. at 470. Moreover, it emphasized that although appellants have not documented any specific instances in which test questions were actually obtained, the regulation requires measures to be taken to prevent that possibility. Ibid. The court concluded that without adequate security measures in place, the integrity of the oral examination becomes suspect. Ibid.
Other jurisdictions also have recognized the importance of a fair, secure, and competitive civil service examination process. In Mangan v. New York State Civil Serv. Comm n, the trial court upheld a Civil Service Commission s decision to cancel the results of a promotional exam after an investigation revealed that a number of candidates potentially had access to previously used questions that appeared on the subject exam. 303 N.Y.S.2d 113, 114 (N.Y. Spec. Term 1969). A candidate informed the Commission that six sheets of paper found in a station house squad room were rumored to contain copies of answers to fifteen questions that appeared on a previous exam. Id. at 114. Based in part on that information, the Commission cancelled the results of the promotional exam because the exam was not truly competitive as mandated under the New York Constitution. See footnote 2 Ibid. See also Katz v. Hoberman, 28 N.Y.2d 970 (N.Y.)(affirming Civil Service Commission s decision to cancel promotional exam after it inadvertently used exam questions previously made public), cert. denied, 404 U.S. 881, 92 S. Ct. 203, 30 L. Ed. 2d 163 (1971); City of Beaumont v. Spivey, 1 S.W.3d 385 (Tex. App. 1999)(affirming trial court s finding that candidate s cheating on civil service exam violated fair and competitive exam contrary to Texas Municipal Civil Service statute).
B
 
 
Petitioners concede that no evidence indicates that Officers Catania, Durkin, or Mejia had access to exam materials prior to sitting for their make-up exams. Our opinion is not intended to suggest otherwise. Instead, we address the fairness and competitiveness of the examination as administered.
The Board and the DOP maintain that in light of the extensive security measures employed in the administration of civil service examinations and the lack of any specific evidence that the make-up candidates had advance access to exam materials, the make-up exams as administered were not unfair or noncompetitive within the meaning of the State Constitution. We disagree. As noted, the Board and the DOP specifically prohibit [c]opying, recording or transcribing any examination question or answer, and/or the removal from any examination room of any question sheet, answer sheet, scrap paper, notes or other papers or materials related to the content of an examination. N.J.A.C. 4A:4-2.10(b)7. It is undisputed that unknown individuals typed out sixty-five of seventy-one questions that appeared on the October 1997 sergeant s exam and disseminated them with answers throughout the Paterson Police Department prior to the administration of any make-up exams. Moreover, petitioners provided the Board and the DOP with copies of exam materials that they secured prior to the administration of any make-up exams. Once the DOP or the Board discovered that those breaches in exam security had occurred contrary to N.J.A.C. 4A:4-2.10(b)7, the make-up exams should have been cancelled and an appropriate remedy fashioned to ensure all candidates a fair and competitive exam.
Allowing candidates to take a make-up exam, despite direct evidence that virtually all of the questions from the original exam had been disseminated throughout the police department, undermined the integrity of the examination process and impaired candidates ability to demonstrate their relative merit and fitness. The public, and those police officers that did not contribute to the breach in exam security, must retain confidence in the integrity of the civil service examination process. Sanctioning the make-up exams as administered would undermine that confidence and the principles of fairness and competitiveness embodied in our State Constitution. Accordingly, we hold that the make-up exams as administered in Paterson violated article VII, section 1, paragraph 2 of the State Constitution.
 
IV.
 
 
Having determined that the make-up exams as administered were fundamentally flawed, we now provide direction for the administration of future make-up examinations and address the difficult issue of remedy.
A
 
 
We recognize that over the years the Board and the DOP have shown concern for the security of civil service examinations. Nonetheless, as the Appellate Division below noted, the Board s self-policing measures may . . . be insufficient to preclude some candidates from gaining access to test content in advance of taking the make-up examination. The panel stated: We are not satisfied that the measures cited by the Board will be sufficient on future occasions to ensure against compromise of test confidentiality and therefore test integrity. If the practice of using identical questions on make-up examination[s] is to be continued, we suggest consideration be given to stronger deterrent measures capable of being enforced. However, although security can and must be improved, it may be that no procedural safeguards can prevent the type of irregularities that occurred in this appeal. Thus, regardless of the deterrent measures imposed, we have serious reservations in respect of the DOP s practice of providing identical exams to original and make-up candidates in the same testing cycle. Indeed, a recently proposed legislative bill that would bar the DOP from administering make-up exams containing the same questions as the original exam echoes our concerns regarding the deficiencies of that practice. N.J. Assem. No. 3374, 210th Leg. (Feb. 27, 2003).
The Board argues that administering identical exams to make-up and original candidates is the only practicable, statistically valid method to ensure a fair and competitive examination. A review of the record suggests, however, that the Board may be able to use an anchor test, which includes a set of questions in addition to, or included among, those administered to both candidate populations as a means of adjusting for differences that may be found to exist between them. Even if that specific methodology is not feasible, we are confident that based on the Board s and the DOP s experience and expertise they can develop a methodology that ensures a fair, merit-based assessment of the examinees. Going forward, the Board and the DOP should administer make-up exams that contain substantially different or entirely different questions from those used in the original examination.
That said, we emphasize that the DOP s and the Board s increased diligence regarding their revised exam procedures will be for naught unless the participants honor their legal and ethical obligations. A breach in exam security corrupts the process and compromises the goal of a fair and competitive exam. It therefore is imperative that all examinees honor the letter and spirit of their certifications in which they attest that they have not obtained or disseminated exam materials.
B
 
In respect of five of the top nine candidates on the eligible list, Officers Angelo Palatucci, Troy Oswald, Philip Pagliaro, Steven Olivo, and John Robb, III, all of whom were appointed to the position of sergeant in the Paterson Police Department, nothing in the record warrants their removal from that position. Those candidates took the exam prior to the dissemination of any information regarding its contents. Because nine positions were available and only three make-up candidates sat for the exam, those five original exam candidates have qualified as sergeants irrespective of any later irregularities in the exam process. Moreover, there is no evidence that those five candidates facilitated or participated in the improper actions that compromised the integrity of the make-up exams. Accordingly, those five candidates should retain their positions as sergeants.
Because we are voiding the results of the make-up examinations and thereby prohibiting any permanent appointments to police sergeant from those candidates who sat for the make-up exams, three police sergeant positions in the career service now are technically open. Those are the positions currently held by Officers Catania, Durkin and Mejia. The DOP indicates that civil service promotional examinations are administered on an annual basis, and that it expects to administer the next sergeant s examination in the fall of 2003. At that time, all currently eligible candidates from the Paterson Police Department, including those who sat for the October 1997 sergeant s exam as well as the three make-up exam candidates, may compete for the open Paterson sergeant positions.
In the interim until a new exam is given and its results certified, we leave unaddressed the question of who, if anyone, should fill the open sergeant positions and the procedures for those appointments to the discretion of the DOP. See N.J.S.A. 11A:4-13; see also N.J.A.C. 4A:4-1.5 (permitting provisional appointments upon certain conditions and upon certification of need for position by appointing authority). The State Constitution and the Civil Service Act charge the DOP, specifically the Board and the DOP Commissioner, with primary jurisdiction in these matters. We rely on their discretion to implement our judgment in a manner that will ensure fairness to all participants in the process.
Our remedy is confined to the October 1997 sergeant s exam administered to the members of the Paterson Police Department.
 
V.
 
 
The judgment of the Appellate Division is affirmed in respect of its holding that identical exams for original and make-up candidates is not per se unconstitutional, reversed in part in respect of its holding that the make-up exams as administered were fair and competitive, and remanded for appropriate relief consistent with our decision.
CHIEF JUSTICE PORITZ and JUSTICES COLEMAN, LONG, VERNIERO, LaVECCHIA, and ALBIN join in JUSTICE ZAZZALI s opinion.

SUPREME COURT OF NEW JERSEY
 
NO. A-138 SEPTEMBER TERM 2001
ON CERTIFICATION TO Appellate Division, Superior Court

IN THE MATTER OF
POLICE SERGEANT (PM3776V)
CITY OF PATERSON.

DECIDED April 16, 2003
Chief Justice Poritz PRESIDING
OPINION BY Justice Zazzali
CONCURRING OPINION BY
DISSENTING OPINION BY

CHECKLIST AFFIRM IN PART/REVERSE IN PART/REMAND
CHIEF JUSTICE PORITZ X
JUSTICE COLEMAN X
JUSTICE LONG X
JUSTICE VERNIERO X
JUSTICE LaVECCHIA X
JUSTICE ZAZZALI X
JUSTICE ALBIN X
TOTALS 7

 
Footnote: 1 As noted, the Paterson Police Department employed the exam at issue to fill nine sergeant positions. Officer James Bishop took the original exam and placed fifth on the eligible list. Although Bishop placed in the top nine on the eligible list, he was not selected for the position of sergeant.
Footnote: 2 Similar to our State Constitution, the New York Constitution provides that [a]ppointments and promotions in the civil service of the state . . . shall be made according to merit and fitness to be ascertained, as far as practicable, by examination which, as far as practicable, shall be competitive . . . . N.Y. Const. art. V, 6.