MAN TECH INFORMATION SYSTEMS & TECHNOLOGY CORP. v. USA, No. 1:2009cv00804 - Document 46 (Fed. Cl. 2010)
Court Description: PUBLISHED OPINION. Signed by Senior Judge Eric G. Bruggink. (jpk1)
Download PDF
<![CDATA[
MAN TECH INFORMATION SYSTEMS & TECHNOLOGY CORP. v. USA Doc. 46 In the United States Court of Federal Claims No. 09-804C (Consolidated with No. 09-805C) (Filed: April 29, 2010)1 ************************ MANTECH, INC., Plaintiff, v. THE UNITED STATES, Pre-award Bid Protest; Competitive Range Determination; Indefinite D elivery/Indefinite Quantity Contract; Price Realism. Defendant. ************************ L-3 SERVICES, INC., Plaintiff, v. THE UNITED STATES, Defendant. ************************ Paul F. Khoury, Washington, DC, for plaintiff ManTech, Inc., and Craig A. Holman, Washington, DC, for plaintiff L-3 Services, Inc. Christopher L. Krafchek, William J. Grimaldi, United States Department of Justice, Civil Division, Commercial Litigation Branch, Washington, DC, with whom were Tony West, Assistant Attorney General, 1 This Opinion was filed under seal on March 29, 2010. On that date, we requested that the parties provide redactions, which they submitted jointly on April 16, 2010. Redactions are indicated by [...]. Dockets.Justia.com Jeanne E. Davidson, Director, and Kirk Manhardt, Assistant Director, for defendant. Max D. Houtz, Assistant General Counsel, Defense Intelligence Agency, Litigation Branch, Washington, DC, of counsel. OPINION BRUGGINK, Judge. This action is brought pursuant to the court’s bid protest jurisdiction. Plaintiffs ManTech, Incorporated and L-3 Services, Incorporated (“ManTech” and “L-3”), unsuccessful offerors in Solicitation HHM402-09-R-0050 (“RFP” or “Solicitation”), allege that the United States, acting through the Defense Intelligence Agency (“DIA” or “agency”) has acted arbitrarily, capriciously, and in violation of law in excluding them from its competitive range of October 7, 2009.2 Currently pending are cross-motions for judgment on the administrative record regarding plaintiffs’ request for permanent injunctive relief. Oral argument was held March 12, 2010. For the reasons set out below, we deny plaintiffs’ motions for judgment on the administrative record and grant defendant’s motion. BACKGROUND On May 26, 2009, the agency issued the “Solutions for the Information Technology Enterprises” (“SITE”) Solicitation setting forth acquisition parameters for delivering information technology (“IT”) services and capabilities to the intelligence community.3 The agency will award about four large businesses and four small businesses an indefinite delivery/indefinite quantity (“IDIQ”) contract. The proposals for the large and small businesses are reviewed separately. The eight winning bidders will be able to compete on a task order basis over the course of a base year and four one-year options. The ceiling amount for the SITE program is $6,600,000,000; the floor is 2 A related protest was filed concerning the competitive range determination by a small business offeror, Hyperion, Inc. The opinion in Hyperion, Inc. v. United States, No. 09-758, will be issued contemporaneously with this opinion. 3 SITE seeks to support the DIA, the Uniformed Services, the Combatant Commands (“COCOMs”), and other intelligence agencies. 2 $50,000. The Solicitation provides for a competitive range determination (“CRD”) in order to limit the number of potential bidders with which the Contracting Officer (“CO”) has to negotiate. The Solicitation sets forth the criteria that DIA must use in making the award. The Solicitation states that proposals “should be complete and accurate to enable the Government to assess fully [the offerors’] ability to provide the required services and whether [the offerors] represent the best value to the government.” Administrative Record (“AR”) 532. Each proposal consists of six volumes, the first a summary and the other five each addressing a substantive factor: the Past Performance Plan; Security; the Small Business Subcontracting Plan, the Technical/Management Factor, and the Cost/Price. The non-price factors are significantly more important than price; price becomes a major factor only when other factors are substantially equal. Among non-price factors, the descending order of importance is (1) Technical/Management; (2) Past Performance Plan; (3) Small Business Plan; and (4) Security.4 AR 513-14. Within the Technical/Management Factor offerors must show competence with respect to five elements arranged in descending order of importance: (1) Technical Experience to functional areas listed in the S ta te m e n t o f O bjectives (“ S O O ” ); (2 ) C o n tra c to r’s S IT E Organization/Management Team and Proposed Management Structure; (3) Contractor's Recruiting, Training, and Employee Retention Plan; (4) Transition Experience; and (5) Logistics Administration and Worldwide Support Capability. AR 514. The Technical Experience element requires the offeror to show “the depth and breadth of its experience and expertise performing as a subcontractor with respect to the functional areas . . . .” 5 The Solicitation 4 The Security factor was pass/fail, but no offerors failed. It is therefore not germane. 5 AR 514. These sixteen functional areas are: (1) Program & Project Management Services; (2) Technology Assessment and Evaluation Services; (3) Systems Engineering; (4) Operations Support Services; (5) Network Operations and Administration; (6) Storage Services; (7) Web Services and Content Management; (8) Acquisition and Property Management Services; (9) Maintenance and Remote Diagnostic Services; (10) Administrative and Special Services; (11) Information Assurance Services; (12) Security Management; (13) SCI Personnel and Information Security Support; (14) Risk 3 requires offerors to illustrate their experience and expertise by mapping the ten contracts used in the Past Performance volume into a matrix in the Technical/Management volume. The Past Performance Factor is based on a narrative of ten contracts or subcontracts of a minimum size.6 Its purpose, in looking at projects similar in scope, complexity, and size, is to “measure the degree to which an Offeror has kept its previous contractual promises and thus satisfied its customers . . . .” AR 518. The Small Business Plan Factor requires offerors to submit an adequate subcontracting plan that explains how the offeror will undertake good faith efforts to use specific, preferred types of small businesses and to provide past performance information demonstrating their experience in utilizing those types of small businesses as subcontractors.7 The standard provided by the Management; (15) Testing and Verification Services; and (16) Training Services. AR 1339-41. 6 The baselines for the contracts submitted in the Past Performance volume are: The offeror’s demonstrated performance was on related efforts completed within the past three years or is on-going. The combined past performance efforts cited were at least valued at $10,000,000 including all options. Customer assessments of previous contracting efforts indicate the scheduling standards were achieved without affecting cost or performance. Quality of services provided were professional and at the level expected by the customer. Past Performance Information demonstrates the offeror’s team can effectively manage large contracts similar in scope, complexity and size. Past Performance information demonstrates previous teaming arrangements and positive business relationships. AR 519. 7 Specifically, the offeror must “explain how the proposed goals are realistic and that the offeror will expend good faith efforts to use small business, veteran owned small business, service-disabled veteran-owned small 4 Solicitation measures the offerors by the organizational process they use for selecting the various types of small businesses, by the offeror’s own broad historical data on usage of the preferred types of small businesses, and by the offeror's own specific past performances “in awarding subcontracts for the same or similar products or services to small business[es].” AR 521. The Cost/Price factor is an evaluation of price reasonableness and price realism based upon the offeror’s proposed ceiling labor rates for a number of positions in several world-wide locations, as well as composite mark-up rates. The agency evaluates the reasonableness and realism of proposed labor rates by multiplying them by a predetermined and undisclosed (to offerors) quantity of hours. Price only “becomes a major factor in award selection when other criteria are substantially equal.” AR 522. The Solicitation calls for a hierarchical, panel-based system to review the proposals. There are five evaluation panels (“panels”)—one for each of the factors—composed of experts in the relevant field. Each panel looks at its corresponding volume only and ignores any cross-references to separate volumes. Each panel may, however, read other volumes if necessary, but it does not have access to the Cost/Price volumes. Rather, a separate evaluation team performs Cost/Price evaluation. After a panel completes its review, it then documents the evaluation, first in a Consensus Report for each offeror, and then in a Panel Report for all offerors under that factor. The Source Selection Advisory Council (“SSAC”), composed of senior representatives of the intelligence community, then prepares a report for the Source Selection Administrator (“SSA”), summarizing the findings of the panels. The SSAC also performs the “Best Value Paired Trade-Off Analysis,” bringing into play the Cost/Price factor. The SSA makes both the CRD and the final decision to award, documenting each. The Contracting Officer is the government official that deals directly with the vendor community. She also conducts the price analysis process. business, HUBZone small business, small disadvantaged business, and women-owned small business subcontractors to the maximum practicable extent . . . [and] provide past performance information demonstrating their experience in [the above].” AR 520. 5 The panels utilize notational ratings in their reports. The four Technical/Management Factor ratings are: “Exceptional (Blue);” “Acceptable (Green);” “Marginal (Yellow);” and “Unacceptable (Red).” AR 524. Each color expresses a certain level of “Technical Capability,” “Strengths,” and “Weaknesses.” Id. For example, the two ratings most applicable to plaintiffs’ protests are more particularly described as follows: Color Technical Capability Strengths Acceptable Green The proposal is satisfactory; the offeror is capable of meeting performance requirements. The proposal can potentially cause some disruption of schedule, and appreciable increase in cost, however, special Contractor emphasis and close government monitoring will be able to overcome difficulties. Some strengths exist that are of benefit to the Government; strengths clearly offset weaknesses. A few weaknesses exist; they are correctable by the Contractor; minimal additional Government contract administration anticipated. The proposal is marginal; Government doubts the offeror will be able to meet all performance requirements. Based on the Offeror’s performance record, doubt exists that the Offeror can satisfactorily perform the proposed effort. Some strengths exist with limited benefit to the government. A few weaknesses exist; they are correctable by the Contractor with moderate additional Government contract administration. Marginal Yellow Weaknesses Id. The Past Performance notational rating descriptions are more general. The five Past Performance ratings (i.e. colors and adjectives) are identical to the Technical/Managment ratings save for the fifth: “Neutral (White).” AR 525. Each rating corresponds only to a “Description.” For example, the pertinent ratings for plaintiffs are: Rating Description 6 Exceptional Blue Excellent past performance ratings in most functional areas. Very Good, ontime performance is consistently demonstrated. No doubt exists, based on the Offeror's performance record that the offeror can satisfactorily perform the proposed effort. Acceptable past performance ratings in most functional areas. The proposal could have an inconsistent achievement of acceptable performance; however, minimum doubt exists, based on the Offeror’s performance record, that the offeror can successfully perform the proposed effort. Acceptable Green Id. The Small Business Plan notational ratings are also germane to this protest. They too follow the color scheme of the previous ratings—from Exceptional (blue) to Unacceptable (red). The two ratings germane to this case are: Rating Description Exceptional Blue Plan contains a robust Small Business Subcontracting Plan, which includes all business types. PPI [past performance information] was relevant and demonstrated significant experience in awarding SB [small business] contracts. Acceptable Green Plan contains an adequate Small Business Subcontracting Plan, which includes a variety of small business types. PPI was relevant and demonstrated an acceptable level of experience in award of small business subcontracts. AR 526. In response to the Solicitation, twenty-nine businesses offered proposals. Eleven of those were large businesses. On October 7, 2009, the 7 Contracting Officer, acting pursuant to the SSA's determination and based on the rankings compiled by SSAC, made a CRD of six small and six large businesses. Neither ManTech nor L-3 was among the six large businesses. ManTech received a debriefing from the DIA on October 23, 2009, in which it learned that its proposal garnered a [...................] for the Technical/Management factor, an [....................] for its Past Performance factor, and an [....................] for the Small Business Plan factor. ManTech’s overall proposal ranked it [.........] out of the eleven large business offerors. L-3 received its debriefing on October 23, 2009, in which it learned that its proposal garnered a [...................] for Technical/Management, an [....................] for Past Performance, and an [....................] for its Small Business Plan. L-3’s overall proposal ranked it [....] out of the eleven large businesses. ManTech and L-3 filed these protests on November 20, 2009. DISCUSSION I. Jurisdiction and Standing The Tucker Act, as amended by the Administrative Dispute Resolution Act (“ADRA”), grants this court jurisdiction “to render judgment on an action by an interested party objecting to a solicitation by a Federal agency for bids or proposals for a proposed contract . . . .” 28 U.S.C. § 1491(b)(1) (2009). Standing to invoke this jurisdiction is afforded to actual or prospective bidders or offerors whose direct economic interests would be “affected by the award of the contract or by the failure to award the contract.” Am. Fed’n of Gov’t Employees v. United States, 258 F.3d 1294, 1302 (Fed. Cir. 2001). The government does not challenge plaintiffs’ standing as “interested part[ies].” See Weeks Marine, Inc., v. United States, 575 F.3d 1352, 1358 (Fed. Cir. 2009). The scope of the Solicitation is vast: about eight businesses, depending on the eventual number of awards, will have access to a significant amount of the government’s purchases of IT security for up to the next five years. ManTech and L-3 both specialize in providing technology and security to the U.S. national security community; both have submitted proposals. Plaintiffs thus satisfy the standing requirement. 8 II. Plaintiffs’ Claims The Administrative Procedure Act (“APA”) limits our review of the merits, allowing us to set aside an agency’s decision only if it is “arbitrary, capricious, an abuse of discretion, or otherwise not in accordance with law.” 5 U.S.C. § 706(2)(A) (2006). The arbitrary or capricious standard means that we will sustain an agency decision that has a rational basis. See, e.g., Motor Vehicle Mfrs. Ass’n of the U.S., Inc. v. State Farm Mut. Auto Ins. Co., 463 U.S. 29, 43 (1983); Advanced Data Concepts, Inc. v. United States, 216 F.3d 1054, 1058 (Fed. Cir. 2000) (“The arbitrary and capricious standard applicable here is highly deferential.”). Specifically, we may not substitute our judgment for that of the DIA. Id. Challenges that allege the decision was not in accordance with applicable law “must show a clear and prejudicial violation of applicable statutes or regulations.” Impresa Construzioni Geom. Domenico Garufi v. United States, 238 F.3d 1324, 1332 (Fed. Cir. 2001). Moreover, the greater the discretion granted to a contracting officer, “the more difficult it will be to prove the decision was arbitrary and capricious.” Galen Med. Assoc. v. United States, 369 F.3d 1324, 1330 (Fed. Cir. 2004). In “best value” procurements such as this, we give greater deference to the DIA than we would “if the contract were to have been awarded on the basis of cost alone.” Id. at 1330. Plaintiffs raise the following issues for disposition: whether the DIA’s ratings on the Technical/Management and Small Business factors were arbitrary with respect to either bidder; whether the DIA’s Past Performance rating for L-3 was arbitrary; whether DIA price realism analysis was arbitrary; and whether the DIA’s overall CRD was arbitrary. A. ManTech 1. Technical/Management Factor a. Risk Assessment ManTech’s primary argument is that DIA improperly performed the required risk assessment. It argues that if the [.............] risk assessment assigned by the Past Performance Panel had been included in the consideration of the Technical Management Panel, which it contends the 9 Solicitation requires, then it is possible or even likely that ManTech would not have received a [..................] rating on its Technical/Management factor. This sharing of information was required, it contends, because the “Risk Assessment” instructions to the Past Performance Panel include the following: The government will perform a risk assessment of each Offeror’s proposal . . . . Proposal risk assessment focuses on the risks and weaknesses associated with the Offeror’s experience and will consider each Offeror’s likelihood of success in performing the requirements stated in the RFP. The risks which will be assessed, are those associated with technical aspects of the program. Risks may occur as a result of a particular technical methodology, operational process, or economic impacts associated with these approaches. . . . For any risk identified, the evaluation will address the Offeror’s proposal for mitigating those risks and why that approach is or is not manageable. All risk assessments will be included as part of the adjectival/Color Rating in the Technical/Management and Past Performance Factors. AR 520 (emphasis added). ManTech emphasizes the italicized language. It contends that the risk assessments referred to are those of the Past Performance Evaluation Panel, thus clearly obligating the DIA to ensure that the [.............] results ManTech achieved were known to the Technical/Management Evaluation Panel prior to its assignment of adjectival ratings. The evidence for potential prejudice, ManTech contends, is that the Past Performance ranking provided that [...........................................................................................................] AR 11840. The Technical Management ranking, on the other hand, provided that, [......................................................................................................................] AR 11966. Defendant argues that the panels were to operate completely independently of each other, and that the procedural step plaintiff argues for is a misreading of the Solicitation. The Solicitation incorporates the agency’s answers to offerors’ questions; the answers, defendant argues, made it clear 10 that the bidders’ submissions to each panel are strictly segregated.8 ManTech therefore was on notice of an apparent inconsistency in the Solicitation which should have been flagged by plaintiff before it bid. We disagree with defendant’s argument that ManTech waived its right to assert its construction of the risk assessment language. The fact that the panels evaluate separate submissions does not mean that the evaluators cannot share their own observations. Nor do we find it necessary to closely parse the Solicitation to determine if defendant is correct that each panel was to do a completely independent risk assessment. There is plainly overlap in the information the bidders were called on to provide to the Past Performance Panel and to the Technical/Management Panel. Both panels had before them the same [...] prior contracts for evaluation, and both evaluations had to be performed in the context of the SOO. But the information was not identical, and more importantly, we find that the inquiries the two panels conducted were completely different in substance and purpose. In both instances, ManTech provided a chart of [...] contracts mapped to SOO areas, but it added [...] contracts to the chart for the Technical/Management Panel’s review. ManTech’s [........] submission to the Technical Evaluation Panel included the substance of the information in the [.......] submission to the Past Performance Evaluation Panel, but went into substantially greater detail linking ManTech’s contract experience to the individual components of the SOO. Past performance evaluators were given the following instructions: [.................................................................................................. .................................................................................................... .................................................................................................... 8 “[P]ast performance will be evaluated separately and at a different time by a different team [panel].” AR 1546. Also, “[o]fferors should not assume that separate teams [panels] will cross reference or even be given access to volumes from another proposal area.” AR 1514. And, “Volume V should stand alone.” Id. 11 .................................................................................................... ..................................................] AR 518. The evaluators were to make the following assessments, after contacting the clients represented by the bidders’ previous contracts: Customer assessments of previous contracting efforts indicate the scheduling standards were achieved without affecting cost or performance. Quality of services provided were professional and at the level expected by the customer. Past Performance Information demonstrates the offeror’s team can effectively manage large contracts similar in scope, complexity and size. Past Performance Information demonstrates previous teaming arrangements and positive business relationships. AR 519. Without in any way denigrating the importance of this evaluation, we view it to be fundamentally different from the task given to the Technical/Management Evaluation Panel. An inquiry into the quality of service, as measured by customer satisfaction, adherence to scheduling, and ability to team with partners, could produce meaningful results without any connection to whether the work done afforded the bidder experience in the precise subject of the Solicitation. Although the prior contract experience had to be “similar” in size, scope, and complexity, and involve “related” work, we view these words as intentionally not calling for detailed experience doing the same work that is the subject of the Solicitation. How a company manages any job is relevant to risk, but it is not the same question put to the Technical/Management Evaluation Panel. Those evaluators were tasked to ensure that the bidders had the “depth and breadth of [] experience and expertise,” AR 514, to “meet the requirements for each of the functional areas.” AR 517. In other words, the level of experience the 12 bidder has doing all aspects of the SOO. Among the more particularized standards, the panel was to ensure that “[t]he offeror demonstrates an understanding of the applicable information systems’ technical standards required to enable information sharing, integration, and interoperability by using best practices and align the evolving architecture with overarching federal, IC and DOD architecture guidance,” and that the offeror “provides an understanding of the US and allied forces’ logistics support system and proposes an integrated solution that allows efficient and rapid distribution of assets between DOD and its strategic partners, especially during times of national crisis.” Id. ManTech believes that the following narrative description, offered to support its [.............] rating for Past Performance, might have persuaded the Technical Evaluation Panel to come to a different result: [.................................................................................................. .................................................................................................... .................................................................................................... .................................................................................................... .................................................................................................... .................................................................................................... .................................................................................................... ..............] AR 11840. Only the last sentence arguably intrudes on the Technical Evaluation Panel’s responsibilities, but it has to be viewed in connection with the previous four sentences, which answer questions raised in the Past Performance standard. An exceptional ability to [........................................................................................................................ .....................]would not help the Technical Evaluation Panel to determine whether ManTech had the necessary expertise and experience to do the constituent elements of the SOO. In short, even if the Solicitation contemplated that the Past Performance rating would be furnished to the Technical Evaluation Panel, we view that omission as non-prejudicial. b. Assignment of [..............] Agencies must evaluate proposals solely on the factors and subfactors specified in the solicitation. See 48 C.F.R. § 15.305 (2009). Here, the Solicitation lays out five elements and nine standards for the 13 Technical/Management Factor. Of the nine standards for the Technical/Management factor, the first addresses technical experience. The technical experience standard ensures that “[t]he offeror has experience to meet the requirements for each of the functional areas identified in the SOO.” AR 517. The offeror must “provide the depth and breadth of its experience and expertise . . . with respect to the functional areas . . . .” AR 514. The Technical Evaluation Panel assigned ManTech [......................] in Operations Support Services and in Information Assurance Services. ManTech disputes [.....] [............................................................................................................. .......................................................................................................................... .......................................................................................................................... .....................] The evaluation panel assessed ManTech a [...................................................] for Operations Support Services because it [........................................................................................................................ ..........................................................................] AR 12084 (emphasis added). It also received a [....................] in the area of Information Assurance Services: [.................................................................................................................] 9 ManTech argues that the Solicitation does not require offerors to describe how they met or exceeded performance requirements on prior contracts, but merely whether “[ManTech] has experience [] to meet the requirements for each of the functional areas identified in the SOO.” AR 517. The evaluator’s emphasis on “how,” ManTech argues, strays from the requirements of the Solicitation. 9 The evaluation went on to explain: [.................................................................................................... ..................................................................................................... ..................................................................................................... ..................................................................................................... ..................................................................................................... ..................................................................................................... ..................................................................................................] 14 The technical experience standard required offerors to detail whether they had previous experience performing work similar to the work required by the SITE SOO. Defendant thus counters that “how” offerors meet the requirements of the Solicitation is not substantively different from whether they meet those requirements and thus was fair game for the evaluators. The word “how” only appears once in the Technical/Management evaluation standards: “The offeror provides a description of the company’s business planning and management processes and demonstrates an understanding of how to apply its resources across the enterprise.” AR 1342. Instructions to the evaluators do direct them to assess “[w]hether the proposal meets or fails to meet the standard.” AR 566; see also AR 564. We agree with defendant that, in light of the emphasis in the standards on “the depth and breadth of [the offeror’s] experience and expertise . . . with respect to the functional areas identified in the SOO,” the term “how” is legitimate shorthand for “whether” the offeror met the evaluation standards. AR 514. ManTech claims that evaluators treated bidders [........................] differently in this respect by not insisting that they show “how” they met or exceeded standards, as reflected by the fact that these [...] companies [................................................................] We disagree. Evaluators concluded that ManTech did not provide sufficient detail in [...........................] the Operations Support Services functional area of the SOO. AR 12084. [.....] evaluation reflects that it provided enough detail in [.......] of the twelve. The evaluators have the discretion to characterize [...] bidder’s proposal [............................................................] Likewise, according to ManTech, [..................] proposal did not receive a [................................] in Operations Support Services, yet it failed to show “how” it performed previously. Our review of [..................] proposal with respect to Operations Support Services, which is nearly [...............], reflects that it offered numerous examples of “how” it performed other work. [........................................................................................................................ .......................................................................................................................... ..........................] The evaluators also assigned ManTech a [....................] in “Information Assurance Services” because it provided little or no detail about its experience 15 in [......................] service areas. ManTech claims that it deserved [............................] which the evaluators gave [...........................................] ManTech’s competitors, however, failed to provide enough experience in three, five, and eight service areas respectively. It was within the discretion of the DIA evaluators to determine that ManTech’s lack of experience in [......] of the areas warranted a [....................]. c. Weight Given to the [..........................] ManTech next argues that the DIA elevated the relative importance of the [...] SOO areas for which it received [......................], Operations Support Services and Information Assurance Services. ManTech’s argument has little support. The evaluators referred to Operations Support Services as a “critical core functional area,” but ManTech had [......................], all relating to different functional areas within the SOO. Other offerors had the same Technical/Management Factor rating, with a similar number of [..........], all within different functional areas. Thus the evaluators did not disproportionately elevate the relative importance of Operations Support Services and Information Assurance Services. d. DIA’s Assignment of [............] to ManTech’s Technical/Management Proposal ManTech goes on to argue that the DIA unreasonably assigned [..........] in its proposal. The evaluators assigned [................] throughout the [....] elements of ManTech’s Technical/Management proposal. ManTech challenges several of these assigned [..........], which pertain to [.............] SOO areas. i. Storage Services First, ManTech argues that it did not deserve [..........] in Storage Services, in part by relying on its Past Performance Rating. But as we explained previously, the Technical/Management inquiry into depth and breadth of experience is distinct from the Past Performance examination of whether ManTech could manage contracts of similar scope, size, and complexity. ManTech also argues that the evaluators did not credit or understand ManTech’s storage experience under [...] previous contracts [................]. ManTech’s proposal did not use the word [.........] in describing its experience 16 as to these contracts. AR 4749. Although the [....] synopsis does show that M a n T e c h [........................................................................................................................ ..............] The evaluator’s discretionary evaluation is not an error. Likewise, the [....] contract synopsis mentions ManTech’s addition of [............................], but the evaluators considered that [..........................................................................................] AR 12083. It is not for us to regrade ManTech’s proposal. See E.W. Bliss Co. v. United States, 77 F.3d 445, 449 (Fed. Cir. 1996) (“technical ratings . . . involve discretionary determinations of procurement officials that a court will not second guess”). ii. Acquisition and Property Management Services ManTech questions the [........] assigned with respect to Acquisition and Property Management Services. The evaluators critiqued ManTech for not addressing [............................................................................] ManTech argues that the evaluators overlooked its reference to a contract which required [....] experience, although the contract synopsis did not use the phrase [.......] [.................................................................................................. .................................................................................................... ...................] AR 4752. As defendant suggests, this language buttresses the evaluators’ rating. Plainly, [....] does not comprise all [...........................................] Rather, [....] constitutes but one area of [............................] “Undoubtedly, an offeror carries the burden of presenting ‘an adequately written proposal . . . .’” See Westech Int’l, Inc. v. United States, 79 Fed. Cl. 272, 296 (2007) (quoting United Enter. & Assocs. v. United States, 70 Fed. Cl. 1, 26 (2006)). The evaluators would need a leap of logic to assume ManTech had [....] experience from its proposal; it is not the evaluators’ job to make that jump. 2. Small Business Plan Factor ManTech argues that its Small Business Plan Factor rating was arbitrary and unequal. First, ManTech points out that the Small Business Plan Panel Report assessed it a weakness for “not demonstrat[ing] a commitment to 17 exceed proposed DIA’s baseline goals.” AR 11902. Defendant responds that the consensus report, the source selection committee report, the source selection decision and the debriefing slides alike do not repeat or even reference this “weakness.” It contends that the panel reference to a weakness was in error and that the error was harmless. The consensus report was the first documentation of the evaluation panel’s findings. The consensus report included no strengths or weaknesses for ManTech’s Small Business Plan proposal and assigned ManTech its [................] rating. ManTech’s rating of [............] was therefore generated initially without the possibility of a taint from the subsequent introduction of the erroneous weakness. The Small Business Evaluation Panel then drafted the July 17, 2009 Small Business Panel Report with the information from the earlier consensus reports, where the “weakness,” along with a strength, makes its first and last appearance. None of the subsequent references to ManTech’s Small Business Plan evaluation, the Source Selection Evaluation Team Report (“SSET Report”), the Source Selection decision, nor the debriefing material, mention that weakness or the strength. We conclude that the error was without injury. See Andersen Consulting v. United States, 959 F.2d 929, 935 (Fed. Cir. 1992) (“De minimis errors are those that are so insignificant when considered against the solicitation as a whole that they can safely be ignored . . . .”). On the assumption that the word “strength” that appears in the Panel Report was intentionally used, ManTech also argues that it must not have been considered, otherwise its treatment would have been inconsistent with the way other bidders were evaluated. It points out that [.................] which received an exceptional rating, was given a strength for its [...........] small business participation, which exceeds the goals of the SITE Solicitation. The strength that appears on ManTech’s Small Business Panel Report also notes that it proposed exceeding that goal. Our initial observation is that, if we are correct that the word “strength” was not intentionally used and was never considered by either the Small Business Plan evaluation panel or by subsequent reviewers, then there is no apparent contradiction. We note that even if the strength on which ManTech relies was intended as an accurate assessment, there is still no necessary contradiction between how ManTech and [................] were treated. The evaluation was not limited to whether they exceeded DIA’s goals numerically. The evaluator’s rationale for the strength assigned to [................] went into substantially more detail and was certainly not identical to the description of 18 ManTech’s strength. Moreover, the initial panel reports reflect substantially more enthusiasm for [.................] plan. It is certainly within the Panel’s discretion to treat one proposal as [..........] and the other as [............] Finally, ManTech contends that it was treated unequally with respect to the Small Business Factor based on a comparison to the Panel’s description of another competitor’s plan, that of [.........] received an [...........] rating after the evaluators noted, among many other things, that the company historically had subcontracted [...] of its work to subcontractors and [................................................................................] AR 11904. ManTech points out that it has historically subcontracted [...] of its work and offered to use [...........] such small business teaming partners. ManTech contends that it too should have been assigned a [.........] based on these numbers. Even if the use of the term “strength” with respect to ManTech was intentional, we note that the description of [.....] “strengths” is far more extensive and particularized and certainly not limited to raw numbers. In fact, the “strengths” paragraph does not even mention [.........................] small business participation. AR 6207. We find that DIA’s evaluation of ManTech’s small business plan was not arbitrary. 3. The Competitive Range Determination The Solicitation sets out the process by which the SSAC and SSA make the final CRD: 7.0 FINAL SOURCE SELECTION DECISION (also applicable to Competitive Range determination and evaluation of Final Proposal Revisions) 1. All proposals will first be ranked in descending order, based on individual aggregate noncost/price evaluation factor scores in one column and next to each score the associated aggregate cost/price. 2. For each proposal that ranks HIGHEST in BOTH non-cost/price as well as cost/price, award will be recommended, based on an aggregate score rank list. If successive proposals have a higher cost than a previous proposal, a paired trade-off best value analysis will be conducted as follows: 19 a. A recommendation will be made with accompanying rationale explaining why or why not the additional delta in cost/price is or is not worth the delta in noncost/price when paired with each subsequent proposal. This trade-off will continue for all proposals to establish a Composite Best Value Final Ranking recommendation. b. From this Composite Best Value Final Ranking Recommendation, following consultation with Source Selection Advisors the SSA will make a final source selection. AR 527. In performing the “paired trade-off best value analysis,” the SSAC first ranked all proposals one through eleven based on non-cost factors. It then did a comparison between a higher non-cost ranked proposal and the next lower ranked proposal if the lower ranked proposal was also lower priced. This produced a final ranking, in which ManTech remained in [.............]. At the end of that re-ranking, the SSAC eliminated the five lowest-ranked offerors. The following table depicts the result of this tradeoff analysis: Final Ranking 1 2 3 4 5 6 7 8 9 9 11 Offeror Non-Cost Ranking Cost Ranking 1 3 2 5 4 6 7 8 10 10 11 8 1 6 2 9 7 3 5 4 11 11 Price In the table above, for example, [....] displaced [...............] because its price was over [............] less than [.................]. By a similar comparison, DIA elected not to have ManTech displace [....] despite the fact that ManTech’s p ro posal cost [..................] . T he SSA C c onc lude d tha t [........................................................................................................................ 20 ....................................................] In accepting that recommendation, the SSA concurred with the elimination of ManTech based on the following analysis: [.................................................................................................... ..................................................................................................... ..................................................................................................... ..................................................................................................... ..................................................................................................... ..................................................................................................... ...................................................................................] AR 12211. ManTech argues that the Solicitation is not clear whether the “trade-off best value analysis” requires the DIA to compare only against the proposal immediately next in line or with every proposal further down the line. In this case, ManTech argues that the DIA also had to justify [.....] higher price in comparison to ManTech. The Solicitation refers to “a paired” analysis: “If successive proposals have a higher cost than a previous proposal, a paired trade-off best value analysis will be conducted.” AR 527. Although it also uses the term “each subsequent proposal,” AR 527, in performing that paired analysis, the DIA interpreted this to mean each successive pairing of offerors. See AR 12167-70. We believe this is a fair interpretation of the Solicitation. A comparison across more than two offerors would by definition put at least three offerors simultaneously at play. In any event, we find that the possibility of prejudice is remote. The SSAC chose to maintain [... ]in a higher final position than [...] despite a price which was somewhat higher. It therefore did not perform a comparison between [...] and ManTech. [...] and [...] were both ranked [.............] with respect to the Small Business Factor, unlike ManTech, which was ranked [.............] As to the Past Performance factor, ManTech was ranked [..............] whereas [...] and [...] were both ranked [.............] But the most important factor was Technical/Management, as to which [...,] like [....] was ranked [.............] unlike ManTech which was ranked [...............] Within that ranking, the differences between [...] and ManTech are noticeable. [...] received 4 strengths, one weakness and one significant weakness. ManTech, on the other h a n d , 21 [........................................................................................................................ .......................................................................................................................... .......................................................................................................................... ............................] Price only became a “major factor in award selection when other [non-cost] criteria are substantially equal.” AR 522. Here they were not. Although we conclude that a direct price tradeoff between [...] and ManTech was not required, we believe that the likelihood of a different result is too remote to warrant a remand, even if it had been contemplated. The agency’s emphasis on the Technical/Management factor was not an abuse of discretion or inconsistent with the Solicitation. The agency has substantial discretion when, as in this case, the award will be based on “best value” rather then simply on price. See Galen Med. Assoc., 369 F.3d at 1330. It was certainly not an abuse of discretion to establish the Technical/Management factor as the most important factor, and, within that factor, to make technical experience in the SOO areas the most important element. AR 513-14. The contracting officer also has wide discretion to limit the competitive range for “efficient competition.” 10 U.S.C. § 2305(b)(4) (2009); 48 C.F.R. § 15.306(c)(2) (2009). Here, the Solicitation specifically allows the contracting officer and the SSA to limit the competitive range on the basis of efficiency to all but the most competitive proposals. The evaluators were well within their discretion in assigning ManTech a [..........] rating despite its [...........] Past Performance rating and the SSA was within his discretion in limiting the competitive range to [............] proposals. ManTech has not shown that the agency acted in an arbitrary way in its technical evaluation, its cost/price tradeoff, or its CRD. The decision to exclude ManTech from the CRD was not improper. B. L-3 1. Technical/Management Factor L-3, like ManTech, disagrees with its [........] Technical/Management rating which was at least partly the result of being assigned [....................] L-3 argues that [......] of these were arbitrarily assigned: Turnover Rate (with the element, Recruiting, Training, and Retention), Computer Network Defense, Application/Tool/Service deployment, TEMPEST, DOD Architecture, NonDIA Organizational Structure, Operations & Maintenance (part of Operations Support Services), Inventory Control and Software Licensing Experience, 22 Systems Administration, Communications Security Support (“COMSEC”) Experience, and Release Management. We will consider each in turn. a. Turnover Rate One of the elements of the Technical Management Factor was the Contractor’s SITE Organization/Management Team and Proposed Management Structure. Within that element, offerors had to provide information regarding recruiting, training, and retention. Part of that submission was information regarding the “company’s turnover rate for the business unit that will support SITE on an annual basis for the past (3) years.” AR 517. Offerors also had to (1) “[d]escribe [their] company’s management structure, [and (2) detail] [h]ow they will ensure adequate experience regarding surge capabilities . . . [and (3) provide] a detailed explanation of how the offeror will ensure that it does not experience a high personnel turnover rate . . . .” AR 516. The evaluators assigned L-3 [....................................................................................................] L-3’s principal argument is that [................] one of the successful offerors, was given an acceptable rating on Technical Management, and given a “meets standard” with respect to SITE Organization/Management Team and Proposed Management Structure, despite the fact that it did not submit turnover rates and submitted retention rates instead. [...............] responded to this issue with the following information: [............................................................................................................. .......................................................................................................................... ........... .......................................................................................................................... .......................................................................................................................... .......................................................................................................................... .......................................................................................................................... .......................................................................................................................... ........................................................................................................................] [...........] This information is plainly relevant to the inquiry, and the agency accepted it as satisfactory, but L-3 contends that it was error not to treat this 23 information as non-compliant, because [...............] submitted its retention rates in lieu of turnover rates. The evaluators described [.................] proposal as follows: [.................................................................................................. .................................................................................................... .................................................................................................... ........................................................................... .................................................................................................... .................................................................................................... .................................................................................................... .................................................................................................... .................................................................................................... .................................................................................................... ..................................................................................................] AR 12074. The Technical Evaluation Panel obviously inferred a turnover rate from the retention rate. L-3 argues, relying on information not within the administrative record, that retention rates are not necessarily equivalent to turnover rates, and that [...............] may have been trying to “hide a high turnover rate.” L-3’s Memo in Supp. of Mot. at 26. There is no evidence that [.................] turnover rate was, in fact, any higher than that of L-3. [...............] plainly assumed, and the evaluation panel accepted the assumption, that there is a close connection between retention rate and turnover rate. On the face of it, that is not an inherently unreasonable assumption. Equally important, the evaluators were considering a standard which encompassed much more than turnover rates. T h e y f o u n d t h a t [...................................................................................................................... ................] AR 12074. The evaluators were not arbitrary in being satisfied with [.................] retention plan as a whole. b. Computer Network Defense The technical evaluation panel assigned a [........] to L-3 on the technical experience element based on its reading of what L-3 wrote in its proposal concerning its experience on the [.....] contract: 24 [.................................................................................................. .................................................................................................] AR 3980. The DIA assigned the [........] because [..........................................................................] AR 12058. L-3 now argues that this is a misinterpretation of the reference. [........................................................................................................................ ..............................................................] That may have been L-3’s intent in providing the information, but it was not irrational for the DIA to draw the inference it did. L-3 had the burden to provide a well-written proposal. See Westech Int’l, Inc, 79 Fed. Cl. at 296. c. Application/Tool/Service Development One of the SOO functional areas asks the offerors to demonstrate that they can capably perform a task order to develop software to improve business and mission processes and improve application effectiveness.” AR 1289. The DIA assigned L-3 [.........] under the technical experience element: [.................................................................................................. .................................................................................................... .................................................................................................... .................................................................................................... .........................] AR 12056. The relevant paragraph from the SOO, 3.3.1, states “perform tasks to improve business and mission processes and improve application effectiveness.” AR 1289. In response, L-3 argues that it did provide the necessary description, referring to language in its proposal descriptive of prior contracts. There is no reason to think that the evaluators “missed” this information. It was plainly in the proposal. Apparently they concluded that it was less detailed than necessary, and that the information provided by [.....................................] to which plaintiff points for its argument of unfair treatment, was sufficiently detailed. This is precisely the type of argument which the court is in a poor position to review. We cannot substitute our judgment for that of the evaluators. 25 d. TEMPEST (“Transient Electromagnetic Pulse Emanation Standard”) Experience L-3 argues that, as required within the technical experience element, it committed to use of the TEMPEST standard. But the DIA assessed it [ ...........] Furthermore, L-3 argues, [....] did not receive [.........] As to L-3’s first contention, we note that it arguably merely reflected the Solicitation’s words back to the agency. Compare “[w]hen applicable, in the performance of any maintenance, or installation of modifications or changes, the Contractor shall ensure the existing [TEMPEST] profile is not degraded unless such a deviation is approved in writing by the appropriate Government authority,” A R 1 2 9 8 , w i t h [........................................................................................................................ ...................................................] [..........] The evaluators were reasonable in assigning L-3 [................................................................] We also note that in contrast to [....] having only a “bare” claim of TEMPEST experience, [....] actually cites [...] years of experience providing repair part support for . . . TEMPEST hardware,” among other citations. AR 6313. Again, the evaluators were reasonable in not assigning [................] e. Technical Standards and DoD Architecture SOO 3.3.3, Systems Engineering Services, calls in subsection 3.3.8, for an understanding of DoD architecture and technical standards. The evaluators f o u n d t h a t L 3 [........................................................................................................................ .......................................................................................................................... ...............................................................................] A R 12061. [........................................................................................................................ .......................................................................................................] AR 12061. L-3 argues that the evaluators’ findings are inconsistent: L-3 asks that if it [.....................................] how did it demonstrate an understanding of applicable standard. The evaluator’s findings are not inconsistent. L-3 demonstrated an understanding of the standards that it mentioned in the proposal, but it [.............................................................................................] Though L-3 cites many areas of its Technical/Management volume to support its contention that it did discuss multiple standards, we are not positioned to decide if L-3’s 26 citations amount to more than [...............]. guessing the evaluators. We have no basis for second f. Non-DIA Organizational Structure Under the second element of the Technical/Management Factor, offerors were to provide “evidence of a sound management structure that demonstrates an adequate organizational structure.” AR 1342. The evaluators c r i t i q u e d L - 3 f o r a p r o p o s a l t h a t [..........................................................................................................] AR 1 2 0 5 3 . T h e y a l s o w r o t e t h a t [........................................................................................................................ ....................................................................................................] [................] L-3 contends that it understood clearly that task orders could originate from many sources other than DIA, and that this understanding was, at a minimum, implicit in its proposal. According to L-3, the [...................................................] with respect to those numerous [.......] potential clients should thus be seen, not as a failure take them into account, but as an indication that L-3 recognizes that they are, at this point, unknown. Defendant responds that the [......................] L-3 offered gave the evaluators concerns about [..................] for oversight of such [.......] relationships. L-3 knew it would be managing multiple entities and working with m u l t i p l e a g e n c i e s . [........................................................................................................................ ...] 10 It was not arbitrary for the agency to treat this as a concern. g. Operations Support Services: Operations and Maintenance The evaluators assigned L-3 [..........] under the technical experience e l e m e n t f o r [........................................................................................................................ 10 As defendant correctly points out, L-3’s assertion that [...] was treated differently in this respect is unavailing. [.....] proposal mentioned the need to manage relationships with [.......] entities. Thus the evaluator’s decision not to assign it [..........] in this respect was not irrational. 27 ...........................................................................................] AR 12056. L-3 argues that the Solicitation does not require offerors to describe in detail how they performed prior contracts. Defendant responds that [........................] . . . for failing to describe ‘how’ it did perform in its previous experience.” L-3's Memo in Supp. of Mot. at 20. We have dealt above with a similar argument made by ManTech. The result here is the same. It was within the agency’s discretion to draw a dinstinction in offerors’ proposals based on how much detail they provided about how their prior contracts satisfied the agency’s need for experience in the various components of the SOO. h. Inventory Control and Software Licensing Experience Under the technical experience element, the evaluators gave L-3 [..........] under the Operations and Maintenance of Hardware and Software o b j e c t i v e f o r [........................................................................................................................ ....................................................................................................................] AR 12053. This objective requires the offeror to show how its depth and breadth of experience allow it to “provide the capability to support installed hardware and software . . . . [R]esponsibilities include but are not limited to hardware and software recapitalization, inventory control, software licensing, software and security management, hardware repair and remote servicing.” AR 1291. L-3 did not mention [...........................................] in any of the previous contract experience it lists within its SOO 3.4.2 proposal area. L-3 does claim that its proposal, albeit elsewhere, shows experience in these areas. The evaluators were within their discretion to determine that L-3 did not adequately address the necessary areas through indirect references. L-3 is left arguing that the evaluators acted unequally when they went outside the relevant SOO 3.4.2 area for [...............] and [....] L-3 asserts that these offerors also did not address [.......................................] under SOO 3.4.2. [...] did address both topics under SOO 3.4.2, referencing its [.....] contract for inventory control experience and its [....] contract for software licensing experience. 28 [...............] did not receive [..........], despite the fact that it mentioned [......................] cost reductions and an [.............................] elsewhere in its proposal, not with respect to SOO 3.4.2. L-3 points out that it also indirectly referenced [........................................] in other areas of its proposal: [.........................................................................] AR 4006; AR 4007. Because [..........] outside references are more extensive, it was within the evaluators’ discretion to distinguish between the two offerors. i. Systems Administration The DIA assessed L-3 [..........] in the Systems Administration area under the technical experience element: [.............................................................................] A R 12057. [........................................................................................................................ ...........................] AR 12057. L-3 contends that its table of [.........] contracts under the Systems Administration part of the SOO, were sufficient to “convey its depth and breadth of experience.” L-3’s Memo in Supp. of Mot. at 24. L-3 also argues that the evaluation is unequal, pointing out that [...............] and [...] fail to mention the experience [........................................] In L-3’s chart, it references [.................] under the [......] and [........] contracts. And it does mention [.........] under the [.........] contract. AR 3948. L-3's chart uses only [............................] for each contract. In contrast, [...............] lists [...] contracts under Systems Administration: each contract explains the challenge to [..................................] solution, and the resulting benefit to the government. The evaluators were within their discretion in assigning a higher rating based on this distinction. In [.....] Systems Administration area in its proposal it lists [....] contracts, each with one or two sentence summaries, none using the words “system upgrades,” “file management,” “back-ups/restorations,” or “system optimization.” AR 2358. Unlike L-3, the prose portion of [.....] proposal addresses [...................] and [...................] systems administration approach which [...] grounds in reference to [..................] and [..................................] L-3’s [.................] merely repeats the language of the SOO. The evaluators could have reasonably concluded that the [........] approach, bolstered by references to contracts that support that approach, was superior to L-3’s proposal. 29 j. COMSEC DIA assigned L-3 [........] under the technical experience element for not [........................................................................................................................ ..........] AR 12057. L-3 argues that its experience is [.............................] and that the Solicitation does not require it to discuss regulations and standards. L-3 argues that [...] and [.........] proposals are similarly [.........]. The COMSEC guidelines require offerors to “provide government personnel with training per applicable regulations.” AR 1293. The DIA was not arbitrary in citing L-3’s failure to [...................] as part of its [........]. Moreover, L-3 does not claim that it [.................................................]. L-3 provides a table of contracts; on many of the contracts L-3 dealt with COMSEC. [...] provides a bit more detail in referencing their own COMSEC contracts. And [.......] also references contracts where it managed COMSEC. We find no clear basis for concluding that the evaluators’ assessment of that material, as contrasted to that of other offerors, was arbitrary. k. Release Management The DIA assigned L-3 [..........] under the technical experience element in the Technical/Management volume with respect to release management e x p e r i e n c e . [........................................................................................................................ .......................................................................................................................... .......................................................................................................................... .........................................................................................] L-3 argues that this misstatement occurs in connection with a different part of its proposal, not directly connected to its release management experience. The evaluators were well within their discretion to read the next page of L-3’s proposal see language indicating [..............................................] and to assign [..........]. l. L-3’s Technical/Management [...............] Rating 30 L-3 argues that it does not deserve a [........] rating for the Technical/Management Factor. First, it contends that it did not deserve [......................] assigned to it. We have dealt with those above and are not persuaded that any of them were assigned arbitrarily. Next it contends that no total number of mere [.............] in its case [........], can be the basis for a [........] rating. It contends that only the assignment of [......................] could support such a rating. L -3 points to the definition of a [........................................................................................................................ ................] AR 561. L-3 concludes that a “[........] therefore reflects an acceptable, even if sub-optimal, solution.” L-3’s Reply in Supp. of Mot. at 27. Furtherm ore , [...............] ratin gs re flect th e agency’s [..........................................] therefore, L-3 argues, “[n]one of [..........] identified should have [...............................................................] Id. We disagree. An [..........] proposal is defined as one in which [ . . . . . . . . . . . . . . . . . . . . . . . . ] A R 5 2 4 . [....................................................................................................] AR 12211, 12167. [.....................................................................................] AR 12167. The Technical/Management Evaluation Panel Report Memorandum provides a d d i t i o n a l r e a s o n s f o r L - 3 ’ s r a n k i n g : [........................................................................................................................ ..........................................................................................................................] AR 11960. The evaluators were operating well within the scope of their discretion in concluding that L-3’s proposal was [........], despite the lack of [.........................] 2. The DIA’s Past Performance Evaluation The DIA had to evaluate L-3 according to the six standards in the Solicitation. L-3 seeks to show that its [..........] rating, a rating awarded to half of those large businesses included in the competitive range, should be an [...........] rating. L-3 also contends that its competitors did not offer a comparable record in terms of the size of its previous contracts. Managing [.....] contracts worth over [...........] L-3 contends, should give it a rating superior to those of competitors with fewer [.....] contracts. Under the fifth standard, the DIA had to evaluate “whether the offeror’s team [panel] can effectively manage large contracts similar in scope, complexity and size.” AR 519. Unhappy [..............................] L-3 claims the 31 “high dollar value [] of previous contracts, combined with [its] strong record of performance . . . should have led DIA to credit [them] with a strength or significant strength.” AR 11830. L-3’s Memo in Supp. of Mot. at 48. For further support, L-3 claims that the DIA “doled” out strengths “much more liberally” to [.................................................] L-3's Memo in Supp. of Mot. at 48. Under the third standard, the evaluators considered “[c]ustomer assessments of [L-3’s] previous contracting efforts [and whether they] indicate the scheduling standards were achieved without affecting cost or performance.” AR 519. A sampling of comments from L-3’s references across several contracts reveals that its performance record [........................................................................................................................ .......................................................................................................................... .......................................................................................................................... .......................................................................................................................... .......................................................................................................................... ..............................................................................................] Though the evaluators were interested in the dollar volume of past contract performance, size was only one consideration. The contracts had to be similar in scope and complexity to the SITE contracts, and the evaluators compared the reactions of past customers as to performance satisfaction. A close examination of the excerpts from its submission and those of other offerors does not warrant the court’s intrusion. We find that there is no basis for a finding that the agency acted in an arbitrary fashion. 3. The DIA’s Small Business Plan Evaluation L-3 received an [............] rating for its small business plan. It argues that it should have received an [.............] rating. The Small Business Evaluation Panel in its Consensus Report wrote that L-3 met all three of the evaluation standards, but it assigned no strengths or weaknesses. In the subsequent Panel Report, however, both a strength and a weakness appear after a paragraph summarizing L-3’s plan. This is the same phenomena discussed earlier in connection with ManTech. The strength shows that L-3’s goals for small business met the standard and that its historical information [........................................................................................................................ ...................................................................................................................] 32 Defendant concedes that the “weakness” listed in the Panel Report was erroneously included. L-3’s first argument is that subsequent consideration of its small business plan was tainted because of this improperly included weakness, causing it to receive an [............] rating and not the [...........] rating it “plainly deserved.” We disagree for the same reasons expressed in connection with ManTech’s proposal. The initial Small Business Evaluation Team Synopsis Report did not contain the error and produced [....................] rating. Nor does the weakness appear in the follow-on SSET Report, the Source Selection Decision, or the debriefing materials. The “weakness,” we conclude was indeed a typographical error which had no impact. Without that weakness, L-3 argues, its evaluation, armed with only a strength and no weaknesses, deserves an [...........] ranking. L-3’s argument rests on three assumptions: that the erroneous weakness affected its rating, that its strength is valid, and that the evaluation was unequal. We have already rejected the first assumption. Taking up the second assumption, L-3’s lone strength has the same features of the erroneous weakness: it appears only within the Small Business Plan Panel Report and neither in the Consensus Report nor the subsequent reports. Unlike the typographical error that lead to the weakness, the strength comprises text listed within the Consensus Report. The Consensus Report labeled the text as [.....................] the Small Business Plan Panel Report apparently simply mislabeled the same text under the “strength” heading. AR 11922. We reiterate our belief that the Consensus Report is most indicative of L-3’s [................] rating. We conclude that the “strength” label is a typographical error as well. L-3 argues that the DIA treated it inconsistently when compared with other bidders who each received an exceptional rating. Even if the evaluators intended that strength as an accurate assessment, L-3’s inequality argument fails. L-3 and defendant spar in great detail over the various aspects of L-3’s proposal as compared to those of [...........................................], but at the end of the exchange, three things can be said with certainty: the [....] proposals are not identical. The evaluation panel was able to articulate why it was considerably more impressed with the other [.....] offerors’ small business plans. And there are no indisputable contradictions in its assessments of those [.....] proposals compared to plaintiff’s plan. We are left with no legitimate basis for reversing L-3’s evaluation. 33 4. DIA’s Competitive Range Determination The SSA excluded L-3 from the competitive range. In doing so, the SSA relied on and concurred in whole with the SSAC’s Competitive Range Determination Report. The SSAC Report briefly outlines its recommendation, summarizes each offeror’s proposal, and then provides an appendix which addresses each offeror’s individual factors. As to L-3, the report reflects: [.................................................................................................. .................................................................................................... .................................................................................................... .................................................................................................... ......] AR 12167. In its summary of its evaluation of L-3, the SSAC concludes that [..................................................................................................................... . . [........................................................................................................................ ...............] AR 12170. In the appendix, the SSAC listed all of L-3’s [.............] In adopting the SSAC’s recommendation with respect to the overall CRD decision, the SSA reasoned that: [.................................................................................................. ..................................................................................................... .................................................................................................... .................................................................................................... .................................................................................................... ...............................................] AR 12211. L-3 argues that the latter two statements are plainly erroneous, in that they refer to [........................] in connection with L-3’s proposal. The statements are correct with respect to [.....] but not with respect to L-3, which was [.........................] In addition, L-3 contends that the CRD “reflects a wholesale failure to determine whether the excluded offerors had a reasonable chance for award” and that “the SSA failed to look behind the adjectival 34 ratings assigned to the offerors.” L-3’s Reply in Supp. of Mot. at 45. We do not agree. First, we are not persuaded that the SSA was under the false impression that L-3 had any [.......................] much less that he relied on that misunderstanding to exclude it from the CRD. The SSA affirmed that he “provided oversight of the entire evaluation process, as well as reviewed all documentation . . . .” AR 12211. The SSA looked at the Consensus Report, the Consensus Synopsis Report, and the Technical/Management Evaluation Panel Report in addition to the SSET report. Only the SSET Report mentions L-3 having [......................] within its Technical/Management proposal. The SSET Report itself notes in its initial outline on page 5 of the report, [......................................................................] The evaluators who assigned the ratings were unequivocal in their three reports: L-3 had [.........................]. The im p re c ise lan g u a g e , q u o ted a b o v e , o b v io u sly re f e rs o n ly [.....................................................................] Next, the DIA did not have to determine whether offerors had a reasonable chance for award. Section 2305(b)(4)(B) of title 10 of the United States Code provides that “the contracting officer may limit the number of proposals in the competitive range, in accordance with the criteria specified in the solicitation, to the greatest number that will permit an efficient competition among the offerors rated most highly in accordance with such criteria.” The evaluation did not condition the CRD on the SSA measuring each offeror’s “reasonable chance.” See AR 509. Thus the SSA did not have to determine whether L-3 or any other offeror had such a chance. In any event, the DIA specifically found that L-3 was [..........................] [CRD]—a finding at odds with an offeror with a “reasonable chance.” AR 12211. L-3 cites Birch & Davis Intern., Inc. v. Christopher, 4 F.3d 970 (Fed. Cir. 1993), for the proposition that the competitive range must include all offerors with a reasonable chance for award. A previous version of the regulation was in place at the time Birch was decided and the difference between that regulation and the current one is material. The older version provided that: The contracting officer shall determine which proposals are in the competitive range for the purpose of conducting written or oral discussion. [T]he competitive range . . . shall include all proposals that have a reasonable chance of being selected for 35 award. When there is doubt as to whether a proposal is in the competitive range, the proposal should be included. See FAR 15.609(a) (1993). That regulation no longer exists and the current relevant instruction appears at FAR 15.306(c)(1): Based on the ratings of each proposal against all evaluation criteria, the contracting officer shall establish a competitive range comprised of all of the most highly rated proposals, unless the range is further reduced for purposes of efficiency pursuant to paragraph (c)(2) of this section. 48 C.F.R. § 15.306(c)(1) (2009). Paragraph (c)(2), in turn, provides that Provided the solicitation notifies offerors that the competitive range can be limited for purposes of efficiency (see 52.215-1(f)(4)), the contracting officer may limit the number of proposals in the competitive range to the greatest number that will permit an efficient competition among the most highly rated proposals. Id. § 15.306(c)(2). The Solicitation here does notify offerors. See AR 1280. The SSA was fully consistent with the regulation and the Solicitation in limiting the competitive range and excluding L-3, [......................................] 5. DIA’s Price Realism Analysis The Solicitation informs offerors: “Offers that are unrealistically low in total price will be considered indicative of a lack of understanding of the complexity and risk in meeting contract requirements and will not be considered for award.” AR 1338. The agency conducted a price realism review and concluded that none of the offeror’s prices were unrealistic. L-3 challenges DIA’s price realism analysis as arbitrary, incomplete, and mechanistic, at best. The Solicitation states that Cost/Price will be evaluated for price reasonableness and realism. It does not specify by what means the agency will conduct its analysis. “The proposed ceiling labor rates and composite mark-up rates (Labor Mark-up and Materiel Handling Fee Mark-up) will be evaluated 36 for price reasonableness and price realism.” 11 AR 1346. No further indication is provided as to how price realism will be measured. Each price proposal consists of hundreds of labor rates for various positions in a multitude of locations around the globe. Prices were to be “evaluated against a predetermined quantity of hours determined by the Government as reasonable and realistic . . . .” AR 1347. The record reflects that DIA prepared Internal Government Cost Estimates (IGCEs) prior to the Solicitation, against which to measure offerors’ prices. Neither the predetermined number of hours nor the ICGEs was disclosed to offerors prior to bidding. When the offerors’ total calculated prices were compared to the IGCEs, they turned out to be uniformly lower than the government estimates. There is no indication that the Cost Panel used the IGCEs for any purpose other than a comparison of total price. Instead, the panel undertook a different approach, which involved a determination of how individual labor rates compared to each other after developing a standard deviation analysis. The CO then made a realism determination based on total cost. The Cost Panel12 took the following steps in its price realism analysis: reviewed price volumes for completeness; compared material handling fees based on price competition; determined the median rates for each labor category, location, and year; calculated the average for each rate; established the range of one standard deviation for each labor category and year; identified each rate falling outside of that range; and counted the number of instances an offeror was outside, above or below, the range. A standard deviation analysis is simply a statistical tool for measuring dispersion of a set of data points which pivot on the mean of the set of numbers. It is particularly useful when examining data sets with many data points. One standard deviation unit above or below the mean encompasses 68% of the data points. The Cost Panel calculated the standard deviations of 11 Price reasonableness is an analysis of whether offered prices are too high. Price realism is an analysis of whether offered prices are too low. DMS All Star Joint Venture v. United States, 09-737, 2010 US Claims LEXIS 50, at *7 n.5 (Fed. Cl. Jan. 10, 2010). 12 Also referred to as the Price Panel. 37 1358 different labor rates for each offeror. AR 6538-8225 (Cost Panel Rate Comparison). The panel also prepared a summary narrative of all offerors, indicating for each labor category and location which offerors offered rates above and below one standard deviation. AR 8226-329. The panel followed this with a summary report containing a section dealing with each offeror. The summary report shows that the Cost Panel considered each price offered, figured the percentage of prices outside of the range of one standard deviation, calculated the total projected cost to the government, and compared this cost with both the median offered total price and the IGCE. The report also ranked the offerors by total cost. E.g., AR 8339 (section of Cost Panel Report dealing with L-3). All offerors had prices outside the range, some completely above the range, some completely below, and some mix of prices both above and below. With regard to realism, the panel concluded that no offeror’s rates for any location were unrealistic, only that they were “below the range.” E.g., AR 8334 (section dealing with [.........]). The CO then took the information assembled by the panel and prepared an individual cost report for each offeror. Included in those reports are detailed breakdowns of each price offered for each location for every year.13 The CO made a price realism determination for each offeror based on each offerors’ calculated total price. She found all offerors’ total prices to be realistic. L-3 finds problematic a number of aspects of the realism analysis. First, it points out that the agency seemingly ignored its own internal estimates. The majority of its attacks, though, center on the use of a standard deviation analysis. L-3 argues that use of a standard deviation analysis is both arbitrary per se and fails to adequately protect the government from risks associated from unrealistically low prices. L-3 also argues that, even if the standard deviation methodology was not arbitrary, DIA ignored the prices that should have been too low because they were outside of the range. As a preliminary matter, L-3 has not shown how it was prejudiced by the contracting officer’s price realism analysis. L-3 was not excluded on that 13 The CO also noted any information missing from the price proposals. 38 basis and it has not named any particular bidders that it contends should have been displaced.14 L-3’s rank was not adjusted on the basis of price nor was any other offeror. The record reflects that no offeror was displaced from the competitive range based on price. This makes sense given that the price factor was considerably less important and was to be considered only if all other factors were substantially equal. L-3 ranks [...................] offerors on price. For price realism to have made a difference here, [....] of the higher ranked bidders would have had to be displaced. L-3 cannot show that it was competitively injured based on the price factor when all offerors were subject to the same allegedly problematic realism analysis and when L-3 has not demonstrated that [....] higher-ranked offerors should have been excluded. It is important also to note that, despite L-3’s protestations that multiple offerors were [.......................................................] lower than the IGCE total sum, L-3 itself was [...........................] below the IGCE. AR 8339 [................................................................] On this basis alone we conclude that, even if there was error in the way the panel conducted its price realism analysis, there is no proof of prejudice to L-3. Price realism concerns itself only with those prices that are too low. Thus, as L-3 concedes, the review for price realism is for the government’s benefit, not that of the offerors. Indeed, a price realism analysis is not required for fixed price contracts because the risk is already born by the offeror. See generally Afghan American Army Servs. Corp. v. United States, 90 Fed. Cl. 341, 356 (2009); PharmChem, Inc., B-291725.3-5, 2003 CPD ¶ 148 (Comp. Gen. 2003). Where the agency elects to use it anyway, we decline to exact a high standard. 14 Although L-3 hints that [.......] the other bidders, [..................................] offered individual labor rates which in plaintiff’s view, were “egregiously underbid,” L-3's Memo in Supp. of Mot. at 17, it does not contend that their overall prices were unrealistic. It merely suggests that, had a proper realism analysis been conducted certain unspecified offerors would have been eliminated and L-3 “would have faced a different competitive fate.” L-3’s Memo in Supp. of Mot. at 9. 39 L-3 argues that the application of a standard deviation analysis is a mechanistic method devoid of any real analysis. L-3 describes this “naked standard deviation analysis” as one that “uses the offerors’ submitted labor prices as the only input and declares prices that are more than one deviation from the mean to be unrealistic, and . . . treats those prices within one deviation from the mean as necessarily realistic.” L-3’s Memo in Supp. of Mot. at 9. According to plaintiff, such a method tells the offeror nothing about the offeror’s understanding of the costs associated or ability to perform the work requested. Instead a standard deviation analysis expresses only the distance from each data point to the mean of all of the data points. Plaintiff further instructs that a standard deviation method is of even less use in the case of a highly deviated data set. This is because, as L-3 rightly points out, standard deviation only measures an average difference between a particular number and the average of all of the numbers included in a set. Put another way, in the case of a disparate set of numbers, the range of one standard deviation, which is about two thirds of all of the numbers, will include a wide range of numbers and is thus, in L-3’s view, not relevant to price realism. L-3 took the court through specific examples of the standard deviation analyses data sets from the record in both closely and widely dispersed sets of prices, to illustrate the point. The range of prices within one standard deviation of a highly deviated labor category is useless as a properlyconducted price realism analysis, in L-3’s view, and a price realism analysis based on a standard deviation analysis must therefore be arbitrary. L-3 then goes further to contend that DIA “ignored the results for offerors that submitted unrealistically low rates.” L-3’s Memo in Supp. of Mot. at 15. Plaintiff argues that DIA ignored both the comparison of offerors’ rates to its own internal estimates (IGCEs) and those prices falling outside of one stand deviation from the mean. Thus, in L-3’s view, DIA simply ignored rates that ought to have alerted it to underbidding and/ or a lack of understanding of contract requirements. L-3 cites Afghan American, 90 Fed. Cl. 341, and Planning Research Corp., GSBCA No. 10697-P, 91-2 BCA ¶ 23,811 (Sept. 11, 1990), in support. L-3 complains, in effect, that neither the IGCE nor the standard deviation analysis was used mechanistically as a disqualifier. It bears pointing out that this argument stands in stark contrast to its first line of attack. Here it argues that the IGCEs and standard deviation should have been applied 40 mechanically to disqualify offerors who fell afoul of these metrics, a method previously alleged to be devoid of real analysis. We disagree on all points. Because the Solicitation does not specify a process, the agency can employ its own judgment about what metrics to employ and how to apply their results. See Afghan American, 90 Fed Cl. at 358. Nor can we say that a standard deviation analysis on its face is per se arbitrary. The Solicitation promises no specific metric by which DIA would measure price realism, and the FAR is silent as to the specific mechanics of price realism. The FAR does not specifically mention price realism, but we have recognized it to be essentially the same analysis as the cost realism analysis found in FAR part 15.404-1(d): Cost realism analysis is the process of independently reviewing and evaluating specific elements of each offeror’s proposed cost estimate to determine whether the estimated proposed cost elements are realistic for the work to be performed; reflect a clear understanding of the requirements; and are consistent with the unique methods of performance and materials described in the offeror’s technical proposal. 48 C.F.R. § 15.404-1(d)(1) (2009). The FAR is silent as to how to conduct such an analysis. Erinys Iraq Ltd. v. United States, 78 Fed. Cl. 518, 530 (2007). Thus, “the nature and extent of a price realism analysis is ultimately within the sound exercise of the agency’s discretion, unless the agency commits itself to a particular methodology in a solicitation.” Afghan American, 90 Fed. Cl. at 348. What it must not be is reliant on irrational assumptions or miscalculations. DMS All Star Joint Venture v. United States, 09-737, 2010 US Claims LEXIS 50, at *31-32 (Fed. Cl. Jan. 10, 2010) (quoting OMV Med., Inc. v. United States, 219 F.3d 1337, 1344 (Fed. Cir. 2000). As L-3’s counsel conceded at oral argument, a standard deviation analysis does provide the government with information useful and relevant to a determination of price realism. It allows the reviewer to quickly assess both the range of prices within a set (by looking at the size of the standard deviation) and how many of the prices in a set fall outside the range. The clustering of prices is thus some indication of the import of outliers. 41 The standard deviation process allowed the panel and CO to view a vast array of numbers in a reasonable period of time, which culminated in the conclusion that, though some offered rates fell outside of the range, taken individually, and as a whole, no rates were unrealistically low. We believe this to have been within the agency’s wide discretion in conducting a price realism analysis, especially when the Solicitation does not promise a particular methodology. See Info Scis. Corp. v. United States, 73 Fed. Cl. 70, 102 (2006). Plaintiff cites the GAO decision in Multimax, Inc., B-294249.6-19, 2006 CPD ¶ 165 (Comp. Gen. 2006) for support. In Multimax, the agency’s price analysis, both for reasonableness and realism, failed because its twostandard deviation range method failed to “to assure that prices at the extreme of the ranges reflected reasonable pricing; rather, the agency mechanistically applied the formula and accepted the results without further analysis.” Id. In contrast, the agency here specifically noted the high and low prices for each labor category. E.g., AR 6544-545 (rate comparison for Installation Specialist). Given the Cost Panel’s conclusion in its report that no particular price was found to have been unrealistic, the agency did engage in the additional step of analysis found to have been missing in Multimax. Second, we also note that a range of two standard deviations is greater than a range of one standard deviation and thus more likely to miss true outlier rates. Finally, in Multimax, the range would have allowed prices that were so egregiously low that they actually fell below the federal minimum wage. Id. (“the lower end, in some instances, was below the federal minimum wage or was even a negative number”). There is no allegation that the one standard deviation range used here permits similar rates. Had the Solicitation promised that the CO would employ either IGCEs or that it would strictly apply a one standard deviation unit analysis to disqualify prices as unrealistic, DIA would have run afoul of the Solicitation. 5 1 Here, the agency retained the discretion to use these methods of comparison to analyze data and concluded nonetheless that no prices were unrealistic. Neither of plaintiff’s other cited cases change the outcome. In Afghan American, the agency relied upon a comparison of prices to internal agency estimates. The problem for the agency was that the estimates used were Although, such a method would seem to to be arbitrary according to L-3’s first argument that a standard deviation analysis is arbitrary per se. 15 42 fundamentally flawed because they omitted a component, “which added significant cost to the . . . services” solicited. 90 Fed. Cl. at 358. The court found that the comparison of offeror’s prices to the agency’s estimates was an “apples-to-oranges comparison” due to the lack of adjustment for the additional category of work included in the solicitation and not in the estimates. Id. at 359. The realism determination was thus based on “irrational assumptions or critical miscalculations.” Id. The circumstances here are not similar. We are not faced with a critical error in assumption or calculation. In Planning Research Corp., the board was confronted with an agency that wholy disregarded the discrepancy between the awardee’s low prices and all of the various “yardsticks” used to measure price realism. GSBCA No. 10697.16 The Board noted that both the agency’s and the awardee’s personnel expressed concern at the low rates offered. Id. The agency awarded the contract notwithstanding. The Board found such disregard for extremely low prices to be untenable. Though plaintiff would like us to see the current situation in a light similar to Planning Research Corp., we cannot. In sum, DIA acted within its discretion in conducting its price realism analysis. It was not based on any irrational assumptions or miscalculations. It bears repeating that price realism is voluntary analysis undertaken to protect the government from contracting to buy a service or good that the offeror might not be able to provide at that price. Here, the CO accepted a range of rates offered by all offerors as realistic. We decline to apply a more exacting standard. We conclude that DIA’s price realism analysis was neither arbitrary, capricious, nor contrary to law or regulation. L-3 has not shown that the agency acted in an arbitrary way in its technical evaluation, its cost/price tradeoff, or its CRD. The decision to exclude ManTech from the CRD was not improper. Due to the sensitive and protected nature of the information contained in the earlier decision, the reported decision of the General Services Board of Contract Appeals is a summary of the actual decision rendered earlier by the board. The summary reported decision unfortunately does not detail what “yardsticks” were used to measure prices against. 16 43 CONCLUSION For the foregoing reasons, plaintiffs’ motions are denied and defendant’s motion is granted. The Clerk shall enter judgment for defendant, dismissing the complaints. No costs. s/Eric G. Bruggink ERIC G. BRUGGINK, Judge 44
]]>
Some case metadata and case summaries were written with the help of AI, which can produce inaccuracies. You
should read the full case before relying on it for legal research purposes.
This site is protected by reCAPTCHA and the Google
Privacy Policy and
Terms of Service apply.
