United States v. Hunt, No. 21-6046 (10th Cir. 2023)
Annotate this Case
Justia Opinion Summary
Want to stay in the know about new opinions from the Tenth Circuit US Court of Appeals?
Sign up for free summaries delivered directly to your inbox. Learn More ›
You already receive new opinion summaries from Tenth Circuit US Court of Appeals. Did you know we offer summary newsletters for even more practice areas and jurisdictions? Explore them here.
Defendant Dominic Hunt appealed his convictions on two charges of being a felon in possession of ammunition. The ammunition was used in two shootings in early 2019. Investigators found three spent cartridges at the scene of one shooting and one spent cartridge at the other. A firearms expert testified that all four cartridges were fired from the same (undiscovered) weapon. Defendant’s sole issue raised on appeal was that the expert testimony should not have been admitted at trial: the expert’s field of firearm toolmark examination was not scientifically valid, and that the district court failed to perform its gatekeeping role in examining the admissibility of expert testimony because it relied on prior judicial opinions rather than the most up-to-date empirical evidence when it denied his pretrial motion to exclude the testimony without conducting a hearing. The Tenth Circuit held only that the district court adequately performed its gatekeeping role and did not err in admitting the testimony in light of the material presented on the pretrial motion and the expert testimony at trial.
Download PDF
Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 1 PUBLISH FILED United States Court of Appeals Tenth Circuit UNITED STATES COURT OF APPEALS March 24, 2023 FOR THE TENTH CIRCUIT _________________________________ Christopher M. Wolpert Clerk of Court UNITED STATES OF AMERICA, Plaintiff - Appellee, v. No. 21-6046 DOMINIC EUGENE HUNT, a/k/a Dime Sack, Defendant - Appellant. _________________________________ Appeal from the United States District Court for the Western District of Oklahoma (D.C. No. 5:19-CR-00073-R-1) _________________________________ Grant R. Smith, Assistant Federal Public Defender (Virginia L. Grady, Federal Public Defender, with him on the briefs), Denver, Colorado, for Defendant-Appellant. Jacquelyn M. Hutzell, Assistant United States Attorney (Robert J. Troester, United States Attorney, and David McCrary, Assistant United States Attorney, with her on the brief), Oklahoma City, Oklahoma, for Plaintiff-Appellee. _________________________________ Before HARTZ, SEYMOUR, and MORITZ, Circuit Judges. _________________________________ HARTZ, Circuit Judge. _________________________________ Defendant Dominic Eugene Hunt appeals his convictions on two charges of being a felon in possession of ammunition. The ammunition was used in two shootings in early 2019. Investigators found three spent cartridges at the scene of one shooting and one Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 2 spent cartridge at the other. A firearms expert testified that all four cartridges were fired from the same (undiscovered) weapon. Defendant’s sole complaint on appeal is that the expert testimony should not have been admitted at trial. He argues that the expert’s field of firearm toolmark examination is not scientifically valid and that the district court failed to perform its gatekeeping role in examining the admissibility of expert testimony because it relied on prior judicial opinions rather than the most up-to-date empirical evidence when it denied his pretrial motion to exclude the testimony without conducting a hearing. Exercising jurisdiction under 28 U.S.C. § 1291, we affirm. We need not declare a general rule on the admission of firearm toolmark testimony. We hold only that the district court adequately performed its gatekeeping role and did not err in admitting the testimony in light of the material presented on the pretrial motion and the expert testimony at trial. I. BACKGROUND On January 20, 2019, a car was stolen from the residence of Defendant’s cousin, Jimmy Jones. The theft was captured on the surveillance camera at Jones’s home. It showed that after his daughter started her car to warm it up and went back inside, a man exited a blue Hyundai that had driven by and then jumped in her car and drove off following the Hyundai. Jones later thought he saw the same blue Hyundai down the street, though it turned out that he was mistaken. Based on this misidentification, however, he, Defendant, Travis Carter (Jones’s brother), and Christopher Dawson (Defendant’s brother) confronted three men that Jones thought 2 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 3 were involved in the theft. This confrontation led to a fistfight, which ended when someone shot Del Lavar Brison, one of the men from the group that Defendant confronted. An officer with the Oklahoma City Police Department (OCPD) was called to the scene and asked Brison who shot him. Brison said something that the responding officer understood as indicating that “it was a black male wearing a . . . maroon jacket.” R., Vol. III at 213. The surveillance video from Jones’s home showed Defendant wearing a maroon hoodie at the time of the theft. An OCPD crime-scene investigator recovered one spent Blazer 9mm Luger cartridge case from the scene—it was the only cartridge case that was found. Less than two weeks later, in the early morning of February 2, 2019, a man named Conilius Wright was found unconscious in his truck after being mortally wounded in a drive-by shooting. One of Wright’s companions testified that Defendant and Wright had issues with one another, and Defendant’s then girlfriend testified that on the night of the shooting Wright spoke with her about possibly being the father of her child. Defendant’s cell-phone location data indicated that he was near Wright’s shooting one minute before it was reported on a 911 call. An OCPD crime-scene investigator recovered three spent cartridge cases near Wright’s vehicle: one Blazer 9mm Luger cartridge case and two Winchester 9mm Luger cartridge cases. The four 9mm Luger cartridge cases recovered from the January and February 2019 incidents were submitted to the OCPD Firearms Laboratory and were analyzed 3 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 4 by Ronald Jones, a firearm and toolmark1 examiner. Jones compared the cases using a microscope and reported on November 4, 2019, that (1) the three cartridge cases recovered at the scene of Wright’s homicide were all fired from the same unknown firearm and (2) that same unknown firearm fired the one cartridge case recovered from the scene of Brison’s shooting. On November 6, 2019, a federal grand jury returned a nine-count third superseding indictment against Defendant. The first seven counts arose from Defendant’s unlawful possession of a firearm and drug-trafficking activities five years before the shooting. This appeal concerns only the last two counts. Count 8 charged that Defendant violated 18 U.S.C. § 922(g)(1) by being a felon in possession of ammunition—the Blazer 9mm Luger cartridge case recovered from the scene of the January 2019 incident. Count 9, which was added in the third superseding indictment, charged that Defendant violated the same provision in February 2019 by possessing the one Blazer 9mm Luger cartridge case and two Winchester 9mm Luger cartridge cases recovered from the scene of Wright’s homicide. After the government disclosed that it intended to present firearms-expert testimony to show that the spent cartridge cases recovered from the January and February incidents were fired from the same gun, Defendant filed in March 2020 his 1 “Toolmarks are generated when a hard object (tool) comes into contact with a relatively softer object. Such toolmarks may occur in the commission of a crime when an instrument such as a screwdriver, crowbar, or wire cutter is used or when the internal parts of a firearm make contact with the brass and lead that comprise ammunition.” Nat’l Rsch. Council, Strengthening Forensic Science in the United States: A Path Forward 150 (2009). 4 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 5 “Motion in Limine to Exclude Ballistics Evidence, or, Alternatively, for a Daubert Hearing.” R., Vol. I at 106. Before our discussion of the arguments advanced in Defendant’s motion to exclude the firearms-expert testimony, an overview of firearm toolmark examination is in order. The method of comparing marks left on different pieces of ammunition to determine whether the ammunition was expended from the same firearm has been used for over a century. See, e.g., Commonwealth v. Best, 62 N.E. 748, 750 (Mass. 1902) (Holmes, C.J.) (rejecting a challenge to the admission of an expert’s firearmsidentification testimony). While advances in science and technology have refined the field of firearm toolmark examination, its object remains the same: “to determine whether ammunition is or is not associated with a specific firearm based on toolmarks produced by guns on the ammunition.” President’s Council of Advisors on Sci. and Tech., Forensic Science in Criminal Courts: Ensuring Scientific Validity of FeatureComparison Methods 104 (2016) [hereinafter PCAST Report]. The underlying concept is that different equipment and processes used in manufacturing firearms produce unique marks on the internal parts of the firearm. Firearm examiners theorize that even the same manufacturing tool will produce “microscopically different” marks on consecutively produced firearms, since “[m]anufacturing tools experience wear and abrasion as they cut, scrape, and otherwise shape metal.” Nat’l Rsch. Council, Strengthening Forensic Science in the United States: A Path Forward 150 (2009) [hereinafter NRC Report]. Also, the internal parts of the firearm may undergo individualized changes through use. Firearm toolmarks are then produced “when the 5 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 6 internal parts of a firearm make contact with the brass and lead that comprise ammunition.” Id. Specifically with respect to cartridge cases (which house the primer, gunpowder, and bullet):2 The brass exterior of cartridge cases receive[s] . . . toolmarks during the process of gun firing: the firing pin dents the soft primer surface at the base of the cartridge to commence firing, the primer area is forced backward by the buildup of gas pressure (so that the texture of the gun’s breech face is impressed on the cartridge), and extractors and ejectors leave marks as they expel used cartridges and cycle in new ammunition. Id. at 151. The experts in this case used the Association of Firearms and Tool Mark Examiners (AFTE) method to compare the toolmarks left on the four cartridge cases. See United States v. Hunt, 464 F. Supp. 3d 1252, 1256 (W.D. Okla. 2020). Under this method, examiners begin by evaluating class characteristics, “which are features that are permanent and predetermined before manufacture.” PCAST Report at 104. They are “family resemblances which will be present in all weapons of the same make and 2 At oral argument Defendant’s appellate counsel suggested for the first time that the district court erred to the extent that it found firearm toolmark examination reliable based on studies involving toolmarks left on bullets (as opposed to cartridge cases). We decline to consider this argument. See United States v. Malone, 10 F.4th 1120, 1124–25 (10th Cir. 2021) (“[I]ssues may not be raised for the first time at oral argument.” (internal quotation marks omitted)). We note, however, that while the processes that transfer toolmarks to bullets are not the same as those leaving marks on cartridge cases, see NRC Report at 151, the examinations of each “involve[] many of the same concepts,” United States v. Williams, 506 F.3d 151, 158 n.4 (2d Cir. 2007); see also Alfred Biasotti, John Murdock & Bruce R. Moran, Introductory Discussion of the Science—The Scientific Methods Applied in Firearms and Toolmark Examination, in 4 Modern Scientific Evidence: The Law and Science of Expert Testimony § 34:9 (David L. Faigman et al. eds., 2021) (discussing the general method of toolmark examination). 6 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 7 model.” United States v. Willock, 696 F. Supp. 2d 536, 557–58 (D. Md. 2010) (internal quotation marks omitted). For cartridge cases this includes caliber and the shape of the firing pin (which affects the firing pin’s impression on the cartridge). “If the class characteristics are similar, the examination proceeds to identify and compare individual characteristics, such as the striae that arise during firing from a particular gun.” PCAST Report at 104. Examiners can make an “Identification”—that is, conclude that two specimens were derived from the same source (such as a firearm)—when there is: Agreement of all discernible class characteristics and sufficient agreement of a combination of individual characteristics where the extent of agreement exceeds that which can occur in the comparison of toolmarks made by different tools and is consistent with the agreement demonstrated by toolmarks known to have been produced by the same tool. Ass’n of Firearm and Tool Mark Exam’rs, Glossary 94 (6th ed. 2013) (defining Identification) [hereinafter AFTE Glossary]. In arriving at an identification the examiner tries to avoid confusing individual characteristics with subclass characteristics—features produced during manufacture, not by design, that are “consistent among items fabricated by the same tool in the same approximate state of wear.” Id. at 121.3 On the opposite end of the spectrum, when there is “[s]ignificant 3 For example, “an imperfection on a rifling tool” can “impart[] similar tool marks on a number of barrels before being modified either through use or refinishing.” Ronald G. Nichols, Defending the Scientific Foundations of the Firearms and Tool Mark Identification Discipline: Responding to Recent Challenges, 52 J. Forensic Sci. 586, 587 (2007). An examiner who is not attuned to this possibility or lacks adequate training or experience may make a misidentification of two bullets as having come from the same weapon, when the two weapons that fired 7 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 8 disagreement of discernible class characteristics and/or individual characteristics,” an examiner can determine that the specimens came from different sources—this is known as an “Elimination” conclusion. Id. at 94 (defining Elimination). In recent years the field of firearm toolmark examination has been subjected to closer scrutiny as part of a general program to examine the validity of featurecomparison forensic methods used in the courts. We discuss three efforts which were of special importance in the district court’s resolution of Defendant’s pretrial motion: (1) the 2009 NRC Report, (2) a 2014 study by the Ames Laboratory, and (3) the 2016 PCAST Report. In 2005 Congress authorized the National Academy of Sciences to create a committee to study the needs and practices of the forensic-science community. See NRC Report at 1–2. The committee issued a report in 2009 in which it recommended that further research be conducted “to address issues of accuracy, reliability, and validity in the forensic science disciplines.” Id. at 22. With respect to firearm toolmark examination, the report found that “[s]ufficient studies have not been done to understand the reliability and repeatability of the methods.” Id. at 154. It also criticized the discipline’s “lack of a precisely defined process,” stating that “AFTE has adopted a theory of identification, but it does not provide a specific protocol.” the different bullets merely share a subclass similarity. See Adina Schwartz, A Systemic Challenge to the Reliability and Admissibility of Firearms and Toolmark Identification, 6 Colum. Sci. & Tech. L. Rev. 1, 8–10 (2005). 8 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 9 Id. at 155. The NRC Report recognized, however, that firearm toolmark evidence can be valuable: The committee agrees that class characteristics are helpful in narrowing the pool of tools that may have left a distinctive mark. Individual patterns from manufacture or from wear might, in some cases, be distinctive enough to suggest one particular source, but additional studies should be performed to make the process of individualization more precise and repeatable. Id. at 154. In response to the NRC Report’s recommendations, research was conducted to better assess the validity of firearm toolmark examination. We discuss here a study by the Ames Laboratory. See David P. Baldwin et al., A Study of False-Positive and False-Negative Error Rates in Cartridge Case Comparisons, Ames Laboratory, USDOE Technical Report # IS-5207 (2014) [hereinafter Ames Study]. The Ames Study tested 218 firearm examiners by sending each of them 15 sets of four spent cartridge cases. See Ames Study at 3–5, 8–10. The researchers generated the spent cartridge cases using 25 Ruger SR9 semiautomatic 9mm handguns.4 See id. at 5, 9– 10. The examiners were to determine whether the fourth cartridge case in each set (the questioned case) came from the same firearm as the three other cartridge cases in that set (the known cases). See id. at 4, 10, 15–16. For each examiner the three cartridge cases came from a different firearm than the fourth case in 10 of the 15 sets. See id. at 10. The examiners thus conducted 2,180 “true different-source 4 While the PCAST Report approved of the Ames Study’s research design, it did note one forensic scientist’s criticism that “the study did not involve consecutively manufactured guns.” PCAST Report at 110 n.331. 9 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 10 comparisons” in total: among these, there were 1,421 correct elimination conclusions, 735 inconclusive findings, and 22 erroneous identification conclusions. Id. at 16. The false-positive rate in the Ames Study, excluding inconclusive determinations, was 1.52%. This may overestimate the error rate for firearm toolmark examinations that most law-enforcement agencies and laboratories conduct. Notably, “[a]ll but two of the 22 false identification calls were made by five of the 218 examiners, strongly suggesting that this error probability is not consistent across examiners.” Id. And “the study specifically asked participants not to use their laboratory or agency peer review process.” Id. at 5. Therefore, the error rate for comparisons that are peer-reviewed could be lower than 1.52%. In 2015 the President’s Council of Advisors on Science and Technology began investigating “whether there [were] additional steps on the scientific side, beyond those already taken by the Administration in the aftermath of the highly critical 2009 [NRC Report], that could help ensure the validity of forensic evidence used in the Nation’s legal system.” PCAST Report at x. Like its predecessor, the PCAST Report—issued in 2016—said that there is a need for additional studies to verify the principles and methods underlying firearm toolmark examination; it found “that firearms analysis currently falls short of the criteria for foundational validity, because there is only a single appropriately designed study to measure validity and estimate reliability”—the Ames Study.5 Id. at 112. The PCAST Report also complained that 5 According to PCAST, “Foundational validity for a forensic-science method requires that it be shown, based on empirical studies, to be repeatable, reproducible, 10 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 11 the standard used by toolmark examiners in making an identification is ill-defined, asserting that the theory of identification is “circular”: “The ‘theory’ states that an examiner may conclude that two items have a common origin if their marks are in ‘sufficient agreement,’ where ‘sufficient agreement’ is defined as the examiner being convinced that the items are extremely unlikely to have a different origin.” Id. at 104. But the PCAST Report did not call for the immediate, wholesale exclusion of firearm toolmark evidence from the courts. Instead it took the position that “[w]hether firearms analysis should be deemed admissible based on current evidence is a decision that belongs to the courts.” Id. at 112. The PCAST Report’s assessment of this field stands in stark contrast to its assessment of the state of at least one other forensic-science method that involves feature comparison—bitemark analysis. See id. at 87 (“[B]itemark analysis does not meet the scientific standards for foundational validity, and is far from meeting such standards.”). With this background in mind, we now turn to Defendant’s motion to exclude expert testimony regarding firearm toolmark examination. His core argument was that “the Government cannot show that the field of ballistics identification is and accurate, at levels that have been measured and are appropriate to the intended application. . . . It is the scientific concept we mean to correspond to the legal requirement, in [Federal Rule of Evidence] 702(c), of ‘reliable principles and methods.’” PCAST Report at 4–5. PCAST recognized that “[s]ome methods that have not been shown to be foundationally valid may ultimately be found to be reliable, although significant modifications to the methods may be required to achieve this goal.” Id. at 14. 11 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 12 scientifically valid.” R., Vol. I at 109.6 After discussing the critiques presented in the PCAST and NRC Reports, he proceeded to argue that exclusion was warranted under Federal Rule of Evidence 702, applying the nonexclusive factors articulated in Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993), for assessing whether an expert’s methodology is sufficiently reliable for the expert’s opinion to be admissible evidence. The Daubert factors are: “(1) whether the [methodology] can be tested; (2) whether it is subject to peer review and publication; (3) the known or potential error rate; (4) the existence and maintenance of standards [to control the methodology’s application]; and (5) the general acceptance in the relevant scientific community.” United States v. Foust, 989 F.3d 842, 845 (10th Cir. 2021) (citing Daubert, 509 U.S. at 593–94). Defendant argued: On each Daubert factor, ballistics identification evidence falls short. First, the very premise of firearms analysis as a field—the theory of uniqueness of firearms, bullets and cartridge cases—rests on an assumption that has not been properly tested. It comes from a time when bullets were hand-cut and unique, but the field simply has not reckoned with standardization and technological developments in production. . . . Second, “peer review and publication” is limited. There has only been one black-box study on firearms identification. See PCAST Report at 11, 111. This study was not published in a scientific journal and was not subject to peer review or publication. Id. Third, [Defendant] is aware of only one black-box study conducted with respect to error rates. See [i]d. at 11. The single study estimated an error rate, but a single study estimating such an error rate is insufficient to determine a known rate of error for the field. . . . Fourth, there are no uniform standards for controlling the technique’s operation. Rather, an individual makes a subjective 6 Defendant’s motion also argued that the government “has not shown that [its] firearm expert, Ron Jones, conducted his examination of the ballistics evidence in a reliable manner.” R., Vol. I at 109. That argument is not renewed on appeal as to Howard Kong, the expert who testified at trial. 12 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 13 determination about an identification, based on his or her own internal compass. There is no required number of points of agreement for making an identification. . . . ... Although courts have also previously allowed firearms examiners to testify, the awareness of the limitations of ballistics identification is recent and growing. It does not appear that courts have yet seriously considered all aspects of the field’s development, or tested its reliability since the PCAST report decided it was not foundationally valid from a scientific perspective. . . . Since firearms identification meets none of the Daubert criteria, this Court should exclude testimony from the government’s proposed ballistics identification expert. R., Vol. I at 118–20. Defendant also contended that if the firearms-identification evidence was admitted, (a) the expert should not be permitted to overstate the certainty of his conclusion that the cartridge cases were a match and (b) the jury (1) should be “made aware that the foundational validity of firearms analysis as a field has not been established” and (2) should be informed of the error rates documented in the only “appropriately designed” study. Id. at 124. The government filed a response to Defendant’s motion in which it identified a second firearm expert, Howard Kong, who is a firearm and toolmark examiner with the Bureau of Alcohol, Tobacco, Firearms and Explosives (ATF) Forensic Science Laboratory in California. Kong became involved in the case after images of the two Blazer cartridge cases—one from each incident—were separately uploaded into the ATF’s National Integrated Ballistics Information Network (NIBIN), a database of “three dimensional digital ballistic images of spent shell casings recovered from crime scenes and from crime gun test-fires” that “can automatically generate a list of potential matches,” purportedly with a “very high level of accuracy.” Erin Aslan, 13 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 14 Crime Gun Intelligence Centers: Using Technology and Intelligence as a Lead Generator to Identify Trigger-Pullers and Focus Enforcement and Prevention Efforts, DOJ J. Fed. L. & Prac., Nov. 2018, at 49, 53. The NIBIN system identified a potential link between the two Blazer cartridge cases; the images were then reviewed by a correlation review specialist, a peer reviewer, and an ATF firearm examiner before a NIBIN lead was generated, advising that further investigation was warranted. All four spent cartridge cases were then sent to the ATF laboratory, where Kong—who had been advised only that there was a NIBIN lead—conducted a microscopic comparison and concluded that “the probability that these cartridge cases were fired in a different firearm is so small that it is negligible.” R., Vol. I at 216. A peer reviewer reached the same conclusion. The district court denied Defendant’s motion to exclude, without a hearing, in a 17-page order that applied the Daubert factors to analyze the validity of firearm toolmark examination as a field and considered whether Jones and Kong reliably applied firearm-examination methods in this case. The court observed that the use of firearm toolmark examination in court is far from novel and that “no federal court has deemed such evidence wholly inadmissible.” Hunt, 464 F. Supp. 3d at 1256. It then analyzed the five Daubert factors. See id. at 1256–60. First, the district court found that “the theory of firearm toolmark identification can be and has been tested.” Hunt, 464 F. Supp. 3d at 1257. It cited a collection of studies compiled by the AFTE and noted that Defendant had presented no authority to the contrary. See id. at 1256–57. 14 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 15 Second, the court found that Daubert’s peer-review factor also favored admission, citing the AFTE Journal (a publication for firearm toolmark research), the PCAST and NRC Reports, and various studies that have been conducted on firearm toolmark examination. See id. at 1257–58. Third, the court found that the low error rate favored admissibility, citing the 1.52% false-positive rate reported in the Ames Study7 and “a Miami-Dade Study that reported a potential error rate of less than 1.2%.” Id. at 1258. The court observed that “[o]ther federal courts examining the AFTE method’s rate of error have likewise found it to be low” and noted that Defendant had not “introduce[d] any contradictory studies.” Id. On the other hand, the district court found that the “standards that control the [methodology’s application]” factor weighed against admissibility because the 7 In Defendant’s view the real error rate of the Ames Study was roughly 35%. His argument is that inconclusive determinations should be considered errors because “the samples were controlled to ensure that they bore sufficient markings and that these markings where [sic] not disturbed by environmental factors.” Aplt. Br. at 27. He cites a portion of the Ames Study that mentions that the fired cartridge cases were collected in a brass catcher and that cases that fell out of the catcher were discarded. See Ames Study at 11–12. But this is weak support for his proposition, considering that the Ames Study did not “prescreen[] the quality of samples provided to the participants” and actually sought to collect data on how many samples “had marks that were suitable for comparison.” Id. at 4. In any event, we find persuasive the government’s argument that the Ames Study’s false-positive rate furnishes the relevant error rate. That is because “a false positive identification . . . is the type of error that could lead to a conviction premised on faulty evidence.” United States v. Harris, 502 F. Supp. 3d 28, 39 (D.D.C. 2020). There is no harm to the defendant if a toolmark examiner makes an inconclusive finding, and Defendant presents no evidence to support his speculation that examiners will feel pressured to render conclusive opinions in the trial setting. 15 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 16 general AFTE method is “subjective in nature.” Id. at 1259 (internal quotation marks omitted). The district court concluded, however, that “the subjectivity of a methodology is not fatal under Rule 702 and Daubert.” Id. at 1260 (internal quotation marks omitted). Finally, with respect to the general-acceptance factor, the district court found that “the AFTE method used by the Government’s expert here[] is the field’s established standard” and that the lack of universal acceptance—citing the NRC and PCAST Reports—did not preclude admission in court. Id. at 1259–60 (internal quotation marks omitted). Despite the court’s weighing of the factors in favor of admissibility, it restricted how the expert’s opinion could be presented. It required that the testimony adhere to certain limitations set forth in guidance issued by the Department of Justice.8 See id. at 1261–62. The experts would need to “refrain from expressing their 8 The following limitations are set forth in the Department of Justice’s “Uniform Language for Testimony and Reports for the Forensic Firearms/Toolmarks Discipline – Pattern Match Examination,” R., Vol. I at 295: An examiner shall not assert that two toolmarks originated from the same source to the exclusion of all other sources. This may wrongly imply that a ‘source identification’ conclusion is based upon a statistically-derived or verified measurement or an actual comparison to all other toolmarks in the world, rather than an examiner’s expert opinion. An examiner shall not assert that examinations conducted in the forensic firearms/toolmarks discipline are infallible or have a zero error rate. An examiner shall not provide a conclusion that includes a statistic or numerical degree of probability except when based on relevant and appropriate data. 16 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 17 findings in terms of absolute certainty, and they [would] not state or imply that a particular bullet or shell casing could only have been discharged from a particular firearm to the exclusion of all other firearms in the world.” Id. at 1261. The court permitted the experts to testify only “that their conclusions were reached to a reasonable degree of ballistic certainty, a reasonable degree of certainty in the field of firearm toolmark identification, or any other version of that standard.”9 Id. at 1262. An examiner shall not cite the number of examinations conducted in the forensic firearms/toolmarks discipline performed in his or her career as a direct measure for the accuracy of a proffered conclusion. An examiner may cite the number of examinations conducted in the forensic firearms/toolmarks discipline performed in his or her career for the purpose of establishing, defending, or describing his or her qualifications or experience. An examiner shall not use the expressions ‘reasonable degree of scientific certainty,’ ‘reasonable scientific certainty,’ or similar assertions of reasonable certainty in either reports or testimony unless required to do so by a judge or applicable law. Id. at 297. Defendant concedes that “Kong was able to follow the letter of the [district court’s] order” limiting the testimony but complains that his testimony nonetheless “provided the false sense of certainty that the order was attempting to avoid.” Aplt. Br. at 33. Defense counsel did not, however, raise any objections during Kong’s testimony at trial. 9 Federal Rule of Evidence 702(d) currently requires that the proponent of expert testimony show that “the expert has reliably applied the principles and methods to the facts of the case.” The Advisory Committee on Evidence Rules has proposed amending Rule 702(d) to require a showing that “the expert’s opinion reflects a reliable application of the principles and methods to the facts of the case.” Committee on Rules of Practice and Procedure, Judicial Conference of the United States, Proposed Amendments to the Federal Rules of Appellate, Bankruptcy, Civil, and Criminal Procedure, and the Federal Rules of Evidence 308–09 (2021) (emphasis added). The committee note explains that “[f]orensic experts should avoid assertions of absolute or one hundred percent certainty—or to a reasonable degree of scientific certainty—if the methodology is subjective and thus potentially subject to error.” Id. at 311. 17 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 18 As relevant here, Defendant proceeded to trial on Counts 8 and 9. Kong testified at Defendant’s trial, but Jones did not. Kong outlined his training and experience, which included earning a bachelor’s degree in materials engineering, completing a one-year course at the ATF Firearms Examiner Academy, and receiving six to eight months of in-house training. Before he became an official firearms examiner, he was required to pass a competency test. In 2009, after having acquired five years of experience, he passed an examination required for certification by AFTE. AFTE certification lasts for five years but is subject to passing yearly proficiency tests. He has maintained his certification and was recertified in 2014 and 2019. Kong testified that he had 18 years of experience in firearms examination and that he had handled an average of 50 cases a year. Also, he has toured the facilities of over a dozen firearms manufacturers. Consistent with his written report (which had been submitted to Defendant and the court in response to the pretrial Daubert motion), Kong opined that the four spent cartridge cases found at the January and February 2019 crime scenes were fired from the same gun. The class characteristics of the cartridge cases “were all similar.” R., Vol. III at 467. With respect to individual characteristics, he explained that he had examined the striated firing-pin aperture shear marks that were created when the “head” of each of the four cartridge cases—that is, “[t]he base of the cartridge case which contains the primer,” AFTE Glossary at 32—scraped against the breech face of 18 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 19 the firearm upon discharge.10 Using a comparison microscope to look at the cartridges side by side, he found that the striations in the shear marks were in “sufficient agreement . . . to identify them as having been fired from the same gun.” R., Vol. III at 467. He also found that the firing-pin impressions—marks left by the firing pin upon striking the cartridge cases—were in “excellent agreement.” Id. at 468. In discussing the striated firing-pin aperture shear marks, Kong said that there were “somewhere in the neighborhood of 15 to 20 . . . consecutively matching stria[e].” Id. Earlier in his testimony he had mentioned a study conducted by Alfred Biasotti and John Murdock that examined the striated marks left on bullets fired from different firearms and concluded that the best agreement between two differentsource bullets was “maybe three or four consecutive matching striations,” that is, lines that are “consecutive and right next to each other.” Id. at 462. According to Kong, the study’s authors concluded that “if you have six consecutive stria[e], then that would signify a match.” Id. He thus opined that this was “not a borderline case in terms of identification.” Id. at 468. He said that he was able to reach his conclusion to “a reasonable degree of certainty within the firearms examination field.” Id. 10 Kong described the breech face as “the backstop or the area that contacts the base of the cartridge case. . . . [W]hen the cartridge fires, it pushes the case back against this breech face and picks up marks.” R., Vol. III at 453. The firing-pin aperture is “[t]he hole in the breech face of a firearm through which the firing pin protrudes.” AFTE Glossary at 52. Aperture shear marks are “[s]triated marks caused by the rough edges of the firing pin aperture scraping the primer metal during unlocking of the breech.” Id. at 52–53. 19 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 20 Kong also testified that, based on differences in class characteristics, the recovered cartridge casings could not have been fired by two firearms he was asked to examine, firearms that belonged to two of the men who accompanied Defendant during the January 2019 incident. The testimony at trial indicated that Defendant was the only other person—of the four men who confronted Brison—who could have carried a gun during that incident. Defense counsel did not conduct any crossexamination of Kong. Defendant was convicted on all counts that went to trial and sentenced to 960 months in prison. II. DISCUSSION We review for abuse of discretion the district court’s application of Daubert in denying Defendant’s motion to exclude the firearms-examination evidence. See Foust, 989 F.3d at 845. “We give the district court substantial deference, reversing only when its ruling was arbitrary, capricious, whimsical, or manifestly unreasonable or when it made a clear error of judgment or exceeded the bounds of permissible choice in the circumstances.” Id. (internal quotation marks omitted). But “we review de novo the question of whether the district court applied the proper standard [in admitting an expert’s testimony] and actually performed its gatekeeper role in the first instance.” Dodge v. Cotter Corp., 328 F.3d 1212, 1223 (10th Cir. 2003). This is because “[w]hile the district court has discretion in the manner in which it conducts its Daubert analysis, there is no discretion regarding the actual performance of the gatekeeper function.” Goebel v. Denver & Rio Grande W. R.R. Co., 215 F.3d 1083, 1087 (10th Cir. 2000) (emphasis in original). Thus, our de 20 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 21 novo review is limited to whether the district court properly followed the Daubert framework and performed an adequate inquiry into the relevance and reliability of the expert testimony. See Kumho Tire Co. v. Carmichael, 526 U.S. 137, 147 (1999); Dodge, 328 F.3d at 1221. “For purposes of appellate review, a natural requirement of the gatekeeping function is the creation of a sufficiently developed record in order to allow a determination of whether the district court properly applied the relevant law.” Adamscheck v. Am. Fam. Mut. Ins. Co., 818 F.3d 576, 586 (10th Cir. 2016) (brackets and internal quotation marks omitted). And the trial court is required to “reply in some meaningful way to the Daubert concerns the objector has raised.” StorageCraft Tech. Co. v. Kirby, 744 F.3d 1183, 1190 (10th Cir. 2014). A. The District Court’s Gatekeeping Role Defendant does not dispute that the district court cited “the [Daubert] factors and looked to them exclusively.” Aplt. Br. at 16. But he contends that the district court “procedurally erred in its application of Daubert” because it failed to conduct a diligent assessment of the available empirical evidence before making its ruling. Id. at 14. He faults the district court for taking “a short-cut around Daubert by relying on prior judicial decisions instead of conducting a meaningful review of the science.” Id.11 11 Defendant appears to argue that this critical assessment should have included a “probing inquiry,” Aplt. Br. at 26, into the methodologies used in the Miami-Dade and Ames Studies, and a thorough analysis of the peer-review procedure and publication practices of the ATFE Journal. But Defendant did not raise these specific concerns in his pretrial brief. As we have explained, the district court’s gatekeeping function is flexible, “requiring the court to focus its attention on the specific 21 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 22 Defendant finds some support in the beginning of the district court’s analysis: The use of this type of firearm toolmark identification in criminal trials is “hardly novel.” United States v. Taylor, 663 F. Supp. 2d 1170, 1175 (D.N.M. 2009). “For decades ... admission of the type of firearm identification testimony challenged by the defendant[ ] has been semiautomatic ....” United States v. Monteiro, 407 F. Supp. 2d 351, 364 (D. Mass. 2006); see also, e.g., United States v. Hicks, 389 F.3d 514 (5th Cir. 2004); United States v. Johnson, 875 F.3d 1265, 1281 (9th Cir. 2017). Indeed, no federal court has deemed such evidence wholly inadmissible. See United States v. Romero-Lobato, 379 F. Supp. 3d 1111, 1117 (D. Nev. 2019). Hunt, 464 F. Supp. 3d at 1256. If this was the court’s only analysis, and the court had rested its decision solely on other courts admitting firearm toolmark identification evidence, it may well have failed to perform its gatekeeping duty. But the district court continued, stating that “because of the seriousness of the criticisms launched against the methodology underlying firearms identification by Defendant in this case, the Court will carefully assess the reliability of this methodology, using Daubert as a guide.” Id. at 1256. The district court then worked through the Daubert factors, while considering the arguments presented in Defendant’s pretrial brief. See id. at 1256– 60. There is nothing improper in a court adopting the reasoning of a prior court. And the decision primarily relied upon by the district court to support its findings, United States v. Romero-Lobato, discussed the PCAST and NRC reports in depth. See 379 F. Supp. 3d 1111, 1117–18 (D. Nev. 2019), aff’d in relevant part, No. 20-10280, 2022 WL 2387214, at *1 (9th Cir. July 1, 2022) (unpublished). Although Defendant argues factors implicated by the circumstances at hand.” StorageCraft, 744 F.3d at 1190. The district court is not required to discuss issues that are not raised by the parties. 22 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 23 that “[t]he district court effectively took an end-run around the PCAST Report by relying on other decisions that either could not or did not consider the report,” Aplt. Reply Br. at 4, he ignores the district court’s discussion of the PCAST Report’s critiques that were raised by Defendant in his pretrial briefing, see Hunt, 464 F. Supp. 3d at 1257–58. Defendant may disagree with the manner in which the district court fulfilled its gatekeeping responsibilities, but because the district court analyzed the Daubert factors and addressed Defendant’s arguments, we conclude that the district court’s gatekeeping role was performed. B. Admissibility of the Expert’s Testimony Defendant argues that the district court abused its discretion in allowing Kong to testify regarding his firearm toolmark examination. He claims that “firearm toolmark examination methods are subjective, unproven, and not subject to meaningful review or acceptance outside the insular community of firearm toolmark examiners.” Aplt. Br. at 15. Although he challenges the reliability of the AFTE method, he does not challenge Kong’s credentials or whether he reliably applied the methodology to the facts of this case. In reviewing the district court’s ruling, we are not limited to its exposition supporting the ruling. Even if the exposition may be deficient in some respects, any shortcoming may be harmless error if the record contains the necessary support. As stated in StorageCraft, “If . . . it is readily apparent from the record that the expert testimony was admissible, it would be pointless to require a new trial at which the very same evidence can and will be presented again.” 744 F.3d at 1191. In particular, 23 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 24 we may evaluate the ruling in light of evidence presented at trial that was not presented at the Daubert hearing. See Kinser v. Gehl Co., 184 F.3d 1259, 1271 (10th Cir. 1998) (In reviewing the admissibility of expert testimony, “we do not think it necessary to confine our review to the materials accompanying the Daubert hearing request. Rather, we believe we may look at the entire record, including testimony presented at trial.”), abrogated in part on other grounds by Weisgram v. Marley Co., 528 U.S. 440 (2000); see also StorageCraft, 744 F.3d at 1191–92 (in response to an argument that the expert made an assumption unsupported by the evidence, the court held that any error was harmless because the assumption was supported by party’s deposition); cf. United States v. Harrison, 296 F.3d 994, 1003 (10th Cir. 2002) (“[T]o affirm the district court’s ruling, we may decide to consider all the evidence at trial, including evidence not presented at the hearing on the motion in limine [to admit evidence].”); United States v. Corral, 970 F.2d 719, 723 (10th Cir. 1992) (“In evaluating the correctness of the district court’s rulings, the appellate court may consider the entire record developed from the trial even though such evidence may not have been presented during the suppression hearing.”). Our concern is the breadth of the district court’s ruling. It can be read as upholding AFTE methodology in general. This would hardly be a remarkable ruling. Both before and after the 2016 publication of the PCAST Report, other circuits have affirmed decisions to admit expert testimony grounded in the AFTE method. See United States v. Brown, 973 F.3d 667, 702–04 (7th Cir. 2020); United States v. Johnson, 875 F.3d 1265, 1281 (9th Cir. 2017); United States v. Williams, 506 F.3d 24 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 25 151, 157–62 (2d Cir. 2007). But in light of the critiques expressed in the PCAST and NRC Reports, we think courts should be cautious, and our holding should go no further than necessary. “Our task is not to determine the admissibility or inadmissibility of [firearm toolmark examination] for all cases but merely to decide whether, on this record, the district judge in this case made a permissible choice in exercising [his] discretion to admit the expert testimony.” United States v. Baines, 573 F.3d 979, 989 (10th Cir. 2009). We therefore restrict our consideration to the specific methodology described by expert Kong at trial. To determine that all four cartridges were fired by the same weapon, Kong employed the consecutive-matching-striae (CMS) method to threedimensional images of the cartridges. As we explain below, the CMS method has impressive empirical support that would plainly permit a court to find it reliable. The rub in this case is that the district court’s Order expressing its ruling on the Daubert motion does not describe that method or the technical literature supporting it. But that is hardly surprising. Until Kong testified, nothing in the record would have informed the court about the specific identification method used by Kong.12 Although the technical literature referenced by the government in its Daubert brief and by the court in its Order says a good deal about the CMS method, there was no reason for 12 The government first identified Kong as an expert in its response to Defendant’s motion to exclude. Defendant filed no reply challenging Kong’s qualifications or expertise. At trial Hunt did not cross-examine Kong or raise any objection to his testimony. He thereby forfeited any challenge that might derive from the failure to disclose this methodology before trial. Nor has Hunt raised such a challenge on appeal. 25 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 26 the Order to focus on that method as opposed to general AFTE methods. But if the court had been called on at trial to opine on this specific subset of AFTE methodology, there can be little doubt that the court, having already declared that the general AFTE methodology is reliable, would have found that the CMS method, with its extensive empirical support, is a fortiori reliable. Hence, any error in the court’s not specifically addressing the CMS method is harmless. We now turn to a discussion of that method and how it would affect the district court’s Daubert analysis. In applying the CMS method, the examiner aligns the objects being compared and counts the number of consecutive striae where the width, morphology, and relative position match exactly. Kong testified that finding six or more consecutive striae that match is sufficient to determine that two cartridges were fired from the same weapon. He also said that each additional consecutive matching stria further reduces the likelihood that two cartridges were fired from different weapons. Thus, because the four cartridge cases here had 15 to 20 consecutive matching striae, it was “not a borderline case in terms of identification.” R., Vol. III at 468. Kong said that the criterion he used was based on a study done some time ago by Biasotti and Murdock. And what they did was they looked at the best agreement and bullets that were fired from different firearms, and the best they came up with was, like, maybe three or four consecutive matching striations, and that’s the absolute best. So what they did was they proposed that if you have six consecutive strias, then that would signify a match. Id. at 462. He further testified that if two bullets (or cartridges) fired from different guns had striae that satisfied the criterion for identifying the bullets as coming from 26 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 27 the same weapon, firearms examiners would learn of the discovery, yet he had never heard of such a discovery. The criterion used by Kong is supported by the research of Biasotti and Murdock that he mentioned in his testimony and by several studies referenced in the government’s pretrial Daubert brief and in the district court’s Order. Biasotti’s original study, first published in 1959, examined 244 bullets and analyzed 1,200 known match comparisons and 1,080 known nonmatch comparisons. It found empirical support for the proposition that a significant number of consecutive matching striae will appear only on bullets fired from the same gun.13 See Biasotti, A Statistical Study of the Individual Characteristics, supra, at 35–36; Alfred Biasotti, John Murdock & Bruce R. Moran, Development of Objective Criteria for Identification, in 4 Modern Scientific Evidence: The Law and Science of Expert Testimony § 34:13, at 1013–18 (David L. Faigman et al. eds., 2021). Biasotti and Murdock continued to research consecutive matching striae in toolmarks, and in 1997 they formulated their “conservative quantitative criteria for identification” (the Biasotti-Murdock criteria). Jerry Miller, An Examination of the Application of the Conservative Criteria for Identification of Striated Toolmarks Using Bullets Fired from Ten Consecutively Rifled Barrels, 33 AFTE J. 125, 126 (2001); see Biasotti, 13 Before this research some experts looked to the total percent of matching striae (regardless of whether they were consecutive) between the two bullets to determine a match. But Biasotti concluded from his data that the total percent of matching striae was an unreliable basis for identification. See Alfred Biasotti, A Statistical Study of the Individual Characteristics of Fired Bullets, 4 J. Forensic Sci. 34, 37–39 (1959). 27 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 28 Murdock & Moran, Development of Objective Criteria, supra, at 1018–20. Under their criteria a match is found for three-dimensional striae “when at least two different groups of at least three consecutive matching striae appear in the same relative position, or one group of six consecutive matching striae are in agreement.”14 Michael Neel & Major Wells, A Comprehensive Statistical Analysis of Striated Tool Mark Examinations Part 1: Comparing Known Matches and Known Non-Matches, 39 AFTE J. 176, 177 (2007). But to apply the criteria, the toolmark examiner must rule out the influence of subclass characteristics.15 14 Biasotti and Murdock also formulated a criterion for examining twodimensional toolmarks. Two-dimensional toolmarks, also sometimes called lines, include “[a]ny impressed or striated toolmark that lacks apparent depth.” Neel & Wells, A Comprehensive Statistical Analysis of Striated Tool Mark Examinations, supra, at 177.) Under the two-dimensional criterion, a match occurs “when at least two groups of at least five consecutive matching striae appear in the same relative position, or one group of eight consecutive matching striae are in agreement in an evidence toolmark compared to a test toolmark.” Id. Biasotti’s original research examined two-dimensional marks. 15 Kong’s notes from his examination of the cartridge cases, which were submitted to the court, discussed his elimination of subclass characteristics: Aperture shear marks: The aperture shapes are similar to a “teardrop[,]” which means there is a ramp at the 6 o’clock position. The aperture shear marks had excellent correspondence, and the agreement was sufficient for identification. Aperture shear marks were generated when the primer of the cartridge case rubbed on the edge of the breech face at the ramp. The edge is the intersecting of these two differently machined surfaces and will not have subclass potential. The ramp was produced by a rotating cutting tool to a tilted breech, therefore producing toolmarks that are perpendicular to the [cartridge cases’] movement during firing; so no potential subclass marks are from the ramp. Firing pin marks: Excellent correspondence of a series of marks observed on the firing pin (FP) impression. A line was visible running from the 6 to 12 o’clock in the center of the FP impression, and 28 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 29 The Biasotti-Murdock criteria, the basis of the CMS method, are supported in several ways by technical papers referenced in the government’s Daubert brief and the district court’s Order. One 2007 empirical study looked at “4188 striated toolmark comparisons from a variety of sources”16 and found no known nonmatches that met the Biasotti-Murdock criteria. Id. at 179, 180. In particular, the study found no more than four consecutive matching three-dimensional striae in a known nonmatch—two less than the six required under the Biasotti-Murdock criterion. The study pointed out that “[t]here has been over 50 years of research regarding CMS run counts, beginning with Biasotti’s thesis in 1955, and no documented 2D or 3D comparisons have been shown to contradict the original criteria set forth by Biasotti and Murdock.” Id. at 190. The Biasotti-Murdock criteria are also supported by a theoretical analysis that calculated the probabilities of consecutive matches of two-dimensional striae appeared to be a mold line from castings or metal injection molding (MIM) parts. Mold surfaces can have features carryover from part to part. However, the firing pin appeared to have a defect that produced the observed corresponding marks, and such a defect could be unique, but can have subclass potential. If the mold has a defect, then there is subclass. R., Vol. I at 272. 16 The sources for this study included “fired cartridge cases, fired bullets, sandpaper of various grit sizes (60, 22, 320) used to scratch emulsion based film, chisels slid across lead foil, photomicrographs of plastic replicas taken from consecutively rifled barrels, fired bullets from cut sections of a Thompson Contender barrel, and fired bullets from consecutively rifled Bar-Sto barrels.” Neel & Wells, A Comprehensive Statistical Analysis of Striated Tool Mark Examinations, supra, at 179. 29 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 30 occurring by chance. See David Howitt, et al., A Calculation of the Theoretical Significance of Matched Bullets, 53 J. Forensic Sci. 868 (2008). The researchers found a “close similarity” between Biasotti’s 1959 experimental observations and their calculated probabilities. Id. 872–73. Based on their calculations, the probability of the random creation of eight consecutive matching two-dimensional striae (the Biasotti-Murdock two-dimensional criterion) would be less than one in a hundred million. In addition, the Biasotti-Murdock criteria have been successfully tested in an automated identification system. See Wei Chu, et al., Automatic Identification of Bullet Signatures Based on Consecutive Catching Striae (CMS) Criteria, 231 Forensic Sci. Int’l 137 (2013). The system minimized subjective factors by using a computer program objectively applying the three-dimensional Biasotti-Murdock criterion. It correctly identified 29 of 30 matching bullet pairs from the “unknown” test set and found no false positives in 12,960 known nonmatch comparisons. To be sure, one can find literature criticizing the CMS method. Mr. Hunt’s reply brief cites a 2005 article asserting that striae counting is inherently subjective and that “the CMS approach fails to place firearms and toolmark identification on adequate statistical empirical foundations.” Schwartz, A Systemic Challenge, supra, at 21. But even that article recognizes advantages of CMS identification as compared to the general AFTE methodology. Id. at 15 (“CMS differs from and is scientifically superior to the subjective approach because it is interpretable in a way that is compatible with the probabilistic nature of identity claims.”). And more importantly, 30 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 31 articles discussed above that were published after 2005 undercut Schwartz’s concerns. The article by Chu, et al., shows that CMS can be applied objectively; and the article by Howitt, et al., provides theoretical statistical support for CMS. We also think it significant (1) that the CMS method is not mentioned in the PCAST report, which questions only AFTE methods in general and (2) that the 2009 NRC Report suggests that its criticism of the lack of studies supporting firearms and toolmark identification in general may not apply to CMS: “Recent research has attempted to develop a statistical foundation for assessing the likelihood that more than one tool could have made specific marks by assessing consecutive matching striae, but this approach is used in a minority of cases.” NRC Report at 154 n.63. We now turn to an examination of the district court’s Daubert analysis as supplemented by specific information on the reliability of the CMS method. Regarding the first Daubert factor, the district court found that “the theory of firearm toolmark identification can be and has been tested.” Hunt, 464 F. Supp. 3d at 1257. Discussion of the CMS method would only have reinforced that conclusion. By providing quantitative criteria for identification, the CMS method is readily testable and has been tested. As discussed above, the CMS method has been empirically tested for over 60 years and no nonmatches have been found that meet the BiasottiMurdock criteria. See Neel & Wells, A Comprehensive Statistical Analysis of Striated Tool Mark Examinations, supra, at 190; see also Biasotti, Murdock & Moran, Development of Objective Criteria, supra, at 1021–23 & n.26 (summarizing the empirical studies evaluating Biasotti-Murdock criteria and concluding that “[n]o 31 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 32 known non-matching . . . toolmarks were found . . . which exhibited agreement in excess of the proposed Biasotti-Murdock criteria.”). The district court also determined that the second Daubert factor—whether the technique has been subjected to peer review and publication—weighed in favor of admissibility. See Hunt, 464 F. Supp. 3d at 1257–58. Again, that determination finds particular support with respect to the CMS method. Biasotti’s original 1959 article was published in the Journal of Forensic Science, which is peer-reviewed. See Biasotti, Murdock & Moran, Development of Objective Criteria, supra, at 1013 & n.5. And the other studies discussed above were each published in the AFTE Journal, the Journal of Forensic Science, or Forensic Science International. The studies supporting the CMS methodology were subject to the same peer-review process as the articles addressed by the district court in its Order. This factor would only be strengthened in support of reliability. The third Daubert factor is whether the technique has a known or potential rate of error. The district court, noting the error rates from the Ames Study (which, as here, involved cartridge-case comparisons) and the Miami-Dade Study, weighed this factor in favor of admissibility. See Hunt, 464 F. Supp. 3d at 1258.17 For the Biasotti- 17 We add that at least one “post-PCAST Report study . . . followed the PCAST recommended black-box model and found that of 1512 possible identifications tested, firearms examiners correctly identified 1508 casings to the firearm from which the casing was fired.” Harris, 502 F. Supp. 3d at 38 (citing Mark A. Keisler et al., Isolated Pairs Research Study, 50 AFTE J. 56, 58 (2018)). Notably, “[n]o false positive . . . results were reported.” Keisler et al., Isolated Pairs Research Study, supra, at 57. 32 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 33 Murdock criteria, there is not a known error rate from empirical testing because no false match has been found. But the theoretical analysis discussed above suggests that the probability of two random bullets satisfying the Biasotti-Murdock criteria would be infinitesimal. Again, consideration of the CMS method would only have increased the support of this factor in favor of reliability. The district court weighed the fourth factor—whether there are standards that control the technique’s operation—against admissibility. See Hunt, 464 F. Supp. 3d at 1259. But even if the court’s assessment is correct with respect to toolmark examination in general, the Biasotti-Murdock criteria provide specific objective standards that control the application of the CMS method. See Romero-Lobato, 379 F. Supp. 3d at 1121 (“The CMS method, standing alone, qualifies as an objective standard under Daubert.”). Although the determination of subclass features and deciding whether two striae match may involve some subjectivity, the toolmark examiner’s discretion is constrained: “CMS is defined as striated markings that ‘line up’ exactly (close doesn’t count) with one another without a break or dissimilarity in between them.” Biasotti, Murdock & Moran, Development of Objective Criteria, supra, at 1015–16 n.8. Finally, the district court found that the general-acceptance factor favored admission because the AFTE method is widely used by firearms examiners. See Hunt, 464 F. Supp. 3d at 1259–60; see also Brown, 973 F.3d at 704 (district court observed that “firearm and toolmark analysis is widely accepted beyond the judicial system”). Like the other factors, consideration of the CMS method would have 33 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 34 further supported this conclusion. Although the CMS method might not be “routinely used by firearm examiners across the country,” Romero-Lobato, 379 F. Supp. 3d at 1122, this may be because “many firearms examiners consider [the BiasottiMurdock criteria] to be too stringent,” United States v. Taylor, 663 F. Supp. 2d 1170, 1178 (D.N.M. 2009). This feature of the CMS method—missing some identifications through the application of stringent criteria that minimize the possibility of a false positive—enhances the reliability of CMS-derived identifications. There is no reason to doubt that CMS identifications are generally accepted, at least within the community of firearms examiners. See Foust, 989 F.3d at 846 (“Although . . . acceptance by unbiased experts is always better, that does not mean this factor cannot support admission.”); Baines, 573 F.3d at 991 (similar).18 We conclude that even if the district court’s opinion is inadequate to support the AFTE methodology for firearm toolmark examination in general, any shortcoming was harmless because the specific CMS method relied on by Kong has compelling support in his testimony and the technical literature referenced by the district court. And Kong’s opinion in this case seems particularly resistant to criticism given the large number of consecutive matching striae and his expertise and experience in applying the methodology. 18 We note that one of the most critical judicial assessments of firearm toolmark identification—repeatedly cited by Defendant—did not consider the CMS method. See generally United States v. Tibbs, No. 2016 CF1 19431, 2019 WL 4359486 (D.C. Super. Ct. Sept. 5, 2019). 34 Appellate Case: 21-6046 C. Document: 010110832100 Date Filed: 03/24/2023 Page: 35 Response to the Concurrence The Concurrence complains that the majority opinion improperly goes outside the record and violates the principle of party presentation. Both complaints are unwarranted. First, the record. The record consists of those documents (broadly construing the term to include electronically stored matters, such as video or audio recordings) filed with the court. The scientific and technical papers referenced in the majority opinion that the concurrence complains were not in the record were summarized on a website referenced in the district court’s opinion and hyperlinked in the government’s memorandum filed with the court for the Daubert hearing. The only relevance of the referenced website was the studies it collected. Under any reasonable notion of record, those papers were part of the district-court record.19 They would surely have been part of the record if they were attached as hard copies to the government’s brief. Is such useless printing to be required by parties in the future? What purpose is advanced by not considering those papers as part of the record? The Concurrence suggests that maybe the papers were part of the record but only for a limited purpose. It says that the reference to the studies by the government was just to establish that such studies existed, “not their quality or content.” 19 That was certainly the view of defense counsel at oral argument, who described the website articles as “evidence.” A few seconds into his argument, counsel noted the citation to the website in the district court’s opinion and suggested that the opinion did not contain a sufficient analysis of the studies themselves, arguing that “[t]he court wasn’t conducting a review of the evidence by merely looking at that website.” Oral Argument at 1:22–1:27. 35 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 36 Concurrence at 1. But surely a court’s role in performing a Daubert analysis is not limited to looking at titles and counting how many there are. What sort of a gatekeeper is that? The court must examine contested studies to protect the system from junk science and self-serving ipse dixits by advocates for particular theories. The opposing party has every right, and every incentive, to challenge the quality of the studies proffered by the offering party and the relevance of their content to the issues before the court. Here, Defendant had retained his own ballistics expert, although the expert was never called to testify. The fact that the opposing party made no such challenge hardly means that the quality and content of the studies are not part of the record. A great deal of the record in every case goes unchallenged. If the district court had in fact read some of the referenced papers to assist it in ruling on the Daubert motion (which is not entirely implausible since the district-court opinion notes that the court visited the site two weeks before filing the opinion, see 464 F. Supp.3d at 1257), would we be required to reverse the court’s decision because it went “outside the record” in reaching its decision? Insisting on a cramped notion of what is part of the record is particularly inappropriate in the present context. The studies at issue do not address adjudicative facts peculiar to the specific case before us. They concern legislative facts that are applicable to a great many, and wide range of, cases; the reliability of firearm toolmark examination must be assessed to determine the admissibility of evidence of such examinations. When the resolution of a dispute turns on legislative facts, courts regularly relax the restrictions on judicial inquiry. For example, while appellate 36 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 37 courts can take judicial notice of adjudicative facts only in limited circumstances, see Fed. R. Evid. 201; Winzler v. Toyota Motor Sales U.S.A., Inc., 681 F.3d 1208, 1212–13 (10th Cir. 2012) (Gorsuch, J.) (appellate court may take judicial notice of facts “at any stage of the proceeding if the facts are not subject to reasonable dispute” (internal quotation marks omitted)), there are no such strict limits with respect to legislative facts, see Fed. R. Evid. 201 advisory committee note (“‘In determining the content or applicability of a rule of domestic law, the judge is unrestricted in his investigation and conclusion. He may reject the propositions of either party or both parties. He may consult the sources of pertinent data to which they refer, or he may refuse to do so. He may make an independent search for persuasive data or rest content with what he has or what the parties present.’” (quoting Edmund M. Morgan, Judicial Notice, 57 Harv. L. Rev. 269, 260–71 (1944))); Edward K. Cheng, Independent Judicial Research in the Daubert Age, 56 Duke L.J. 1263, 1293 (2007) [hereinafter Cheng] (“If one takes the Advisory Committee’s adoption of the Morgan view seriously, this conclusion [that the scientific facts used for Daubert determinations should be treated as legislative facts] means that the Federal Rules free judges to do independent research in the Daubert context.”). As noted by Prof. Frederick Schauer in The Decline of “The Record”: A Comment on Posner, 51 Duq. L. Rev. 51 (2013) [hereinafter Schauer], courts sometimes rely on nothing more than the personal intuitions of its members (as when it selected the burden of persuasion for libel actions based on how it thought the press would react to different standards, see New York Times v. Sullivan, 376 U.S. 254, 282 (1964); Schauer at 57–58, 63), 37 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 38 and they sometimes rely on their review of sources never presented to the court (as with the per curiam majority in Bush v. Gore, 531 U.S. 98, 103 (2000), and Justices Stevens and Breyer on multiple occasions, including for the court in Kumho Tire Co. v. Carmichael, 526 U.S. 137, 142–43 (1999); see Schauer at 51 n.3, 56). The propriety of this independent research is somewhat controversial, see, e.g., Schauer at 51–52; Cheng at 1280–84, but when the highest court in the land uses non-record materials to resolve legislative facts, it is hard to justify barring appellate consideration of materials that were referenced in the district court and were subject to prior examination and dispute by all the parties. The Concurrence also misapplies the party-presentation principal. As we recently explained, that principle generally restricts the court from raising issues sua sponte, but it does not preclude a court from resolving a presented issue in a way not proposed by any party. See United States v. Cortez-Nieto, 43 F.4th 1034, 1052 (10th Cir. 2022). For example, what if each party in a contract dispute argues that a provision of a contract is unambiguous but they disagree on what that unambiguous meaning is? It would be wholly proper for the court to decide that the provision has a third meaning or that the provision is ambiguous and further factual development is necessary. Here, the government argues that AFTE methodology—as a whole—is reliable (so admissibility should turn only on the expertise of the examiner and the application of the methodology), while Defendant argues that the methodology is never reliable enough to be used in court. In reviewing the issue, we are not limited 38 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 39 to choosing between those alternatives. We can, and do, reserve ruling on the reliability of the AFTE methodology in general, largely because of the prestige of the government-sponsored reports questioning whether the accuracy of the methodology has been sufficiently tested. On the other hand, there has been considerable testing, over an extended period of time, of the accuracy of one subset of AFTE methodology—the CMS methodology used in this case. We therefore can affirm the conviction here without going further and addressing the more general question of reliability of AFTE methodology. Given that liberty is at stake in this case, and in most cases where expert firearm toolmark testimony is offered, we are reluctant to go beyond what is necessary in this case and provide a judicial imprimatur to AFTE methodology in general. Although the concurrence believes that the majority opinion takes appellate review beyond its proper sphere, we think our approach is fully consonant with traditional appellate practice and, in the circumstances, not only prudent but less adventurous than full endorsement of the district-court ruling. We are particularly reluctant to adopt the ad hoc approach of the Concurrence, which would admit the testimony in this case while reserving judgment on admissibility in other cases, without providing needed guidance to the lower courts. Of course, appellate judges must be particularly careful in vetting scientific or technical research that was not debated in district court. But nothing precluded Defendant, who had retained his own ballistics expert, from challenging the cited literature. In any event, it is not uncommon for an appellate court to perform an 39 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 40 independent Daubert analysis, as when it is assessing whether a district court’s failure to perform the analysis was harmless error. See Smith v. Jenkins, 732 F.3d 51, 64–68 (1st Cir. 2013); Sarkees v. E. I. Dupont De Nemours & Co., 15 F.4th 584, 589–93 (2d Cir. 2021); UGI Sunbury LLC v. A Permanent Easement for 1.7575 Acres, 949 F.3d 825, 833–36 (3d Cir. 2020); Sardis v. Overhead Door Corp., 10 F.4th 268, 285–96 (4th Cir. 2021); Anderson v. Raymond Corp., 59 F.4th 279, 283– 85 (7th Cir. 2023); United States v. Ruvalcaba-Garcia, 923 F.3d 1183, 1190–91 (9th Cir. 2019). This court explicitly approved that practice in StorageCraft Technology Corp. v. Kirby, 744 F.3d 1183, 1190–92 & n.2 (10th Cir 2014) (Gorsuch, J.). What the majority opinion does in this case is significantly less comprehensive. We have merely supplemented the district court’s analysis from the Daubert hearing with the new information provided at trial. As previously noted, it is accepted practice for appellate courts to affirm a ruling on a pretrial motion in limine after considering what happened at trial. See, e.g., Kinser, 184 F.3d at 1271 (In reviewing the admissibility of expert testimony, “we do not think it necessary to confine our review to the materials accompanying the Daubert hearing request. Rather, we believe we may look at the entire record, including testimony presented at trial.”). At trial, expert witness Kong reported for the first time his use of the CMS methodology in comparing the cartridge shells and he described the leading study supporting that methodology. If there were flaws in that methodology or the studies, Defendant could have raised challenges; but he did not even cross-examine the expert. The concurrence complains that neither the parties nor the district court 40 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 41 discussed or analyzed the studies relevant to the CMS technique that were described on the AFTE website referenced by both the government and the court. But when Defendant raises no challenge to those studies or the methodology, we can properly assume their validity. Our approach, which the concurrence disapproves, has been simply to conduct our own examination of the relevant studies to confirm that the CMS methodology is not subject to the criticisms of firearm toolmark methods in general which have been made by distinguished panels of forensic experts. We see that as our duty, not a usurpation of power. In short, we determine that the support the district court found for the general AFTE methodology is particularly strong with respect to the specific CMS methodology used in this case, making it unnecessary to resolve the reliability of the general methodology. We recognize that our standard of review under Daubert is abuse of discretion. But the issues are much too important to be resolved by a simple wave of the hand and deference to the decision below. Respect for the courts is significantly reduced when litigation is resolved on the basis of pseudoscience that is rejected by highly educated and intelligent citizens who begin to think that the overlap between truth and judicial judgment is too slim. Appellate judges have an obligation to educate themselves and engage in the sometimes difficult work of assessing scientific and technical material. At least in this case, the analysis is not particularly difficult because of the extensive testing of the CMS methodology. 41 Appellate Case: 21-6046 III. Document: 010110832100 CONCLUSION We AFFIRM Defendant’s convictions. 42 Date Filed: 03/24/2023 Page: 42 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 43 United States v. Hunt, 21-6046 MORITZ, J., concurring. I agree that the district court did not abuse its discretion when it deemed Howard Kong’s toolmark-identification methodology reliable under Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993). But unlike the majority, I would reach that conclusion without supplementing the district court’s analysis or engaging in an independent review of studies and literature about consecutive matching striae (CMS). Initially, I can’t join the majority’s reliance on and discussion of the CMS studies because those studies are not part of the record in this case. Cf. United States v. Kennedy, 225 F.3d 1187, 1191 (10th Cir. 2000) (“This court will not consider material outside the record before the district court.”). To be sure, citations to most of the studies discussed by the majority appear on a website that the government cited once in its response to the motion to exclude and that the district court cited once in its order. This website, published by the Association of Firearm and Tool Mark Examiners (AFTE), offers “literature citations for the more important studies that qualify as material principally concerned with the validity of firearm and toolmark identification,” including “[a] short summary follow[ing] each citation.” Testability of the Scientific Principle, The Association of Firearm and Tool Mark Examiners, https://afte.org/resources/swggunark/testability-of-the-scientific-principle (last visited March 23, 2023). But the government’s and district court’s citations merely supported the existence of the studies listed on the website, not their quality or content. The government cited the Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 44 website to support its assertion that “[t]he AFTE theory is regularly tested both on an individual level, by peer review and verification, and on a larger level with numerous studies.” R. vol. 1, 192. Likewise, the district court cited the website to support the conclusion that “the theory of firearm toolmark identification can be and has been tested.” United States v. Hunt, 464 F. Supp. 3d 1252, 1257 (W.D. Okla. 2020). Thus, contrary to the majority’s characterization—that Kong’s CMS methodology is supported “by several studies referenced in the government’s pretrial Daubert brief and in the district court’s [o]rder”—neither the government nor the district court “referenced” any of the studies the majority relies on. Maj. Op. 27. Nor are the government’s and district court’s passing citations to the AFTE website sufficient to incorporate any or all of the listed studies—over 100 studies grouped into six topics—into the record for purposes of appeal. The majority cites no authority to support its decision to do so, and I have found none.1 Although a reviewing court may sometimes affirm the admission of expert testimony despite possibly flawed reasoning by the district court, such harmlessness must be “readily apparent from the record.” StorageCraft Tech. Corp. v. Kirby, 744 F.3d 1183, 1190–91 (10th Cir. 2014) (emphasis added); see also Kinser v. Gehl Co., 184 F.3d 1259, 1271 (10th Cir. 1999). 1 The majority also fails to explain whether it reviewed all the website’s listed studies or how it chose the handful that it relies on. Nevertheless, because I would not consider such studies at all, I will not substantively engage with the majority’s assessment of the literature. 2 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 45 Because the studies analyzed, discussed, and applied by the majority are not part of the record, I can’t join the majority’s decision to rely upon them.2 Moreover, and perhaps more critically, even if these studies are part of the record, neither the district court nor the parties ever mentioned such studies in their briefing below, let alone discussed or analyzed those studies. As a result, the court best positioned to evaluate the CMS literature and its reliability (the district court) had no opportunity to consider it. See Goebel v. Denver & Rio Grande W. R.R. Co., 215 F.3d 1083, 1088–89 (10th Cir. 2000) (“Performance of the gatekeeping function on the record [e]nsures that a judgment in favor of either party factors in the need for reliable and relevant scientific evidence. It is not an empty exercise; appellate courts are not well-suited to exercising the discretion reserved to district courts.”). And the same is true on appeal: We lack the benefit of the parties’ positions on this literature and the reliability of the CMS method. Thus, the majority’s reasoning—which turns heavily, if not entirely, on its “own examination” of CMS literature that is included amongst a variety of other toolmarkidentification studies on an AFTE website, Maj. Op. 41—significantly departs from “the principle of party presentation.” United States v. Sineneng-Smith, 140 S. Ct. 1575, 1578 (2020). This principle “restricts courts from raising new issues,” yet the majority here does exactly that when it affirms the district court’s admission of Kong’s expert 2 The majority also appears to rely on Kong’s brief mention during his trial testimony of “a study done some time ago by Biasotti and Murdock.” Maj. Op. 26 (quoting R. vol. 3, 462). But this passing mention doesn’t meaningfully incorporate the majority’s swath of scientific literature into the appellate record. Additionally, the study Kong mentioned doesn’t appear to be included on the AFTE website, which lists one study by Biasotti and two by Murdock, but none by both individuals. 3 Appellate Case: 21-6046 Document: 010110832100 Date Filed: 03/24/2023 Page: 46 testimony based on a theory of CMS reliability that no party has either “raised” or “responded to.” United States v. Cortez-Nieto, 43 F.4th 1034, 1052 (10th Cir. 2022) (emphasis omitted). In so doing, the majority does far more than “supplement[] the district court’s analysis”; it ignores the district court’s analysis and conducts its own. Maj. Op. 40. Because of these concerns, I would follow the parties’ lead and analyze the district court’s actual application of the Daubert factors. Doing so, I’m not persuaded that the district court erred; “nothing in the controlling legal authority” requires the “extremely high degree of intellectual purity” that Dominic Hunt presses on appeal. United States v. Baines, 573 F.3d 979, 989 (10th Cir. 2009). To be sure, neither this concurrence nor the majority opinion should be read to endorse the rote admission of toolmark-identification experts. See United States v. Smith, 756 F.3d 1179, 1193 n.13 (10th Cir. 2014) (“Firearm toolmark analysis has recently come under attack for depending on subjective judgment, rarely using control weapons, and risking an observer effect.”). But I see no abuse of discretion in the district court’s decision in this case. See United States v. Foust, 989 F.3d 842, 847 (10th Cir. 2021) (finding no abuse of discretion in admitting handwriting expert despite “criticism of handwriting expertise in both the courts and academic literature”). I accordingly concur. 4
Some case metadata and case summaries were written with the help of AI, which can produce inaccuracies. You should read the full case before relying on it for legal research purposes.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
