Kohls et al v. Ellison et al, No. 0:2024cv03754 - Document 46 (D. Minn. 2025)
Court Description: ORDER granting in part and denying in part 29 Plaintiffs' Motion to Exclude Expert Testimony and denying 34 Defendant's Motion for Leave to File an Amended Expert Declaration (Written Opinion) Signed by Judge Laura M. Provinzino on 1/10/2025. (TJC)
Download PDF
<![CDATA[
Kohls et al v. Ellison et al Doc. 46 UNITED STATES DISTRICT COURT DISTRICT OF MINNESOTA CHRISTOPHER KOHLS and MARY FRANSON, Case No. 24-cv-3754 (LMP/DLM) Plaintiffs, ORDER GRANTING IN PART AND DENYING IN PART PLAINTIFFS’ MOTION TO EXCLUDE EXPERT TESTIMONY AND DENYING DEFENDANT’S MOTION FOR LEAVE TO FILE AN AMENDED EXPERT DECLARATION v. KEITH ELLISON, in his official capacity as Attorney General of Minnesota, and CHAD LARSON, in his official capacity as County Attorney of Douglas County, Defendants. M. Frank Bednarz, Hamilton Lincoln Law Institute, Chicago, IL; and Douglas P. Seaton, James V. F. Dickey, and Alexandra K. Howell, Upper Midwest Law Center, Minnetonka, MN, for Plaintiffs. Elizabeth C. Kramer, Peter J. Farrell, Angela Behrens, and Allen C. Barr, Office of the Minnesota Attorney General, Saint Paul, MN, for Defendant Keith Ellison, in his official capacity as Attorney General of Minnesota. Kristin C. Nierengarten and Zachary J. Cronen, Rupp, Anderson, Squires & Waldspurger, Minneapolis, MN, for Defendant Chad Larson, in his official capacity as County Attorney of Douglas County. Plaintiffs Christopher Kohls and Mary Franson (collectively, “Plaintiffs”) move to exclude expert declarations offered by Defendant Keith Ellison, in his official capacity as Attorney General of Minnesota (“Attorney General Ellison”), in opposition to Plaintiffs’ motion for a preliminary injunction. ECF No. 29. Recognizing errors in one of the declarations, Attorney General Ellison has moved for leave to file an amended declaration 1 Dockets.Justia.com by that expert. ECF No. 34. For the following reasons, Plaintiffs’ motion is granted in part and denied in part, and Attorney General Ellison’s motion is denied. FACTUAL BACKGROUND Minnesota law prohibits, under certain circumstances, the dissemination of “deepfakes” with the intent to injure a political candidate or influence the result of an election. See Minn. Stat. § 609.771. Plaintiffs challenge the statute on First Amendment grounds and seek preliminary injunctive relief prohibiting its enforcement. See ECF No. 10. With his responsive memorandum in opposition to Plaintiffs’ preliminary-injunction motion, Attorney General Ellison submitted two expert declarations: one from Jevin West, a Professor at the University of Washington who also serves as the Director of the Center for an Informed Public, an interdisciplinary research center dedicated to studying misinformation in the digital age (ECF No. 24 (the “West Declaration”)); and another from Jeff Hancock, Professor of Communication at Stanford University and Director of the Stanford Social Media Lab (ECF No. 23 (the “Hancock Declaration”)). The declarations generally offer background about artificial intelligence (“AI”), deepfakes, and the dangers of deepfakes to free speech and democracy. ECF No. 23 ¶¶ 7–32; ECF No. 24 ¶¶ 7–23. Plaintiffs moved to exclude these declarations, arguing that they are conclusory and contradicted by the experts’ prior writings. See ECF No. 30 at 9–34. Plaintiffs also alleged that Professor Hancock included fabricated material in his declaration. Id. at 4–9. After reviewing Plaintiffs’ motion to exclude, Attorney General Ellison’s office contacted 2 Professor Hancock, who subsequently admitted that his declaration inadvertently included citations to two non-existent academic articles, and incorrectly cited the authors of a third article. ECF No. 37 at 3–4. These errors apparently originated from Professor Hancock’s use of GPT-4o—a generative AI tool—in drafting his declaration. ECF No. 39 ¶¶ 11, 21. GPT-4o provided Professor Hancock with fake citations to academic articles, which Professor Hancock failed to verify before including them in his declaration. Id. ¶¶ 12–14. In response, Attorney General Ellison candidly acknowledged the fake citations in the Hancock Declaration while asserting that his office had no idea that the Hancock Declaration contained fake, AI-generated citations. ECF No. 38 ¶¶ 4–6. Because the deadline to submit his response to Plaintiffs’ preliminary-injunction motion had already elapsed, Attorney General Ellison requested the Court’s leave to file an amended Hancock Declaration, citing excusable neglect to allow the late filing. ECF Nos. 34, 37 at 4–5. Plaintiffs oppose Attorney General Ellison’s request to file an amended Hancock Declaration. ECF No. 42. Plaintiffs contend that the “excusable neglect” exception is not available to Attorney General Ellison because the substance of the Hancock Declaration would change and prejudice Plaintiffs. Id. at 6–10. Plaintiffs continue to maintain that the fake citations in the Hancock Declaration taint the entirety of Professor Hancock’s opinions and render any opinion by him inadmissible. ECF No. 44 at 8–9. ANALYSIS “[A] preliminary injunction is customarily granted on the basis of procedures that are less formal and evidence that is less complete than a trial on the merits.” Univ. of Tex. v. Camenisch, 451 U.S. 390, 395 (1981). Accordingly, courts within this Circuit have noted 3 that the Federal Rules of Evidence—including Rule 702 and the related Daubert analysis— are either relaxed, Cooke v. Randolph, Neb. City Council, No. 8:23-cv-249 (JMG), 2023 WL 6519374, at *2 n.3 (D. Neb. Oct. 5, 2023), or inapplicable, USA Visionary Concepts, LLC v. MR Int’l, LLC, No. 4:09-cv-874 (DGK), 2009 WL 10672094, at *5 (W.D. Mo. Nov. 17, 2009), at the preliminary-injunction stage. See also Guntert & Zimmerman Constr. Div., Inc. v. Gomaco Corp., No. 20-cv-4007 (CJW/KEM), 2020 WL 6948364, at *1 (N.D. Iowa Oct. 14, 2020) (explaining that affidavits submitted at the preliminaryinjunction stage “need not meet the requirements of affidavits under . . . the Federal Rules of Evidence, but the Court may consider the ‘competence, personal knowledge and credibility of the affiant’ in determining the weight to give the evidence”) (citation omitted); Parks v. City of Charlotte, No. 3:17-cv-00670-GCM, 2018 WL 4643193, at *4 (W.D.N.C. Sept. 27, 2018) (“[R]ather than making a final determination under Rule 702 and Daubert of whether [the proposed experts] qualify as experts for purposes of this trial, the Court will make a less formal review of the affidavits to see if they present the indicia of reliability common to expert testimony.”). Plaintiffs ask the Court to undertake a full Daubert analysis of the West and Hancock Declarations, ECF No. 30 at 9–21; ECF No. 42 at 4–8, but because the evidentiary rules at this juncture are “less formal” than at trial, the Court declines to do so. Camenisch, 451 U.S. at 395. Rather, the Court will evaluate the “competence, personal knowledge and credibility” of the West and Hancock Declarations. Gomaco Corp., 2020 WL 6948364, at *1. Nevertheless, the line of cases under Daubert remain “useful guideposts” for 4 evaluating the credibility of the experts’ opinions. St. Michael’s Media Inc. v. Mayor of Baltimore, 566 F. Supp. 3d 327, 353–56 (D. Md. 2021). I. West Declaration As for the West Declaration, Plaintiffs argue that it is conclusory because it lacks a reliable methodology under Daubert. ECF No. 30 at 17–21. But Daubert does not apply with full force at the preliminary-injunction stage, see Camenisch, 451 U.S. at 395, and the Court is satisfied that the “competence, personal knowledge and credibility” of Professor West’s testimony weigh in favor of admitting his declaration at this early stage. Gomaco Corp., 2020 WL 6948364, at *1 (citation omitted) (internal quotation marks omitted). Moreover, as Attorney General Ellison points out, even under Daubert, experts may offer more “generalized testimony . . . to educate the factfinder on general principles.” Fed. R. Evid. 702 advisory committee’s note to 2000 amendment. The West Declaration largely fits this mold, 1 so to the extent that Plaintiffs criticize the declaration as “conclusory” or “generalized,” the Court finds that the West Declaration is admissible for the purpose of educating this Court generally on AI, deepfakes, and the psychological and speech-related impacts of deepfakes. The Court also rejects Plaintiffs’ strained argument that the West Declaration contains legal conclusions. ECF No. 30 at 22–25. Although an expert may not testify as A portion of paragraph 23 of the West Declaration touches on applying Professor West’s testimony to the facts of this case. But the majority of that paragraph simply provides background information on the psychological impacts of deepfakes and the impact of deepfakes on the marketplace of ideas, so the Court considers this paragraph for the purpose of obtaining helpful background knowledge on these issues. 1 5 to whether “a legal standard has been met,” an expert “may offer his opinion as to facts that, if found, would support a conclusion that the legal standard at issue was satisfied.” Scalia v. Reliance Trust Co., No. 17-cv-4540 (SRN/ECW), 2021 WL 795270, at *19 (D. Minn. Mar. 2, 2021) (citation omitted). Here, Plaintiffs take issue with the West Declaration’s discussion of the shortcomings of counterspeech to respond to deepfakes. But whether counterspeech is effective in combatting deepfakes is not a legal standard; rather, it is a fact relevant to the ultimate legal inquiry here: the First Amendment meansfit analysis. See ECF No. 11 at 29–31 (Plaintiffs analyzing the availability of counterspeech within the means-fit analysis of their preliminary-injunction briefing). The West Declaration does not offer an impermissible legal conclusion because nowhere does Professor West testify whether “a legal standard has been met.” Scalia, 2021 WL 795270, at *19. Plaintiffs finally argue that the West Declaration contradicts Professor West’s 2020 book about misinformation because in that book, Professor West purportedly dismissed the idea of regulating misinformation. ECF No. 30 at 27–30. As an initial matter, this would not justify excluding the West Declaration, as “the factual basis of an expert opinion goes to the credibility of the testimony, not the admissibility,” and “[o]nly if the expert’s opinion is so fundamentally unsupported that it can offer no assistance to the jury must such testimony be excluded.” Hartley v. Dillard’s, Inc., 310 F.3d 1054, 1061 (8th Cir. 2002) (citation omitted) (internal quotation marks omitted). As to credibility, the Court gives little weight to Plaintiffs’ argument because Professor West’s book was written over four 6 years ago, and it is plausible that in the fast-changing world of AI technology, his expert opinion has evolved in that four-year gap. Moreover, the Court discerns some selective quotation on Plaintiffs’ part. For example, Plaintiffs quote a passage from the book about the regulation of misinformation: “it runs afoul of the First Amendment to the US Constitution, which guarantees freedom of speech.” ECF No. 31-7 at 12. But later in the same paragraph—in a passage that Plaintiffs do not quote—Professor West and his co-author recognize that “[f]or a democracy to function properly, a country needs an informed populace with access to reliable information,” and concludes that such an argument “could justify governmental regulation of social media.” Id. This passage alone undermines Plaintiffs’ assertion that Professor West’s 2020 book categorically dismisses the possibility of regulating misinformation. In sum, the Court finds that the “competence, personal knowledge and credibility” of the West Declaration weigh in favor of its admissibility at this stage. Gomaco Corp., 2020 WL 6948364, at *1. Plaintiffs’ motion to exclude the West Declaration—whether in whole or in part—is therefore denied. The Court will consider the West Declaration in ruling on Plaintiffs’ preliminary-injunction motion. II. Hancock Declaration The Hancock Declaration is a different matter. Attorney General Ellison concedes that Professor Hancock included citations to two non-existent academic articles and incorrectly cited the authors of a third article. ECF No. 37 at 3–4. Professor Hancock admits that he used GPT-4o to assist him in drafting his declaration but, in reviewing the 7 declaration, failed to discern that GPT-4o generated fake citations to academic articles. ECF No. 39 ¶¶ 11–14, 21. The irony. Professor Hancock, a credentialed expert on the dangers of AI and misinformation, has fallen victim to the siren call of relying too heavily on AI—in a case that revolves around the dangers of AI, no less. Professor Hancock offers a detailed explanation of his drafting process to explain precisely how and why these AI-hallucinated citations in his declaration came to be. Id. ¶¶ 10–22. And he assures the Court that he stands by the substantive propositions in his declaration, even those that are supported by fake citations. Id. ¶ 22. But, at the end of the day, even if the errors were an innocent mistake, and even if the propositions are substantively accurate, the fact remains that Professor Hancock submitted a declaration made under penalty of perjury with fake citations. It is particularly troubling to the Court that Professor Hancock typically validates citations with a reference software when he writes academic articles but did not do so when submitting the Hancock Declaration as part of Minnesota’s legal filing. ECF No. 39 ¶ 14. One would expect that greater attention would be paid to a document submitted under penalty of perjury than academic articles. Indeed, the Court would expect greater diligence from attorneys, let alone an expert in AI misinformation at one of the country’s most renowned academic institutions. To be clear, the Court does not fault Professor Hancock for using AI for research purposes. AI, in many ways, has the potential to revolutionize legal practice for the better. See Damien Riehl, AI + MSBA: Building Minnesota’s Legal Future, 81-Oct. Bench & Bar of Minn. 26, 30–31 (2024) (describing the Minnesota State Bar Association’s efforts to 8 explore how AI can improve access to justice and the quality of legal representation). But when attorneys and experts abdicate their independent judgment and critical thinking skills in favor of ready-made, AI-generated answers, the quality of our legal profession and the Court’s decisional process suffer. The Court thus adds its voice to a growing chorus of courts around the country declaring the same message: verify AI-generated content in legal submissions! See Mata v. Avianca, Inc., 678 F. Supp. 3d 443, 466 (S.D.N.Y. 2023) (sanctioning attorney for including fake, AI-generated legal citations in a filing); Park v. Kim, 91 F.4th 610, 614–16 (2d Cir. 2023) (referring attorney for potential discipline for including fake, AI-generated legal citations in a filing); Kruse v. Karlan, 692 S.W.3d 43, 53 (Mo. Ct. App. 2024) (dismissing appeal because litigant filed a brief with multiple fake, AI-generated legal citations). To be sure, Attorney General Ellison maintains that his office had no idea that Professor Hancock’s declaration included fake citations, ECF No. 38 ¶¶ 4–6, and counsel for the Attorney General sincerely apologized at oral argument for the unintentional fake citations in the Hancock Declaration. The Court takes Attorney General Ellison at his word and appreciates his candor in rectifying the issue. But Attorney General Ellison’s attorneys are reminded that Federal Rule of Civil Procedure 11 imposes a “personal, nondelegable responsibility” to “validate the truth and legal reasonableness of the papers filed” in an action. Pavelic & LeFlore v. Marvel Ent. Grp., 493 U.S. 120, 126–27 (1989). The Court suggests that an “inquiry reasonable under the circumstances,” Fed. R. Civ. P. 11(b), may 9 now require attorneys to ask their witnesses whether they have used AI in drafting their declarations and what they have done to verify any AI-generated content. The question, then, is what to do about the Hancock Declaration. Attorney General Ellison moves for leave to file an amended version of the Hancock Declaration, ECF No. 34, and argues that the Court may still rely on the amended Hancock Declaration in ruling on Plaintiffs’ preliminary-injunction motion. Plaintiffs seem to accept that Professor Hancock is qualified to render an expert opinion on AI and deepfakes, and the Court does not dispute that conclusion. See ECF No. 30 at 4 (accepting that Professors Hancock and West “are well-published in their fields”). Nevertheless, Plaintiffs argue that the Hancock Declaration should be excluded in its entirety and that the Court should not consider an amended declaration. ECF No. 44 at 3. The Court agrees. Professor Hancock’s citation to fake, AI-generated sources in his declaration—even with his helpful, thorough, and plausible explanation (ECF No. 39)—shatters his credibility with this Court. At a minimum, expert testimony is supposed to be reliable. Fed. R. Evid. 702; see also Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 589 (1993) (explaining that expert testimony must be “not only relevant, but reliable”). More fundamentally, signing a declaration under penalty of perjury is not a mere formality; rather, it “alert[s] declarants to the gravity of their undertaking and thereby have a meaningful effect on truthtelling and reliability.” Acosta v. Mezcal, Inc., No. 17-cv-0931 (JKB), 2019 WL 2550660, at *2 (D. Md. June 20, 2019); see also In re World Trade Ctr. Disaster Site Litig., 722 F.3d 483, 488 (2d Cir. 2013) (explaining that the “under penalty of perjury” affirmation “impresses upon the declarant the specific punishment to which he or she is subjected for 10 certifying to false statements”). The Court should be able to trust the “indicia of truthfulness” that declarations made under penalty of perjury carry, but that trust was broken here. Davenport v. Bd. of Trs. of State Ctr. Comm. Coll. Dist., 654 F. Supp. 2d 1073, 1083 (E.D. Cal. 2009). Moreover, citing to fake sources imposes many harms, including “wasting the opposing party’s time and money, the Court’s time and resources, and reputational harms to the legal system (to name a few).” Morgan v. Cmty. Against Violence, No. 23-cv-353WPJ/JMR, 2023 WL 6976510, at *8 (D.N.M. Oct. 23, 2023). Courts therefore do not, and should not, “make allowances for a [party] who cites to fake, nonexistent, misleading authorities”—particularly in a document submitted under penalty of perjury. Dukuray v. Experian Info. Sols., 23 Civ. 9043 (AT) (GS), 2024 WL 3812259, at *11 (S.D.N.Y. July 26, 2024) (quoting Morgan, 2023 WL 6976510, at *7). The consequences of citing fake, AIgenerated sources for attorneys and litigants are steep. See Mata, 678 F. Supp. 3d at 466; Park, 91 F.4th at 614–16; Kruse, 692 S.W.3d at 53. Those consequences should be no different for an expert offering testimony to assist the Court under penalty of perjury. To be sure, the Court does not believe that Professor Hancock intentionally cited to fake sources, and the Court commends Professor Hancock and Attorney General Ellison for promptly conceding and addressing the errors in the Hancock Declaration. But the Court cannot accept false statements—innocent or not—in an expert’s declaration submitted under penalty of perjury. Accordingly, given that the Hancock Declaration’s errors undermine its competence and credibility, the Court will exclude consideration of Professor Hancock’s expert testimony in deciding Plaintiffs’ preliminary-injunction 11 motion. Gomaco Corp., 2020 WL 6948364, at *1. Because the Court declines to consider Professor Hancock’s testimony in deciding Plaintiffs’ preliminary-injunction motion, the Court will deny as moot Attorney General Ellison’s motion for leave to file an amended Hancock Declaration. CONCLUSION Based upon the foregoing, and all the files, records, and proceedings herein, IT IS HEREBY ORDERED that: 1. Plaintiffs’ Motion to Exclude Expert Testimony (ECF No. 29) is GRANTED IN PART AND DENIED IN PART. The motion is granted as to the Hancock Declaration and denied as to the West Declaration. 2. Defendants’ Motion for Leave to File an Amended Expert Declaration (ECF No. 34) is DENIED AS MOOT. Dated: January 10, 2025 s/Laura M. Provinzino Laura M. Provinzino United States District Judge 12
]]>
Some case metadata and case summaries were written with the help of AI, which can produce inaccuracies. You
should read the full case before relying on it for legal research purposes.
This site is protected by reCAPTCHA and the Google
Privacy Policy and
Terms of Service apply.
