Edwards et al v. National Milk Producers Federation et al, No. 3:2011cv04766 - Document 155 (N.D. Cal. 2013)

Court Description: ORDER GRANTING 154 STIPULATION RE THE USE OF PREDICTIVE CODING TECHNOLOGY. Signed by Judge Jeffrey S. White on April 17, 2013. (jswlc3, COURT STAFF) (Filed on 4/17/2013)
Download PDF
Edwards et al v. National Milk Producers Federation et al Doc. 155 Case3:11-cv-04766-JSW Document154 Filed04/16/13 Page1 of 9 1 7 EIMER STAHL LLP NATHAN P. EIMER (pro hac vice) Email: neimer@eimerstahl.com VANESSA G. JACOBSEN (pro hac vice) Email: vjacobsen@eimerstahl.com DANIEL D. BIRK (pro hac vice) Email: dbirk@eimerstahl.com SARAH E. MALKERSON (pro hac vice) Email: smalkerson@eimerstahl.com 224 South Michigan Avenue, Suite 1100 Chicago, IL 60604-2516 Telephone: (312) 660-7600 Fax: (312) 692-1718 8 Attorneys for Defendant Land O’Lakes, Inc. 9 [Additional Counsel Appear on Signature Page] 2 3 4 5 6 10 11 UNITED STATES DISTRICT COURT 12 NORTHERN DISTRICT OF CALIFORNIA 13 SAN FRANCISCO DIVISION 14 15 16 MATTHEW EDWARDS, GEORGIA BROWNE, and TORAH MONTESSORI SCHOOL, individually and on behalf of all others similarly situated, 17 Plaintiffs, 18 CASE NO.: 3:11-CV-04766-JSW [consolidated with 11-cv-04791-JSW and 11-cv-05253-JSW] CLASS ACTION v. 19 20 21 22 23 NATIONAL MILK PRODUCERS FEDERATION aka COOPERATIVES WORKING TOGETHER; DAIRY FARMERS OF AMERICA, INC.; LAND O'LAKES, INC.; DAIRYLEA COOPERATIVE INC.; and AGRI-MARK, INC., JOINT STIPULATION AND [PROPOSED] ORDER RE THE USE OF PREDICTIVE CODING TECHNOLOGY ACTION FILED: September 26, 2011 Defendants. 24 25 26 27 28 JOINT STIPULATION AND PROPOSED ORDER RE THE USE OF PREDICTIVE CODING TECHNOLOGY CASE NO.: 3:11-CV-04766-JSW Dockets.Justia.com Case3:11-cv-04766-JSW Document154 Filed04/16/13 Page2 of 9 1 2 3 Plaintiffs and Defendant Land O’Lakes, Inc. (collectively “the Parties” and each, individually, a Party) hereby stipulate, agree, and propose as follows: I. DEFINITIONS A. 4 “Document review corpus” refers to the body of documents remaining after 5 the exclusion of known system files, incompatible file types (e.g., database files), duplicates, 6 and documents that fall outside the agreed date range. B. 7 8 “Confidence level” means the statistical reliability of a result and in this instance refers to the likelihood that a measurement reached through sampling is accurate. “Estimation interval” refers to the statistical error rate of a measured 9 C. 10 confidence level. 11 II. 12 The Parties have discussed the methodologies or protocols for the search and review of PROTOCOL 13 documents collected from Land O’Lakes and have agreed that Land O' Lakes will utilize 14 Recommind's Axcelerate software for its review workflow and production in this case. Axcelerate 15 incorporates predictive coding (also known as technology-assisted review) functionality, which Land 16 O'Lakes will leverage for a more cost efficient and higher quality review. The Parties agree that the 17 following protocol will be followed to identify potentially responsive documents for review: 18 1. Step One – Document Collection: Land O’Lakes will collect documents in the 19 possession, custody, or control of each custodian agreed upon by the Parties and in additional 20 locations under the possession, custody, or control of Land O’Lakes (such as shared drives or 21 departmental files) immediately upon entry of this order and isolate the document review corpus. 22 2. Step Two – Control Set: After loading the document review corpus into Axcelerate, 23 Land O’Lakes will generate an initial control set of documents. The control set will be a random, 24 statistically valid sampling of documents to estimate the number of responsive documents in the 25 corpus. The control set sample shall be determined using a 95% confidence level and 2% estimation 26 interval. The control set sample will then be reviewed for responsiveness and privilege, and Land 27 O’Lakes will provide Plaintiffs with a tracking report of the results of the control set generation and 28 review. At any point during the predictive coding process, Land O’Lakes’ e-discovery liaison will 1 JOINT STIPULATION AND PROPOSED ORDER RE THE USE OF PREDICTIVE CODING TECHNOLOGY CASE NO.: 3:11-CV-04766-JSW Case3:11-cv-04766-JSW Document154 Filed04/16/13 Page3 of 9 1 meet and confer at Plaintiffs’ request to discuss the results and method of the process. Land O’Lakes 2 agrees to confer in good faith regarding the incorporation of any input from Plaintiffs on the 3 application process. 3. 4 Step Three – Seed Set and Initial Training: All responsive documents identified 5 during review of the control set will be included in an initial seed set, which will be used to train 6 Axcelerate to identify other potentially responsive documents in the document review corpus. The 7 seed set also will include responsive documents identified by Land O’Lakes through the use of search 8 and analytical tools, which will include the following: 9 a) Approximately 400 documents already identified as responsive by Land O’Lakes; 10 b) The 500 most highly ranked1 responsive documents hit upon by an application of the 11 Boolean search terms selected by Land O’Lakes (listed in Appendix A), excluding 12 documents already contained in (a), above; and c) 13 The 500 most highly ranked responsive documents hit upon by application of Boolean 14 search terms selected by Plaintiffs (listed in Appendix A), excluding documents 15 already contained in (a) and (b), above. 16 Land O’Lakes will provide Plaintiffs with a search hit count of all searches conducted and will 17 produce all non-privileged documents included in the seed set. 4. 18 Step Four – Iterative Review and Further Training: Once Axcelerate has 19 identified and prioritized potentially responsive documents based on the initial training, Land 20 O’Lakes will review a sample set of those documents for responsiveness. The sample set will consist 21 of at least the 500 documents ranked most highly by Axcelerate in order of relevance. After this 22 second round of review, the system will be trained again based on reviewer feedback to identify and 23 prioritize more potentially responsive documents. Land O’Lakes will provide Plaintiffs with tracking 24 data for this round of review. 5. 25 26 Step Five – Review: Land O’Lakes will then conduct a manual review of all documents identified by Axcelerate as potentially responsive. During this review, the iterative 27 28 1 Axcelerate ranks returned documents in order of responsiveness to search terms. 2 JOINT STIPULATION AND PROPOSED ORDER RE THE USE OF PREDICTIVE CODING TECHNOLOGY CASE NO.: 3:11-CV-04766-JSW Case3:11-cv-04766-JSW Document154 Filed04/16/13 Page4 of 9 1 process will continue; as more responsive documents are reviewed and coded, they will be used to 2 further hone the system’s ability to identify and prioritize other potentially responsive documents 3 among the remaining, unreviewed portion of the document review corpus. Newly identified 4 documents also will be subject to manual review. Review and iterative training will proceed until all 5 identified documents have been reviewed and the system indicates that the remainder of the 6 document review corpus is not likely to be responsive. Land O’Lakes will continue to provide 7 tracking data to Plaintiffs, and all responsive, non-privileged documents will be produced. 8 9 6. Step Six – Validation: After Land O’Lakes has conducted enough rounds of iterative review and training to generate a conclusion from Axcelerate that the remaining documents 10 in the document review corpus are not likely to be responsive, Land O’Lakes will perform a 11 validation test by reviewing a statistically valid and random sampling of unreviewed documents to 12 confirm that the number of potentially responsive documents in the unreviewed corpus is statistically 13 insignificant. Using a 99% confidence level and a 1% estimation interval, a random sampling of the 14 unreviewed documents will be reviewed for responsiveness. If the number of responsive documents 15 in the validation sampling results in higher than a 1% responsiveness rate, the review process will 16 continue until the validation test achieves a 1% or less responsiveness rate. All tracking data 17 regarding the validation process will be provided to the plaintiffs, and any responsive, non-privileged 18 documents identified during the process will be produced. 19 THE PARTIES HEREBY STIPULATE AND AGREE TO THE ABOVE TERMS. 20 21 22 23 24 25 26 27 28 DATED: April 16, 2013 Respectfully submitted, By:/s/ Matthew S. Kahn George A. Nicoud (SBN 106111) Matthew S. Kahn (SBN 261679) GIBSON, DUNN & CRUTCHER LLP 555 Mission Street, Suite 3000 San Francisco, CA 94105-2933 Telephone: (415) 393-8200 Fax: (415) 393-8200 tnicoud@gibsondunn.com mkahn@gibsondunn.com By:/s/ Nathan P. Eimer Nathan P. Eimer (pro hac vice) Vanessa G. Jacobsen (pro hac vice) 3 JOINT STIPULATION AND PROPOSED ORDER RE THE USE OF PREDICTIVE CODING TECHNOLOGY CASE NO.: 3:11-CV-04766-JSW Case3:11-cv-04766-JSW Document154 Filed04/16/13 Page5 of 9 6 Daniel D. Birk (pro hac vice) Sarah E. Malkerson (pro hac vice) EIMER STAHL LLP 224 South Michigan Avenue, Suite 1100 Chicago, Illinois 60604 Phone: (312) 660-7601 Fax: (312) 692-1718 neimer@eimerstahl.com vjacobsen@eimerstahl.com dbirk@eimerstahl.com smalkerson@eimerstahl.com 7 Attorneys for Defendant Land O’ Lakes, Inc. 8 By: /s/ Elaine T. Byszewski Elaine T. Byszewski HAGENS BERMAN SOBOL SHAPIRO LLP 301 North Lake Avenue, Suite 203 Pasadena, CA 91101 Telephone (213) 330-7150 Facsimile (213) 330-7152 elaine@hbsslaw.com 1 2 3 4 5 9 10 11 12 Steve W. Berman George W. Sampson Craig R. Spiegel HAGENS BERMAN SOBOL SHAPIRO LLP 1918 Eighth Avenue, Suite 3300 Seattle, WA 98101 Telephone (206) 623-7292 Facsimile (206) 623-0594 steve@hbsslaw.com george@hbsslaw.com craig@hbsslaw.com 13 14 15 16 17 18 Attorneys for the Plaintiffs 19 20 ATTESTATION: I, Matthew S. Kahn, attest that concurrence in the filing of this document has been 21 obtained from each of the other signatories. 22 23 24 [PROPOSED] ORDER PURSUANT TO STIPULATION, IT IS SO ORDERED. 25 26 27 April 17, 2013 Dated: _____________________ ___________________________ HONORABLE JEFFREY S. WHITE UNITED STATES DISTRICT JUDGE 28 4 JOINT STIPULATION AND PROPOSED ORDER RE THE USE OF PREDICTIVE CODING TECHNOLOGY CASE NO.: 3:11-CV-04766-JSW Case3:11-cv-04766-JSW Document154 Filed04/16/13 Page6 of 9 Appendix A: Search Terms 1 2 Land O’Lakes’ Search Terms 3 19 “cooperatives working together” nmpf OR “national milk producers federation” herd /10 retir* cow /10 retir* cwt AND retir* “self help” “herd buyout” “milk diversion program” cwt AND beef cwt /5 program export /10 assistance cwt /5 ambassador cwt AND (independent /5 producer) cwt AND (independent /5 farmer) cwt AND (target /10 remov*) cwt AND cull* cwt AND slaughter* cwt AND “field staff” cwt AND assessment cwt AND delegate cwt AND (milk /5 suppl*) cwt AND (price /10 increas*) cwt AND contribut* cooperative /10 individual cwt AND (cow /5 numbers) cwt AND “farm id” cents /5 hundredweight amount /10 checks cwt AND bid* cwt AND “leadership council” cwt /10 member* cwt /10 summary cwt AND presentation 20 Plaintiffs’ Search Terms 21 buyout reentry re-entry reenter re-enter rejoin re-join join /10 again price /10 increase price /10 rise price /10 raise price /10 hike price /10 stabiliz* price /10 maintain price /10 maintenance 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 22 23 24 25 26 27 28 5 JOINT STIPULATION AND PROPOSED ORDER RE THE USE OF PREDICTIVE CODING TECHNOLOGY CASE NO.: 3:11-CV-04766-JSW