A Case Study of the Scientific Articles Cited in the US Mifepristone Court Case
The recent retraction of three published papers, two that were instrumental in a prominent abortion drug legal case in the United States, highlights the implications of potentially biased research influencing critical legal decisions. This post shares details of a Digital Science-conducted investigation into 11 papers used in court evidence – including the now-retracted papers – and what that means for both science and law.
Science as evidence
As scientists investigating trust in and of science, we began to explore how science – specifically published research – is used in the law. A notable concern is if the court system utilizes scholarly articles to establish scientific truths, then it relies on a system that may be vulnerable to influence by stakeholders who aim to advance their agendas rather than advancing impartial science.
When science is incorrectly used – specifically in the case of law – there may be far-reaching consequences. Science can be misused or manipulated to add gravitas, particularly in cases that are polarizing in society, are emotionally charged and fall along societal and political lines. Yet, ignorance of science is no excuse for misuse of science in the law.
For scholarly information to be considered credible and trustworthy science, it must meet multiple criteria, including: be a representative sample of papers over a given topic and delivered through trustworthy mechanisms. After those initial checks, the science itself must be sound. Then, the evidence must also be appropriately understood by the courts.
If the court system utilizes scholarly articles to establish scientific truths, then it relies on a system that may be vulnerable to influence by stakeholders who aim to advance their agendas rather than advancing impartial science.
Mifepristone legal challenge
Our case study questions whether the scientific evidence presented in a high-profile court case in the United States (Alliance for Hippocratic Medicine, et al vs the Federal Drug Administration) provides an unbiased presentation of the science.
Heard earlier in 2023 in a Texas federal court, the legal action was taken against the US Food and Drug Administration (FDA) and was aimed at overturning FDA approval of the abortion drug Mifepristone. The case will be heard before the US Supreme Court on 26 March 2024.
Eleven published research articles were offered as evidence in this court case. (For the full list, see our Methods and Data section below). The federal judge ordered a suspension of mifepristone’s FDA approval, citing scientific evidence presented to the court.
This case was interesting to us as it made US and international news in the wake of the 2022 Dobbs case in the US Supreme Court, which had overturned Roe v. Wade. Because of our interest in research integrity, we felt this court case was worthy of further investigation, in which we asked ourselves: How were scientific papers and science presented and used as evidence?
What we would have expected: The strongest research on mifepristone, published in quality journals, with rigorous peer reviews and any conflicts of interests described. What we found, however, did not meet these standards.
Retracted articles
Three of the 11 articles cited in the court case were published in one journal, Health Services Research and Managerial Epidemiology. This journal has a recognized publisher behind it who has the ability to investigate possible manipulation of the scientific process. And after six months since one expression of concern, the publisher retracted three of Dr Studnicki’s papers based on undisclosed conflicts of interest and unreliable research methodology. Two of those papers were cited in the Texas court case. In the retraction Sage “confirmed that all but one of the article’s authors had an affiliation with one or more of Charlotte Lozier Institute, Elliot Institute, and American Association of Pro-Life Obstetricians and Gynecologists, all pro-life advocacy organizations, despite having declared they had no conflicts of interest when they submitted the article for publication or in the article itself.”
Expected results
What should we have expected from credible published research in this field?
Representative sample of papers over a given topic
Using the world’s largest linked database, Dimensions, we queried for (mifepristone OR RU-486) AND “medical abortion” over the years of 2019-2022, to assess the experts, organizations, and journals in this field of study. We also queried for (mifepristone OR RU-486) AND “medication abortion” over the years of 2019-2022. The results varied slightly but not significantly in the conclusions and are not shown below.
Experts
Due to the sensitivity of this research topic, we have not shown the authors’ names but summarize the findings.
Expected authors of the publications
When we query for mifepristone (or RU-486) and “medical abortion” over the years of 2019-2022, we find 25 top researchers (by number of publications, citations, citation mean, FCR to those publications). We note that none of the top 25 researchers – or their papers – were cited in the US court case on mifepristone.
Actual authors of the publications used in court
Using the data from Dimensions, we ascertained that none of the authors of publications cited in the mifepristone court case are among the top 24 in their field in the world. The authors also do not appear to be connected with the top 24 researchers in the field. Instead, the authors of four of the 11 papers presented in the mifepristone court case are all well-connected co-authors with each other.
Organizations
Expected organizations to have affiliations on the publications
We identified 500 institutions (by number of publications and citations) publishing research in this field. We would have expected to see these institutions and their work cited in the mifepristone court case.
Actual organizations to have affiliations on the publications
The authors of the publications used in the mifepristone court case are primarily affiliated with the Charlotte Lozier Institute (CLI), who have co-authored publications with members of the American Association of Pro Life OB/GYNS (AAPLOG) among other organizations. These authors do not have connections to those highly-cited organizations shown in the previous image. Two of the 11 papers have authors from those institutions in the above figure – those from Planned Parenthood and Princeton University.
Journals
Expected publication journals to have been represented
We would have expected journal representation from the top 20 journals by mean number of citations. ‘Citations (Mean)’ is the mean average citation number for a given group of publications being analyzed. Other metrics can also be used to determine the top journals of the field.
| ID | Journal Name | Citations (mean) |
| 1 | Current Opinion in Cell Biology | 103 |
| 2 | Nature | 102 |
| 3 | Frontiers in Immunology | 82 |
| 4 | Endocrine Reviews | 74 |
| 5 | MMWR Surveillance Summaries | 58 |
| 6 | Human Reproduction Update | 46 |
| 7 | Social Science & Medicine | 44 |
| 8 | The Lancet Regional Health – Americas | 35 |
| 9 | Advances in Pediatrics | 33 |
| 10 | Stem Cell Research & Therapy | 33 |
| 11 | Human Reproduction Open | 32 |
| 12 | Journal of Women’s Health | 32 |
| 13 | Reproductive Toxicology | 31 |
| 14 | Journal of Clinical Nursing | 30 |
| 15 | The Lancet Global Health | 29 |
| 16 | JAMA Network Open | 28 |
| 17 | Pharmaceuticals | 28 |
| 18 | Journal of Health Economics | 26 |
| 19 | Drug Delivery and Translational Research | 26 |
| 20 | Social Science Research | 24 |
Which journals were used in the case
| Source: Titles in Mifepristone 2023 Texas Case | Citation Mean | Rank |
| Obstetrics and Gynecology | 11 | 78 |
| Health Communication | 11 | 85 |
| Contraception | 7 | 74 |
| Health Services Research and Managerial Epidemiology | 3 | 225 |
| The Linacre Quarterly | 1 | 310 |
| Human Reproduction | 0 | 436 |
| BMJ Evidence-Based Medicine | – | – |
| Issues in Law & Medicine | – | – |
Science should be delivered through trustworthy mechanisms
We would have expected that authors, editors, and peer reviewers abide by the guidelines for conflicts of interest
For assessing conflict of interest, we are guided by the International Committee of Medical Journal Editors (ICMJE) definition, wherein “all participants in the peer-review and publication process – not only authors but also peer reviewers, editors, and editorial board members of journals – must consider and disclose their relationships and activities when fulfilling their roles in the process of article review and publication”. (ICMJE, n.d.).
For the journal Health Services Research and Managerial Epidemiology, a Sage journal, its policy on declaring conflicts of interest (COI) can be found here. In summary: All authors must provide a ‘Declaration of Conflicting Interests’ statement to be published with the article, disclosing any financial ties to sponsoring organizations or for-profit interests related to products discussed in the text. If no conflicts exist, the statement should say “None Declared”. Any interests that could appear as conflicts should also be disclosed to the Editor in the cover letter.
COI could exist if those involved in writing, editing, and peer-reviewing the papers have strong political affiliations, are employees, members, leaders or founders of, or writes papers for or acts as an expert witness solely aligning with organizations affiliated with the topic. Multiple peer reviewers with diverse affiliations helps balance potential biases.
We would expect a balance of scientific experts on a topic with none or declared conflicts of interest across the authors, peer reviewers, and article editors. If they have COI, we would expect them to vary from person and role (e.g., the author, peer reviewers, and/or article editor should not work for, have investments in, etc. in Company X). Note that the role and control of the editorial board varies by journal and publisher and can have some to no decision authority of the articles.
In this case, we looked for activities of perceived conflicts of interest at the individual and paper level to assess a ‘risk profile’. An individual may have political affiliations but still be objective; they should still disclose these. The reason for having multiple peer reviewers is to balance those potential biases through different reviews. Because none of the journals have open peer reviewers, and we do not know who the specific article editor was for a paper, we could assess those items for COI. However, the retraction notice does state “Sage became aware that a peer reviewer who evaluated the article for initial publication also was affiliated with Charlotte Lozier Institute at the time of the review.”
For further details on understanding conflicts of interest in scientific papers, see this publication: Safeguarding Scientific Integrity: Examining Conflicts of Interest in the Peer Review Process.
What we found regarding declared and undeclared conflicts of interest
Of the 11 papers used in the case, we found eight with undeclared conflicts of interest, where there is the potential for conflicts of interest. Note that declaring conflicts of interest (also known as ‘competing interests’) in publications has moved to an expected practice across disciplines within the past five years.
| Publication Year | Journal Title | Article Title | Authors’ affiliated advocacy organization | Declared Conflict of Interest | Possible Conflict of Interest |
| 2012 | Obstetrics and Gynecology | Extending outpatient medical abortion services through 70 days of gestational age. | Planned Parenthood | No | Yes |
| 2009 | Obstetrics and Gynecology | Immediate Complications After Medical Compared With Surgical Termination of Pregnancy | No | Yes (Pharma) | Yes |
| 2015 | Contraception | Efficacy and safety of medical abortion using mifepristone and buccal misoprostol through 63 days | Planned Parenthood | No | Yes |
| 2011 | Human Reproduction | Immediate adverse events after second trimester medical termination of pregnancy: results of a nationwide registry study | No | Yes (Pharma) | Yes |
| 2020 | Health Communication | #AbortionChangesYou: A Case Study to Understand the Communicative Tensions in Women’s Medication Abortion Narratives | Anti-abortion | No | Yes |
| 2013 | The Linacre Quarterly | The Maternal Mortality Myth in the Context of Legalized Abortion | Anti-abortion | No | Yes |
| 2021 | Health Services Research and Managerial Epidemiology | RETRACTED: A Longitudinal Cohort Study of Emergency Room Utilization Following Mifepristone Chemical and Surgical Abortions, 1999–2015 | Anti-abortion | No | Yes |
| 2021 | Issues in Law & Medicine | Deaths and Severe Adverse Events after the use of Mifepristone as an Abortifacient from September 2000 to February 2019. | Anti-abortion | No | Yes |
| 2021 | Health Services Research and Managerial Epidemiology | Mifepristone Adverse Events Identified by Planned Parenthood in 2009 and 2010 Compared to Those in the FDA Adverse Event Reporting System and Those Obtained Through the Freedom of Information Act | Anti-abortion | No | Yes |
| 2022 | Health Services Research and Managerial Epidemiology | RETRACTED: A Post Hoc Exploratory Analysis: Induced Abortion Complications Mistaken for Miscarriage in the Emergency Room are a Risk Factor for Hospitalization | Anti-abortion | No | Yes |
| 2011 | BMJ Evidence-Based Medicine | Adolescent girls undergoing medical abortion have lower risk of haemorrhage, incomplete evacuation or surgical evacuation than women above 18 years old | No | No | None Found |
Discussion
Logically, scientific articles have been used as exhibits in this Texas court case because mifepristone is a chemical drug. Yet, much of the scientific evidence appeared to be authored by the plaintiffs or organizations affiliated with the plaintiffs.
If the scientists are the authorities on the subject and have no conflicts of interest, there might be a case for using those studies as evidence. However, we would expect extreme rigor of the science and the ethics of those involved. Those organizations listed on the scholarly papers are either part of the plaintiff or affiliated with them. Two of the papers have evidence of undisclosed conflicts of interest and unreliable methods according to the retraction notice. Hence, the ‘science’ that has been cited appears to have been compromised at the very least, and potentially manipulated to serve the aims of those organizations.
At this point in time, two of the eleven papers used in evidence have now been retracted – a move towards correcting the scientific record. While it is now too late for the Texas court, which has already considered the retracted papers as part of its proceedings, it should not be too late for some further serious questioning of scientific publications presented in the original case. This issue highlights that more must be done to detect and prevent the manipulation or misuse of science and conflicts of interest in the courts.
Methods
Court Case
Alliance for Hippocratic Medicine, American Association of Pro-Life Obstetricians and Gynecologists, American College of Pediatricians, Christian Medical & Dental Associations[1], Dr. Shaun Jester, Dr. Regina Frost-Clark, Dr. Tyler Johnson[2] Dr. George Delgado[3] vs the Federal Drug Administration (2:22-cv-00223-Z).
Data
Eleven peer-reviewed articles were offered as evidence in this court case and used in this study. ‘Exhibits’: https://doi.org/10.6084/m9.figshare.25203248 We did not examine the statements or prior court cases mentioned in the exhibits.
| DOI | Title | Source title | Publisher |
| 10.1177/23333928221103107 | A Post Hoc Exploratory Analysis: Induced Abortion Complications Mistaken for Miscarriage in the Emergency Room are a Risk Factor for Hospitalization | Health Services Research and Managerial Epidemiology | SAGE |
| 10.1177/23333928211068919 | Mifepristone Adverse Events Identified by Planned Parenthood in 2009 and 2010 Compared to Those in the FDA Adverse Event Reporting System and Those Obtained Through the Freedom of Information Act | Health Services Research and Managerial Epidemiology | SAGE |
| 10.1177/23333928211053965 | A Longitudinal Cohort Study of Emergency Room Utilization Following Mifepristone Chemical and Surgical Abortions, 1999–2015 | Health Services Research and Managerial Epidemiology | SAGE |
| – | Deaths and Severe Adverse Events after the use of Mifepristone as an Abortifacient from September 2000 to February 2019. | Issues in Law & Medicine | |
| 10.1080/10410236.2020.1770507 | #AbortionChangesYou: A Case Study to Understand the Communicative Tensions in Women’s Medication Abortion Narratives | Health Communication | Taylor & Francis |
| 10.1016/j.contraception.2015.01.005 | Efficacy and safety of medical abortion using Mifepristone and buccal misoprostol through 63 days | Contraception | Elsevier |
| 10.1179/2050854913y.0000000004 | The Maternal Mortality Myth in the Context of Legalized Abortion | The Linacre Quarterly | SAGE |
| 10.1097/aog.0b013e31826c315f | Extending outpatient medical abortion services through 70 days of gestational age. | Obstetrics and Gynecology | Wolters Kluwer |
| 10.1136/ebm.2011.100064 | Adolescent girls undergoing medical abortion have lower risk of haemorrhage, incomplete evacuation or surgical evacuation than women above 18 years old | BMJ Evidence-Based Medicine | BMJ |
| 10.1093/humrep/der016 | Immediate adverse events after second trimester medical termination of pregnancy: results of a nationwide registry study | Human Reproduction | Oxford University Press (OUP) |
| 10.1097/aog.0b013e3181b5ccf9 | Immediate Complications After Medical Compared With Surgical Termination of Pregnancy | Obstetrics and Gynecology | Wolters Kluwer |
[1] Conducts lobbying activities
[2] https://www.indianasenaterepublicans.com/johnson Mentioned on AAPLOG https://aaplog.org/caring-for-both-a-curbside-consult-series/
[3] Associated with CLI and AAPLOG board member
