THE BLACK BOX ON THE BENCH: CONSTITUTIONAL FRICTION BETWEEN ALGORITHMIC CERTAINTY AND DUE PROCESS IN CRIMINAL SENTENCING
Abstract
This article examines the constitutional viability of algorithmic risk assessment tools in criminal sentencing, contrasting the US judiciary’s reliance on proprietary "trade secrets" with the European Union’s newly enforced transparency mandates under the AI Act (2024/2026). By analyzing the post-Loomis legal landscape and the 2025 surge in "Generative Pre-Sentence Reports," it argues that the current use of predictive policing software violates the Due Process Clause of the Fourteenth Amendment and the Equal Protection Clause. The article applies the Mathews v. Eldridge balancing test to demonstrate that the state’s interest in efficiency does not outweigh the defendant’s liberty interest in an explainable sentence. It concludes by proposing a "Qualified Transparency" framework, requiring open-source auditing for any algorithm used to deprive a citizen of liberty, effectively ending the era of the "Black Box" in the courtroom.
Keywords
For centuries, criminal sentencing was a fundamentally human act—a flawed, often capricious, but deeply personal exercise of discretion. A judge would look a defendant in the eye, weigh the severity of the act against the potential for redemption, and issue a judgment. In the last decade, however, this discretion has been quietly ceded to the
References
- Assessment of the US Sentencing Commission, The Use of Technology in Federal Sentencing, Annual Report (2025), at 14.
- Cynthia Rudin, Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead, 1 Nature Machine Intelligence 206 (2019).
- Julia Angwin et al., Machine Bias, ProPublica (May 23, 2016); see also Crystal Yang, Algorithmic Fairness and the Law, 138 Harv. L. Rev. 45 (2024).
- Sandra Mayson, Bias in, Bias Out, 128 Yale L.J. 2218 (2019).
- State v. Loomis, 881 N.W.2d 749 (Wis. 2016).
- Danielle Keats Citron, Technological Due Process, 85 Wash. U. L. Rev. 1249 (2008).
- Gardner v. Florida, 430 U.S. 349, 362 (1977).
- Washington v. Davis, 426 U.S. 229 (1976).
- United States v. Curry, 965 F.3d 313 (4th Cir. 2020) (en banc).
- See In re Use of Artificial Intelligence in Preparation of Pre-Sentence Reports, Standing Order 25-04 (D. Mass. Jan. 12, 2025).
- Townsend v. Burke, 334 U.S. 736 (1948).
- Regulation (EU) 2024/1689 of the European Parliament and of the Council (EU AI Act), Art. 6(2) & Annex III.
- United States v. Kincaid, No. 1:24-cr-00892 (S.D.N.Y. Mar. 15, 2025) (Order Granting Motion to Compel).
- European Commission, Standard Contractual Clauses for the Transfer of Personal Data to Third Countries, Decision 2021/914.
- Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information, Harvard University Press (2015).
- Mathews v. Eldridge, 424 U.S. 319 (1976).