Local
Federal Judges in Philadelphia Quietly Adopt AI Chatbots, Raising Accuracy and Transparency Questions
A small group of federal judges in Philadelphia's Eastern District has begun using AI chatbots in their chambers, prompting Penn researchers to study how to prevent hallucinations and ensure reliability. The move raises concerns about transparency and fairness in a court system that handles thousands of cases each year.
A handful of federal judges in the U.S. District Court for the Eastern District of Pennsylvania have started using artificial intelligence chatbots in their chambers, a development that is raising questions about accuracy, transparency, and the role of automation in the justice system.
The judges, who were not named in court records, have adopted the tools quietly, according to researchers at the University of Pennsylvania's Carey Law School who are studying the use of AI in legal settings. The researchers hope to prevent "hallucinations"âinstances where chatbots generate false or misleading informationâfrom affecting judicial work.
Polk Wagner, a professor at Penn Carey Law, said the technology is already in use but cautioned that it must be carefully monitored. "The challenge is to make sure that the tools are accurate and that judges understand their limitations," Wagner said in an interview. "We're trying to develop best practices so that the use of AI doesn't undermine the integrity of the courts."
The Eastern District of Pennsylvania, which includes Philadelphia, is one of the busiest federal trial courts in the country. Its judges handle a wide range of cases, from drug trafficking and firearms offenses to complex civil litigation. The adoption of AI tools in chambersâeven for routine tasks like drafting opinions or summarizing filingsâcould have significant implications for how justice is administered.
Legal experts have warned that AI chatbots can produce convincing but incorrect legal citations, misstate facts, or reflect biases present in their training data. In one widely publicized case earlier this year, a federal judge in New York sanctioned lawyers for submitting a brief that included fictitious case law generated by ChatGPT. The incident underscored the risks of relying on AI without verification.
It is unclear exactly how the Philadelphia judges are using the chatbotsâwhether for legal research, writing assistance, or administrative tasksâand whether they have disclosed the practice to litigants. The court's public information office did not respond to requests for comment.
Penn researchers say they are working with the judges to evaluate the tools and develop safeguards. The project, which is still in its early stages, aims to create guidelines for the responsible use of AI in the judiciary. "We want to be proactive," Wagner said. "The genie is out of the bottle, and we need to make sure it's used in a way that enhances, rather than detracts from, the fair administration of justice."
The use of AI in courts has been expanding across the country, with some state and federal judges experimenting with machine learning for tasks like bail decisions and sentencing recommendations. But the adoption of chatbots in judicial chambers remains largely unregulated, and there is no uniform standard for disclosure or oversight.
For Philadelphia residents, the development raises concerns about accountability. Criminal defendants and civil litigants have a right to know whether the judge deciding their case is relying on an automated tool that could introduce errors. Without transparency, the public cannot assess whether the technology is being used fairly.
"Judges are entrusted with enormous power, and any tool that influences their decisions should be subject to public scrutiny," said one local civil rights attorney who asked not to be named. "The courts need to be transparent about how these tools are being used and what safeguards are in place."
The Eastern District's move comes as the federal judiciary as a whole is grappling with the rapid advancement of AI. The Judicial Conference of the United States, the policy-making body for federal courts, has established a committee to examine AI issues, but it has not yet issued binding rules.
Penn's research team plans to publish its findings next year. In the meantime, the judges using the chatbots are proceeding with caution, according to people familiar with the matter. They are said to be using the tools only for low-stakes tasks and to verify all outputs manually.
Still, the lack of public disclosure troubles some advocates. "The public has a right to know if AI is being used in their cases," said a spokesperson for the Philadelphia chapter of the American Civil Liberties Union. "We need transparency and accountability, not a black box."
The court did not respond to questions about whether it plans to issue a formal policy on AI use. For now, the experiment in Philadelphia reflects a broader trend: as AI becomes more powerful, institutions are adopting it faster than they can regulate it. The question for the Eastern Districtâand for courts across the countryâis whether they can harness the technology's benefits without sacrificing the principles of fairness and due process.