Overreliance on legal AI chatbots and the erosion of reasoning skills
AI can be used for routine tasks, like for document review or proofreading - this can save time. But interpreting its results should trigger critical thinking, not blind faith.
Published on: 14 October 2025, 12:02 pm
This is an extension of an earlier essay on the difficulties surrounding legal AI chatbots published here.
BEYOND SYSTEMIC ISSUES, THERE IS A HUMAN COST to misusing AI in law. Many lawyers worry that treating AI as an oracle will erode their own legal reasoning. If an AI can quickly draft a petition or a plaint or summarize a case, will younger lawyers still learn how to do those tasks themselves? Professors caution that legal reasoning is a skill honed by years of learning and struggle through messy and difficult problems; handing that to an AI chatbot “on autopilot mode” may lead to deskilling.
There has to be a balance. Keith Porcaro notes that errors [from LLMs] look different from human errors, and even savvy users may fail to catch them. Without deliberate safeguards - like cross-checking AI output against trusted sources - users may become complacent. Over time, junior lawyers who lean more on AI might miss the chance to develop core competencies. The American Bar Association has recognized this risk in issuing ethics opinions: it reminds lawyers that they cannot abdicate professional judgment to software. The tool must assist the lawyer’s mind, not replace it.
This leads us to a normative stance that AI should be supplemental, and not determinative. In practical terms, courts and law firms can adopt workflows where AI suggestions are treated as draft proposals and require human revision(s) before submitting to the court of law. AI can be used for routine tasks, like for document review or proofreading - this can save time. But interpreting its results should trigger critical thinking, not blind faith.
If courts and law firms do not encourage the dual approach of using AI to flag potential issues while also doing deeper analysis of how those issues map to current law(s) and ethics, generations of lawyers may lose confidence in their own reasoning and over-rely on technology.
In practical terms, courts and law firms can adopt workflows where AI suggestions are treated as draft proposals and require human revision(s) before submitting to the court of law.
Ethical and Institutional Risks: Bias, Privacy, and Misinformation
Beyond AI inaccuracy, there are also broader ethical and societal risks, whenever AI attempts or is allowed to supplement the law.
Bias replication: AI models learn from human-generated data; any biases in that data can be amplified. ChatGPT and similar models will regurgitate racist, sexist, or otherwise prejudiced stereotypes. Risk assessment algorithms may misclassify defendants at twice the rate of other defendants on discriminatory aspects, namely caste, religion, et cetera.
If a legal AI chatbot inherits such biases, it could generate advice that disadvantages certain groups. For instance, it might implicitly assume some default facts about defendants that reflect societal prejudice, conflicting with legal ethics and constitutional equal protection principles.
As the world relies more on technology, AI could amplify existing inequalities, eroding public trust in legal institutions.
Data privacy and confidentiality: AI models often require vast datasets for training. Imagine: When a lawyer inputs client details into a third-party chatbot, who controls that information?