By: Sean Jordan
AI can read a contract in seconds and catch a lot of what matters. It also misses things a lawyer would not, and it sounds just as sure of itself either way. Used well, it saves time and money. Used alone, it can cost you both.
Many business owners already paste contracts into ChatGPT, Claude, or Copilot before signing. That is a reasonable instinct. I see these tools from two sides. I use them in my own practice, and part of my work is testing AI models on legal tasks built from realistic deals and disputes, then grading the answers the way an experienced commercial lawyer would. Here is what the tools do well, where they fail, and how to use them without taking on risk you can't see.
What AI does well
· Summaries. It turns a 30-page agreement into a one-page summary of term, price, renewal, and termination rights.
· Spotting standard issues. It flags a missing liability cap, a one-sided indemnity, or an automatic renewal buried in the fine print.
· Comparing versions. It shows what changed between the draft you sent and the draft that came back.
· Checking against a standard. Given a clear checklist, it applies the same review to every NDA or vendor agreement, every time.
· Drafting a first pass. It produces a workable starting draft of a redline, a cover email, or a term sheet.
My testing backs this up. When the job is mechanical and the file is complete, the models are good. I once tried to build a test around rewriting a company's standard terms from a well-organized file to see where the model would fail, and ultimately I could not produce a failure, the models kept getting it right. Adding more detail only improved their output.
At my last in-house role, I led an AI-assisted review program that automated half of the department's NDA volume against a 13-clause standard. The tools are fast. But, the judgment still has to come from a person.
Where it fails
When I test a model, I give it a full deal file and the request a business person would actually send, such as "draft the complaint" or "mark up their agreement for Thursday's call." Most answers get the bulk of the work right. The misses land in the same places again and again.
It does what you ask, even when it shouldn't. Ask for a lawsuit and you get a complaint. In one test, a sales team asked for a complaint against a customer, and the file showed the customer's counterclaim would likely be larger than the company's claim. Some models drafted it without questioning the decision. The ones that advised against it still never put a priced settlement on the table. An experienced in-house lawyer would say, "I can draft this, but here is what it means for the company," and would point out that suing a customer rarely wins back the business.
It gives away your walkaway. In a negotiation test, management had approved a deal only if the company kept its existing liability cap. Three of the four models I graded offered a higher cap as a fallback. The fourth hinted that a higher cap could come with a higher price, which invites the other side to ask for it. If you let AI draft your fallback positions, it may offer terms nobody approved.
Dates and deadlines. AI miscounts notice periods, misses that "business days" differs from "days," and gets renewal windows wrong. In my tests, models noticed that a nonrenewal deadline had passed and still never told the client to send notice now so the next renewal would be the last one. Another built its whole plan on an extension right that had expired weeks earlier. Another accepted a sales manager's view that the contract ended the day notice went out, when the contract counted from the day the other side received it. Miss a 90-day nonrenewal notice and you are locked in for another year.
Misread terms. It confuses which party an obligation falls on, or treats "may" as "shall." It treats an unsigned draft amendment as if someone signed it, or trusts an internal email summary over the contract itself. A summary that flips who indemnifies whom is worse than no summary.
Missing context. It reviews the document you give it. It does not know about the side letter, the purchase order terms, or the amendment you forgot to attach, and it won't ask. Even with the whole file in front of it, it misses how the papers fit together: a newer approval policy that overrides an old playbook, a lien filing that lapsed, an insurance certificate that expired years ago.
No questions about authority. It rarely asks whether the person making the request can approve the deal. In one test, the models prepared an agreement for a sales executive to sign, and none caught that the CFO had required written approval the day before.
Invented law. AI can cite cases, statutes, and rules that do not exist. Lawyers have been sanctioned for filing briefs with fake citations from AI tools.
No sense of what matters to you. It treats every issue as equally important. It can't tell you which three points are worth a fight with this customer and which ten are not. The answers I grade often score in the high 80s or 90s on a long checklist and still miss the one call that decides the deal. A memo that gets twenty points right and the liability cap wrong is a bad memo.
Confidence. It is wrong in the same calm, certain tone it uses when it is right. Nothing in the answer tells you which is which.
The risks you can't see
Confidentiality. Free and consumer versions of some AI tools may keep what you paste in and use it to train future models. Pasting a customer contract, pricing, or employee data into one may breach a confidentiality clause you signed. Check the tool's data terms, or use a business version that does not train on your data.
Privilege. A conversation with your lawyer is generally privileged. A conversation with a chatbot is not. If a dispute follows, what you typed into an AI tool may be discoverable.
Reliance. An AI tool owes you nothing. It carries no malpractice insurance, has no duty of care, and its terms of service disclaim responsibility for its answers. If it misses something, the loss is yours.
False comfort. The biggest risk is signing faster because the summary looked clean. A review that misses the one clause that matters saves you nothing.
How to use AI safely
Use a business account. Choose a tool whose terms say it will not train on your data, and check before you paste anything confidential.
Remove what you don't need. Strip names, pricing, and personal data when the question doesn't depend on them.
Give it the whole deal. Attach the amendments, order forms, and side letters, or tell it they exist.
Tell it what you are trying to do. Say what you want from the deal, what you can't give up, and who has to approve it. Then ask what could go wrong if you do exactly what you asked for.
Ask for the clause, not just the conclusion. Have it quote the exact language behind each point, then read that language yourself.
Check every date and number by hand. Count notice periods and renewal windows on a calendar.
Never rely on a legal citation you haven't verified. If it names a case or statute, confirm it exists and says what the tool claims.
Know when to stop. Use AI to get oriented. Bring in a lawyer before you sign anything with real money, real liability, or a long term at stake.
The bottom line
AI is a good first reader and a poor last one. Let it do the first pass. Have a person who knows contracts check what matters before you sign.
If you already use AI to review contracts, my Second Look service fits in at the end. Send me the agreement and your AI tool's output. I check it against the document, flag what it missed or got wrong, and walk you through it in one call. Flat fees start at $100.
Contact me (sean@seanjordanlaw.com) to send a document for review.
This article is general information, not legal advice, and reading it does not create an attorney-client relationship. Sean Jordan Law PLLC, Winter Springs, Florida. Sean Jordan is admitted to practice in Florida.
