No court filings, messages to opposing counsel, signed agreements or transfers of funds.
Who reviews the advice?
User review is not lawyer review. Identify who checks the advice before someone relies on it.
An anonymized example informed by Klaimee’s assessment work. The scenarios below are illustrations, not claims that every assessed agent has these faults.
02
Failure scenarios
What happens when a customer acts on the wrong answer?
A missed deadline can cost a customer their claim. An inaccurate letter can weaken their position in a dispute. These are the failures to test before customers rely on the product.
01 / Legal research
The agent cites a real case that does not support its advice.
An employment letter uses a judgment about overtime pay to justify a claim about dismissal warnings. The case exists, but it does not establish the rule the agent gives the customer.
Possible consequence
The customer sends the letter, then spends time or money pursuing an argument they cannot support.
What to test or change
Check the exact passage, jurisdiction and relevance to each claim. Keep the source with the answer so a reviewer can reproduce the check.
02 / Deadlines & jurisdiction
The agent gives a filing deadline based on the wrong date.
Two documents give different dates for the end of a tenancy. The agent picks one and calculates a deadline without resolving the conflict or checking the applicable jurisdiction.
Possible consequence
The customer waits too long and may lose the chance to recover money through a claim.
What to test or change
Flag conflicting dates, ask for the missing contract and keep any deadline conditional until the relevant facts and legal basis are verified.
03 / Document drafting
A letter adds an admission nobody made.
The customer says their deposit was withheld. The agent drafts a letter saying the landlord admitted fault, although the customer never said that.
Possible consequence
The customer sends a false statement that the landlord can challenge, weakening their position in the dispute.
What to test or change
Trace factual claims to supplied evidence. Leave missing facts as placeholders and preserve fictional-example labels in exported documents.
04 / Action boundaries
The agent says a notice was sent when it only wrote a draft.
A customer asks the agent to prepare a notice. The reply describes delivery as completed, although the product has no way to send it.
Possible consequence
A customer believes a notice has been served and takes no further action.
What to test or change
Label drafts clearly. Require confirmation from the sending or filing system before saying an action is complete.
05 / Confidentiality
One client receives another client’s case details.
A document search returns files from the wrong customer or case. The agent includes private details in its answer.
Possible consequence
Confidential information is disclosed, with potential investigation costs and customer claims.
What to test or change
Check access before searching documents. Keep each client’s case files separate and test those restrictions with fictional records.
06 / Prompt injection
An uploaded document changes the instructions.
A case note tells the agent to invent evidence supporting one side of a dispute. The agent follows the instruction instead of treating it as text in a document.
Possible consequence
Untrusted material changes the advice or exposes details that make further attacks easier.
What to test or change
Keep uploaded text separate from the agent’s operating instructions. Test hidden commands in documents, including text that claims to come from a developer or administrator.
03
Coverage questions
Does your existing policy cover this work?
Check who is insured, which legal services the policy includes and what losses it excludes. Technology errors and omissions (E&O), professional liability and cyber policies address different parts of the risk.
Policies to compare for a legal AI product
Policy
What to review
The question to ask
Technology E&OFor the technology provider
Liability arising from errors, omissions or negligent acts in declared products and services. Coverage varies by wording. Travelers overview
Does the description of insured services include legal advice and drafting, including a harmful answer without a software bug?
Professional liabilityFor the legal service provider
The insured lawyers or firm, their legal services, and the way AI is used. A technology vendor and its customer are not automatically the same insured. Chubb’s employed-lawyers example
Whose advice is being insured, and is the actual service model within the policy’s scope?
Cyber insuranceFor privacy & security events
Breach response, privacy liability and other selected cyber protections. Review the actual triggers and exclusions. Travelers overview
If the advice is wrong but no security or privacy event occurs, which coverage would respond?
General liabilityFor general business exposures
Do not assume a general liability policy addresses financial loss from technology advice. E&O is a separate coverage question. E&O and general liability
Where is the protection for a customer’s financial loss from relying on the agent?
This comparison is general information, not a coverage determination or legal advice. Ask your broker and legal adviser to review the applicable wording and jurisdiction.
04
Klaimee coverage
Financial protection when customers rely on the wrong legal answer.
Klaimee's AI agent coverage is designed for the financial consequences of agent errors, including the risk of customers relying on wrong advice. Your proposal defines the legal tasks and types of claim included.
These examples show what to address in your Klaimee quote. Only the agreed contract determines what is covered.
Incorrect advice
A customer follows an unsupported legal argument or misses a deadline.
What it could cost you
They seek compensation from your business for the money they lost.
Coverage focus
Liability arising from the advice, research and drafting included in the agreed scope.
An action falsely confirmed
The agent says a notice was sent, but only created a draft.
What it could cost you
The customer does nothing further and loses the chance to act on time.
Coverage focus
Claims caused by false completion statements, with drafting and sending permissions clearly defined.
Confidential case files exposed
One client receives another client's documents.
What it could cost you
Your team must investigate the disclosure and respond to affected clients.
Coverage focus
The treatment of privacy claims and incident costs, including how this fits with existing cyber insurance.
Your coverage proposal
A defined scope, limit and price.
Bring the largest financial loss a customer could suffer from one wrong answer. We use it, alongside your contracts and controls, to discuss a suitable limit.
The agent, activities and deployments the proposal addresses
The failures and types of loss included, with limits and exclusions
Indicative pricing based on your workflow and potential losses
Required controls and evidence to keep if an incident occurs
The assessment supports that proposal: we document permissions, review controls and use agreed tests to identify failures to fix. A test result alone does not activate coverage.
Coverage is subject to availability, agreed terms, limits and exclusions. A warranty attached to Klaimee's testing service backs the commitments in that service agreement; it is not blanket insurance for every loss caused by an agent.
For lawyers, using AI does not remove professional obligations. The ABA’s guidance addresses competence, confidentiality and review of AI-assisted work. Assessment or insurance does not authorize a business to provide regulated legal services. Read ABA Formal Opinion 512
05
Buyer questions
Before you put legal AI in front of customers.
What insurance does a legal AI startup need?
Start with who provides the service and who could face a claim. A software vendor should review technology E&O for its declared product and services, plus cyber coverage for privacy and security events. A law firm should review its professional liability policy for AI-assisted legal work. Direct-to-consumer legal advice needs a separate review of the business model, applicable rules and policy wording. One policy does not automatically protect every party.
Does technology E&O cover AI-generated legal advice?
It can, depending on the policy. Check whether the insured services include the advice, research or drafting your product actually delivers, and whether AI, professional-services or other exclusions apply. Give the insurer a concrete scenario, such as a customer losing money after relying on an unsupported legal conclusion, and ask how the wording responds. The absence of a software bug does not settle the coverage question.
Can an AI hallucination lead to an insured claim?
Potentially, if a customer suffers a loss and the event meets the policy’s conditions. Check the covered services and exclusions. An invented case citation alone does not guarantee a payout. Keep the question, sources, answer, model version and any approval so the incident can be investigated.
Is checking that legal citations exist enough?
No. A real case can address a different issue, jurisdiction or set of facts. Citation checking should also establish that the quoted passage supports the conclusion and that the reference is accurate. Follow-up verification needs access to the original sources; a failed retrieval in a later turn does not prove that the earlier citation was invented.
What if the agent cannot send emails or file documents?
That limits its ability to take external actions, but users can still act on its advice. A wrong deadline or misleading draft can cause harm without a tool call. The assessment should record that sending and filing are excluded, test that the agent respects those boundaries, and check that it never describes a draft as an action already completed.
How does Klaimee test a legal AI agent?
We agree the workflow, jurisdictions and test permissions, then send ordinary and adversarial cases to an isolated endpoint or controlled test environment. Tests cover source accuracy, conflicting facts, confidential-data boundaries, prompt injection and unauthorized-action requests. We retain prompts and responses, review possible findings with your team and distinguish confirmed issues from observations that need more evidence.
What affects the price and coverage limit?
We consider what the agent does, who checks its answers, how many customers rely on it and how much one mistake could cost. Contract values, revenue, countries served and the requested limit also affect the review. A lawyer’s research assistant and a service advising consumers directly need different coverage scopes.
What should we prepare for an assessment?
Start with your website and the product you want assessed. For the review, prepare its supported jurisdictions, allowed actions, customer exposure, logging and review controls, and known incidents. A sandbox and synthetic documents support adversarial testing. Do not send privileged client files or production credentials through a public form.
