Law firms now field AI pitches weekly — research tools, drafting assistants, intake systems, transcript summarizers — and the adoption decision differs from ordinary software procurement in one decisive way: client confidences flow through these systems, and professional obligations follow the data wherever it goes. Bar guidance across jurisdictions has converged on the same expectation: lawyers must understand, at a working level, what the technology does with their information before using it. That understanding is buildable with a structured hour of diligence per vendor. This is the question set — and the answers that should end the meeting early.
The data questions: where confidences actually go
Start where the professional risk lives. Is our data used to train your models — or your providers' models? The only acceptable answer for client data is a contractual no, in writing, covering the vendor's upstream model providers as well as the vendor itself; a settings-page toggle that defaults to yes is not a control, it is a trap for the busy. Where is data processed and stored, and for how long? Jurisdiction matters for privilege and privacy law both; indefinite retention of prompts and outputs is a discovery surface nobody needs. Who at the vendor can read our content? "Engineers may review inputs to improve the service" is a confidentiality problem wearing a support policy's clothes. And what happens on deletion — including from backups and logs? Vague answers here predict vague answers everywhere, because data lifecycle is where careful vendors are visibly careful.
The security layer: evidence, not adjectives
"Bank-grade security" is marketing; artifacts are diligence. Ask for the current SOC 2 Type II report (or ISO 27001 certificate) and actually skim the exceptions section — the audit's findings, not its existence, are the information. Confirm encryption in transit and at rest, single-sign-on and multi-factor support so the tool joins the firm's existing access controls, role-based permissions that can mirror ethical walls, and audit logs the firm can export — because when a client or regulator asks "who accessed this matter's data," the answer must not depend on the vendor's goodwill. Ask about breach notification timelines in the contract, not the FAQ. A vendor serving law firms should answer all of this fluently; hesitation on basics is itself a finding, and the pattern generalizes: firms that ask these questions get better contracts than firms that accept the deck.
The capability questions: what the tool actually does
- What model powers this, and what happens when your provider changes it? (Silent model swaps change output quality under the firm's name)
- How does the tool handle what it doesn't know — cite, hedge, or invent? Ask for a live demo on a question you know the answer to, including one with no answer
- What accuracy evaluation exists for legal tasks specifically, and can we see the methodology rather than the headline number?
- What does human review look like in the intended workflow — and what does the vendor say the tool must never be used for?
The hallucination demo is the highest-yield test in the entire process: give the tool a question whose answer you can verify and a question that has no answer, and watch whether it invents. A vendor confident in its guardrails will run this test happily; a vendor who steers the demo back to the script has answered the question anyway. Whatever the outcome, the firm's own AI use policy — not the vendor's claims — defines where the tool's output may be relied upon and what verification is mandatory.
The business questions: exit, lock-in, and the vendor's own odds
Legal AI is a young market and some vendors will not exist in three years, so diligence includes the unsentimental questions. Can the firm export its data — matters, transcripts, work product, configurations — in usable formats, at any time, without a fee negotiation? What happens to stored data if the vendor is acquired (acquirers inherit data, and their policies differ) or shuts down? What are the contract's renewal mechanics — auto-renew with 90-day notice windows is where procurement regret concentrates? And is pricing per seat, per matter, or per usage, with what caps — because usage-priced AI tools can surprise at scale. None of these questions is adversarial; every serious vendor has been asked them a hundred times, and the quality of the answers maps closely to the quality of the company.
Running it as a repeatable process
Turn the above into a one-page checklist and a standing routine: a named partner or director owns AI procurement; every tool — including the free ones individual lawyers adopt quietly, which are the riskiest category precisely because they skip procurement — goes through the same checklist; answers are filed with the contract; and approved tools get a scheduled annual re-review, because vendors change models, policies, and owners without asking. Pair the process with the firm's AI use policy so approval and permitted-use are decided together, and keep a short internal list of approved tools with their allowed uses — which converts diligence from a gate that slows adoption into a menu that speeds it. The firms handling this well are not the ones saying no to AI; they are the ones who can say yes quickly, because the questions are already asked and the boundaries already drawn.
Frequently Asked Questions
Grow your AI in Legal Practice practice with AI
Lexscale.ai builds AI search visibility, websites, and intake systems for ai in legal practice firms across North America. Book a free strategy call to see what would move the needle for your practice.
Book a Free Strategy Call →