As generative AI tools become increasingly integrated into legal education and practice, understanding how to use and evaluate them is also becoming an increasingly important skill for law students. All generative AI tools claim to offer exciting possibilities, but not all are created equal. This section provides key factors to consider, helping you choose the right AI tools to enhance your studies and prepare for your legal career.
| What to Consider | Example | Key Questions to Ask | |
| Accuracy and Reliability | Whether the AI tool provides correct and reliable outputs. | Use AI tools to draft a case summary and then cross-check the key facts and holdings against primary sources. | How frequently does the tool update its training data? What methods does the tool use to reduce errors? |
| Bias and Fairness | The degree to which the AI tool's outputs reflect biases that could lead to unfair outcomes. | Consider how different tools handle data related to race, gender, or socioeconomic status. | What datasets were used to train the AI? How does the tool mitigate bias? Is there a way to assess bias in the output? |
| Transparency and Explainability | How well the AI tool explains its decision-making process and how it reaches conclusions. | Use an AI tool to draft a legal argument and review any explanations provided on how it arrived at its conclusions, such as which sources or legal principles it relied on. | Does the tool provide information about its training data sources? How does the tool access and use external information (like databases and websites)? Can you understand the reasoning behind its conclusions? |
| Data Privacy and Security | How well the AI tool protects information. | Check the AI tool's technical documentation to assess whether it anonymizes or deletes user prompts. | How does the tool protect sensitive data? What does it do with user input prompts? Are they stored, reused, or deleted after processing? What encryption methods are used? Does the tool comply with ISO and NIST standards and data protection laws like the GDPR? |
| Cost vs. Benefit | Value of the tool relative to its price. | Compare the cost and features of AI tools, such as subscription-based options versus free alternatives, to determine which provides better value for your specific needs. | What are the costs, and what features are included? How does the price compare to similar tools? Does it offer a free trial or demo? |
A prompt is a text input that triggers a response from an AI model. Prompt engineering involves creating and refining prompts to help the AI understand tasks and produce responses.
System prompts and user prompts both guide generative AI output. The system prompt, set by the system designer, defines the AI’s overall behavior, tone, and rules for responding to user prompts. The user prompt is the specific input or question entered by the user that drives the AI’s immediate responses. System prompts for general-purpose AI tools are typically designed to guide the AI to generate conversational responses across a wide range of topics. For example, the system prompts developed by Anthropic for their Claude models broadly guide the tone of the model (e.g., "Claude is very smart and intellectually curious") and provides it with information like the current date at the start of each conversation.
To get the best results from general-purpose AI models, follow a few key prompt engineering tips: be specific, provide clear constraints, and include context. For example, instead of a broad prompt like "Create a study plan for my first semester of law school," a more effective prompt would be, "Create a weekly study plan that includes two hours a day for reading, one hour for group study, and one hour for outlining, focusing on preparing for final exams in three weeks." This gives the AI clear parameters to work within, leading to a more customized and useful response. Always specify time frames, subjects, or priorities to make the output more tailored to your needs. Always verify AI output for accuracy.
System prompts for AI models using retrieval-augmented generation (RAG), including those designed for legal research like Lexis+ AI, Ask Practical Law AI, and Westlaw Precision with CoCounsel, are more focused and domain-specific. Because of these differences, many prompt engineering techniques developed for general-purpose AI tools may be ineffective or counterproductive when applied to RAG systems. Neither Lexis nor Westlaw have disclosed the system prompts for their AI models, but both vendors advertise that their models are designed to prioritize accuracy by focusing on retrieving legal information (e.g., primary law and selected secondary sources) and generating outputs based on those sources. While RAG systems can be powerful tools, they should complement, not replace, critical thinking and analysis. Always approach AI-generated results with a critical eye and use them as a starting point for deeper research and understanding.
While more transparency from vendors and independent research is needed to establish best practices for interacting with proprietary AI models designed for legal research, we can still apply some basic prompt engineering principles based on the limited guidance provided by the developers of these tools. For more specific guidance, follow the prompting suggestions provided by developers of each model. Keep in mind that these are preliminary suggestions and will be updated when more research and guidance is available.
| Focus on content, not instructions. | Keep your input focused on the content you're seeking, not on how you want the information presented. Avoid including formatting instructions. | Instead of: "Give me a five-paragraph summary of recent Supreme Court decisions on Fourth Amendment search and seizure." | Try: "What are the most recent Supreme Court decisions about Fourth Amendment search and seizure?" |
| Be specific and concise. | Use clear, specific language to describe what you're looking for. Avoid unnecessary words or context that might "confuse" the retrieval system. | Instead of: "I'm working on a case and need to know how long someone has to file a lawsuit against a doctor who made a mistake in California, can you tell me about the time limits?" | Try: "What is the statute of limitations for filing a medical malpractice suit in California?" |
| Avoid role prompting. | Stick to straightforward questions or requests about the legal information you need. Avoid trying to give the system a specific role (like "act as a constitutional law expert") or using complex multi-step instructions. | Instead of: "Pretend you are a famous defense attorney and explain Miranda rights." | Try: "What are the rights in a Miranda warning?" |
| Use domain-specific terminology. | Include relevant legal terms that might appear in the sources you're trying to retrieve. | Instead of: "What are judges saying about decisions by government agencies?" | Try: "How are courts interpreting the major questions doctrine in cases involving federal agency regulations?" |
| Understand system-specific features. | Learn about any special features or syntax the RAG system you are using might offer (e.g., Boolean operators, citation searching, jurisdictional filters). | In Lexis+ AI, instead of: "What's the law on firing employees in California?" | In Lexis+ AI, try: (1) Limit jurisdiction to the California by using the built-in selection tool. (2) Enter your query: "What are the considerations for terminating an at-will employee?" |
| Test and iterate. | Try different phrasing of your prompt to see which yields the best results and "prompt chaining," using successive prompts that build off the AI's output. Pay attention to the sources the RAG system cites. Use your human legal research skills to determine if they are relevant to your question. If they are, curate additional prompts based on those sources. | Instead of: "How do courts decide if an expert witness is qualified to testify?" | Try using prompt chaining: (1) How do courts decide if an expert witness is qualified to testify? (2) What recent challenges have been made to expert witness qualifications in federal courts? (3) How have federal district courts applied the Daubert standard to non-scientific testimony? |