A5052391231- Topic Modeling
- Artificial Intelligence in Law
- Legal Education and Practice Innovations
- Natural Language Processing Techniques
- Domain Adaptation and Few-Shot Learning
- Law, Economics, and Judicial Systems
- Adversarial Robustness in Machine Learning
- Comparative and International Law Studies
Stanford University
2021-2025
While self-supervised learning has made rapid advances in natural language processing, it remains unclear when researchers should engage resource-intensive domain-specific pretraining (domain pretraining). The law, puzzlingly, yielded few documented instances of substantial gains to domain spite the fact that legal is widely seen be unique. We hypothesize these existing results stem from NLP tasks are too easy and fail meet conditions for can help. To address this, we first present CaseHOLD...
Language models (LMs) are becoming the foundation for almost all major language technologies, but their capabilities, limitations, and risks not well understood. We present Holistic Evaluation of Models (HELM) to improve transparency models. First, we taxonomize vast space potential scenarios (i.e. use cases) metrics desiderata) that interest LMs. Then select a broad subset based on coverage feasibility, noting what's missing or underrepresented (e.g. question answering neglected English...
One concern with the rise of large language models lies their potential for significant harm, particularly from pretraining on biased, obscene, copyrighted, and private information. Emerging ethical approaches have attempted to filter material, but such been ad hoc failed take context into account. We offer an approach filtering grounded in law, which has directly addressed tradeoffs material. First, we gather make available Pile Law, a 256GB (and growing) dataset open-source...
While self-supervised learning has made rapid advances in natural language processing, it remains unclear when researchers should engage resource-intensive domain-specific pretraining (domain pretraining). The law, puzzlingly, yielded few documented instances of substantial gains to domain spite the fact that legal is widely seen be unique. We hypothesize these existing results stem from NLP tasks are too easy and fail meet conditions for can help. To address this, we first present CaseHOLD...
The use of words to convey speaker's intent is traditionally distinguished from the `mention' for quoting what someone said, or pointing out properties a word. Here we show that computationally modeling this use-mention distinction crucial dealing with counterspeech online. Counterspeech refutes problematic content often mentions harmful language but not itself (e.g., calling vaccine dangerous same as expressing disapproval vaccines dangerous). We even recent models fail at distinguishing...
Instruction tuning is an important step in making language models useful for direct user interaction. However, many legal tasks remain out of reach most open LLMs and there do not yet exist any large scale instruction datasets the domain. This critically limits research this application area. In work, we curate LawInstruct, a dataset, covering 17 jurisdictions, 24 languages total 12M examples. We present evidence that domain-specific pretraining improve performance on LegalBench, including...