Niklaus, Joël; Chalkidis, Ilias; Stürmer, Matthias (2 October 2021). Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction Benchmark In: Aletras, Nikolaos; Androutsopoulos, Ion; Barrett, Leslie; Goanta, Catalina; Preotiuc-Pietro, Daniel (eds.) Proceedings of the Natural Legal Language Processing Workshop 2021. Stroudsburg PA, USA: Association for Computational Linguistics
|
Text
2110.00806.pdf Available under License Creative Commons: Attribution (CC-BY). Download (1MB) | Preview |
In many jurisdictions, the excessive workload of courts leads to high delays. Suitable predictive AI models can assist legal professionals in their work, and thus enhance and speed up the process. So far, Legal Judgment Prediction (LJP) datasets have been released in English, French, and Chinese. We publicly release a multilingual (German, French, and Italian), diachronic (2000-2020) corpus of 85K cases from the Federal Supreme Court of Switzerland (FSCS). We evaluate state-of-the-art BERT-based methods including two variants of BERT that overcome the BERT input (text) length limitation (up to 512 tokens). Hierarchical BERT has the best performance (approx. 68-70% Macro-F1-Score in German and French). Furthermore, we study how several factors (canton of origin, year of publication, text length, legal area) affect performance. We release both the benchmark dataset and our code to accelerate future research and ensure reproducibility.
Item Type: |
Conference or Workshop Item (Paper) |
|---|---|
Division/Institute: |
Business School > Institute for Public Sector Transformation Business School > Institute for Public Sector Transformation > Data and Infrastructure Business School |
Name: |
Niklaus, Joël 0000-0002-2779-1653; Chalkidis, Ilias; Stürmer, Matthias 0000-0001-9038-4041; Aletras, Nikolaos; Androutsopoulos, Ion; Barrett, Leslie; Goanta, Catalina and Preotiuc-Pietro, Daniel |
Publisher: |
Association for Computational Linguistics |
Language: |
English |
Submitter: |
Joël Niklaus |
Date Deposited: |
07 Aug 2023 16:07 |
Last Modified: |
07 Aug 2023 16:22 |
Related URLs: |
|
ARBOR DOI: |
10.24451/arbor.19665 |
URI: |
https://arbor.bfh.ch/id/eprint/19665 |
Actions (login required)
 |
View Item |