Redes neuronales para extracción de información relevante de sentencias legales
Loading...
Date
2023
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
En los últimos anos el Procesamiento de Lenguaje Natural, desde ahora PLN, ha utilizado técnicas de Aprendizaje Automático para representar fragmentos de texto. La introducción de la arquitectura del Transformer (Vaswani et al., 2017) y posteriormente de BERT (Devlin et al., 2018) junto con su versión más pequeña ALBERT (Lan et al., 2019) revolucionaron el estado del arte en PLN, imponiéndose como estándar para resolver tareas que involucren el modelamiento computacional de lenguaje. Una de estas tareas corresponde a sumarización extractiva, donde el objetivo es crear un resumen de un texto dado seleccionando y extrayendo frases y oraciones clave del documento original. Una de las limitaciones que aparecen con el uso de BERT en este tipo de tareas corresponde al tamaño máximo que tienen los transformers para procesar el texto de entrada, lo que dificulta el trabajo con documentos largos. En este trabajo utilizamos BERT y otros modelos de lenguaje similares para construir un sistema que permita obtener la jurisprudencia de una sentencia legal de la Corte Suprema. Para ello, se propone una arquitectura capaz de encapsular la información en dos niveles: a nivel de bloque de texto y a nivel de documento, para luego realizar una clasificación binaria de cada una de los bloques. Para validar que el modelo propuesto es capaz de resolver la tarea se realizaron pruebas sobre el dataset de documentos legales BillSum (Kornilova & Eidelman, 2019), alcanzando resultados comparables con modelos del estado del arte en términos de ROUGE.
In recent years, Natural Language Processing (NLP) has used Machine Learning techniques to represent text fragments. The introduction of the architecture of the Transformer (Vaswani et al., 2017) and later of BERT (Devlin, Chang, Lee, & Toutanova, 2018) together with its smaller version ALBERT (Lan et al., 2019) revolutionized the state of the art in NLP, establishing itself as a standard for solving tasks that involve the computational language modeling. One of these tasks corresponds to extractive summarization, where the goal is to create a summary of a given text by selecting and extracting key phrases and sentences from the original document. One of the limitations that appear with the use of BERT in this type of task corresponds to the maximum size that the transformers have to process the input text, which makes it difficult to work with long documents. In this work we use BERT and other similar language models to build a system that allows us to obtain the jurisprudence of a legal sentence of the Supreme Court. For this, an architecture capable of encapsulating the information in two levels is proposed: a text block level and a document level, to then carry out a binary classification of each one of the blocks. To validate that the proposed model is capable of solving the task, tests were carried out on the legal documents dataset BillSum (Kornilova & Eidelman, 2019), reaching comparable results with state-of-the-art models in terms of ROUGE.
In recent years, Natural Language Processing (NLP) has used Machine Learning techniques to represent text fragments. The introduction of the architecture of the Transformer (Vaswani et al., 2017) and later of BERT (Devlin, Chang, Lee, & Toutanova, 2018) together with its smaller version ALBERT (Lan et al., 2019) revolutionized the state of the art in NLP, establishing itself as a standard for solving tasks that involve the computational language modeling. One of these tasks corresponds to extractive summarization, where the goal is to create a summary of a given text by selecting and extracting key phrases and sentences from the original document. One of the limitations that appear with the use of BERT in this type of task corresponds to the maximum size that the transformers have to process the input text, which makes it difficult to work with long documents. In this work we use BERT and other similar language models to build a system that allows us to obtain the jurisprudence of a legal sentence of the Supreme Court. For this, an architecture capable of encapsulating the information in two levels is proposed: a text block level and a document level, to then carry out a binary classification of each one of the blocks. To validate that the proposed model is capable of solving the task, tests were carried out on the legal documents dataset BillSum (Kornilova & Eidelman, 2019), reaching comparable results with state-of-the-art models in terms of ROUGE.
Description
Tesis (Magíster en Ciencias de la Ingeniería)--Pontificia Universidad Católica de Chile, 2023.
Keywords
Procesamiento de Lenguaje Natural, Resumen extractivo, Transferencia de Aprendizaje, Transformer, Modelos Preentrenados, Documentos Largos, Sentencias Judiciales, Natural Language Processing, Extractive Summarization, Transfer Learning, Transformers, Pretrained Models, Long Documents, Legal Sentences