Extraction of Entities from Stock Analyst Reports via Language Models

This work investigates the possibility of extracting structured data from financial analyses by stock analysts with the help of large language models. For this purpose, experts compiled and annotated a dataset of 346 documents, including 425 analyses varying in layout and format from analysts from 14 banks. The conceived method represents a 3-stage extraction process that enables a detailed format definition and prompting strategy. The first two steps represent basic classification tasks, while the third step performs entity extraction. The simplistic nature of the first steps causes different models to execute them with equal performance. Thus, the evaluation focuses on utilizing different models for the extraction step, specifically employing OpenAI's GPT-4o and GPT-4o Mini. The evaluation shows that the conceived method improves F1 scores from 0.763 to 0.972 for GPT-4o Mini and from 0.856 to 0.997 for GPT-4o compared to a generic single-step extraction. These results indicate that the multi-step approach may help in improving the effectiveness of lightweight models in sophisticated tasks.