Optimizing App Review Classification with Large Language Models: A Comparative Study of Prompting Techniques

Miriam Palmetshofer, David C. Schedl, Andreas Stöckl

Publikation: Beitrag in Buch/Bericht/TagungsbandKonferenzbeitragBegutachtung

Abstract

This paper explores the application of large language models in classifying app reviews into predefined cate-gories. The study leverages various sizes of Mistral models (tiny, small, medium, and large). It examines the effectiveness of dif-ferent prompting techniques, including zero-shot, one-shot, and few-shot prompting. The research utilizes a dataset of 6406 app reviews, previously categorized into bug/problem report, inquiry, and irrelevant, to evaluate the models' classification performance. Results indicate that few-shot prompting techniques significantly enhance the performance of all model sizes, suggesting that sophisticated prompting can almost offset the limitations of smaller models. The study also finds a diminishing return on investment with increasing model size when applying advanced prompting techniques. This highlights the importance of prompt design in optimizing large language models' performance and suggests that even smaller models can achieve competitive results with the right prompting strategy.

OriginalspracheEnglisch
TitelInternational Conference on Electrical, Computer, Communications and Mechatronics Engineering, ICECCME 2024
Herausgeber (Verlag)IEEE
Seiten1-6
Seitenumfang6
ISBN (elektronisch)9798350391183
ISBN (Print)979-8-3503-9119-0
DOIs
PublikationsstatusVeröffentlicht - 6 Nov. 2024
Veranstaltung4th International Conference on Electrical, Computer, Communications and Mechatronics Engineering, ICECCME 2024 - Male, Malediven
Dauer: 4 Nov. 20246 Nov. 2024

Publikationsreihe

NameInternational Conference on Electrical, Computer, Communications and Mechatronics Engineering, ICECCME 2024

Konferenz

Konferenz4th International Conference on Electrical, Computer, Communications and Mechatronics Engineering, ICECCME 2024
Land/GebietMalediven
OrtMale
Zeitraum04.11.202406.11.2024

Schlagwörter

  • Mechatronics
  • Reviews
  • Computational modeling
  • Large language models
  • Investment

Fingerprint

Untersuchen Sie die Forschungsthemen von „Optimizing App Review Classification with Large Language Models: A Comparative Study of Prompting Techniques“. Zusammen bilden sie einen einzigartigen Fingerprint.

Zitieren