Natural Language Interface for Data Visualization with Deep Learning Based Language Models

Publikation: KonferenzbeitragPapierBegutachtung

1 Zitat (Scopus)


In this work, we investigate the possibilities of integrating a deep learning language model for a natural language interface (NLI) of information visualization software. For this purpose, we develop a prototype web application that uses the deep learning model OpenAI Codex from the generative pretrained transformer 3 (GPT-3) family to create visualizations from text input. For comparison, we create a second prototype through a classical natural language processing (NLP) approach based on the Natural Language for Data Visualization (NL4DV) toolkit (with subtasks like part-of-speech (POS) tagging, entity recognition, and dependency parsing) and an almost identical interface. The two variants are subject to a study with test persons, and the advantages and disadvantages of the two approaches and their suitability for the most common visualization types are investigated.
The deep learning approach offers greater expressiveness in terms of describing the graphics but also carries the danger of not always being entirely comprehensible. Participants are able to use this approach to create more complex visualizations but, sometimes, face problems in terms of finding the appropriate text input to solve tasks. In our preliminary usability study, the deep learning prototype performs slightly better than does the comparison prototype and achieves a useful usability score.
PublikationsstatusVeröffentlicht - 2022
Veranstaltung26th International Conference Information Visualisation (IV) - Technische Universität Wien, Wien, Österreich
Dauer: 19 Juli 202222 Juli 2022


Konferenz26th International Conference Information Visualisation (IV)


  • Visualization systems and tools, Empirical studies in interaction design, Data analytics, Natural language processing


Untersuchen Sie die Forschungsthemen von „Natural Language Interface for Data Visualization with Deep Learning Based Language Models“. Zusammen bilden sie einen einzigartigen Fingerprint.