Natural Language Interface for Data Visualization with Deep Learning Based Language Models

Research output: Contribution to conferencePaperpeer-review

1 Citation (Scopus)


In this work we investigate the possibilities of integrating a Deep Learning language model for a Natural Language Interface (NLI) of an information visualisation software. For this purpose, we have developed a prototype web application that uses the deep learning model OpenAI Codex from the GPT3 family to create visualisations from text input. For comparison, we created a second prototype with a classical NLP approach based on NL4DV toolkit (with subtasks like part-of-speech (POS) tagging, entity recognition, and dependency parsing) and an almost identical interface. The two variants were subjected to a study with test persons, and the advantages and disadvantages of the two approaches and the suitability for the most common visualisation types were investigated. The Deep Learning approach offers greater expressiveness for describing the graphics, but also the danger of not always being entirely comprehensible. The participants were able to use it to create more complex visualisations, but also sometimes had problems finding the right text input to solve the tasks. In our preliminary usability study, the Deep Learning prototype performed slightly better than the comparison prototype and achieved a useful usability score.

Original languageEnglish
Number of pages7
Publication statusPublished - 2022
Event26th International Conference Information Visualisation (IV) - Technische Universität Wien, Wien, Austria
Duration: 19 Jul 202222 Jul 2022


Conference26th International Conference Information Visualisation (IV)
Internet address


  • Data analytics
  • Empirical studies in interaction design
  • Natural Language Processing
  • Visualization systems and tools


Dive into the research topics of 'Natural Language Interface for Data Visualization with Deep Learning Based Language Models'. Together they form a unique fingerprint.

Cite this