The doctoral dissertation in the field of Computer Science will be examined at the Faculty of Science, Forestry and Technology, Joensuu campus and online.
What is the topic of your doctoral research? Why is it important to study the topic?
My doctoral research investigated the creation of virtual reality content from textual input. This research is important because it makes powerful technologies like virtual reality and animation more accessible to people who are not experts in 3D modeling, such as teachers and language learners.
Currently, implementing animation in educational settings requires advanced technical skills, creating a significant barrier for many educators. By automating the creation process, this work provides a tool to enhance education, especially for early language learning.
What are the key findings or observations of your doctoral research?
The key finding is the development of a teacher-in-the-loop text-to-VR model. For the public and educators, this provides a practical tool that allows teachers to create their own customized VR language-learning games using simple text, solving the common problem of curriculum misalignment in educational software.
For the scientific community, the research contributes a new animation pipeline, unique datasets for children's language, and syntax-aware AI models for visual semantic parsing, offering a new application for integrating generative AI into education.
How can the results of your doctoral research be utilised in practice?
The primary practical application of my doctoral research is Imikathen-VR, a system that enables K-1 and K-2 teachers to create their own customizable VR language learning games using natural language. This tool solves a key challenge by allowing educators to design exercises that directly align with their specific curriculum goals.
What are the key research methods and materials used in your doctoral research?
The methodology for my doctoral research was design science research, a framework focused on building and evaluating an artifact, in this case, the Text-to-VR system. Within this research framework, my work was heavily data-driven, centering on the fine-tuning of several pre-trained generative models for specific natural language understanding tasks. I worked with models such as T5 and BART for sentence simplification and adapted a BART-based model for converting reported speech to direct dialogue.
For the complex task of labeling the visual elements, I developed a syntax-aware RoBERTa-based model. I also used RoBERTa as a masked language model, alongside the ConceptNet knowledge base, to infer missing visual information from underspecified text. A significant component of the research was the creation of custom datasets, which were necessary because no suitable corpora existed. I manually annotated texts from Project Gutenberg's Children's Picture Books.
The doctoral dissertation of Nacir Bouali, MSc, entitled Virtual reality generation from natural language will be examined at the Faculty of Science, Forestry and Technology, Joensuu Campus. The opponent will be University Lecturer, Docent Jouni Smed, University of Turku, and the custos will be Professor Markku Tukiainen, University of Eastern Finland. Language of the public defence is English.
