The doctoral dissertation in the field of Computer Science will be examined at the Faculty of Science, Forestry and Technology, Joensuu campus and online.
What is the topic of your doctoral research? Why is it important to study the topic?
My doctoral dissertation explores a novel approach to augment decisions made by autonomous agents via active recall of past experiences. To do so, an agent stores meaningful experiences in a “memory module” and retrieves them using search. Autonomous agents currently incorporate past experience only implicitly, by enriching an observed state with encoded historical information. Despite some success, agents’ capabilities remain limited, and lack critical features of “intelligence”, such as recollection, re-evaluation, and adaptation.
The studies included in my doctoral dissertation provide an alternative view of this problem and propose a basic recollection mechanism to improve decision making. Additionally, re-using experience could allow autonomous agents to be naturally more “aligned” and “understandable”, two critical aspects in e.g. real world deployment as human helpers.
What are the key findings or observations of your doctoral research?
My doctoral research proposes and analyses a possible criterion to include active past recollection in autonomous agents acting in complex, simulated pixel-based environments.
The conclusions of my work can be summarized as follows: (1) actively retrieving and recalling past experience successfully steers the behavior of an agent in a controllable and aligned way, when the memories are drawn from a pool of behaviors demonstrated by a human expert; and (2) active recall of past memories can provide a fuzzy, but surprisingly effective estimate of the future. Even more surprisingly, this is achieved in a lightweight, reliable manner that involves no learning at all.
From a more general perspective, my contributions to the field could be interesting for a plethora of real world applications, especially where aligning with a set of core directives and reflecting on the consequences of an agent’s action is critical.
How can the results of your doctoral research be utilised in practice?
The results of my doctoral dissertation constitute novel, basic work on integrating search and retrieval-based mechanisms in learning agents. The research questions answered in the dissertation open a wide variety of new questions that could motivate additional studies in the field.
Moreover, different studies may have different applications: some of them might, for instance, be applied to video games, to improve the quality of opponents or non-playable characters (NPCs); on the other hand, my latest study has received positive feedback from colleagues working in the biomedical field, such as cancer and proteomics research. While the method has not been tested in these fields, I believe they could be applied to any dynamical, evolving system with reasonable--for now--complexity.
What are the key research methods and materials used in your doctoral research?
My research revolves around the realms of Reinforcement and Imitation Learning (RL, IL), two frameworks that teach an autonomous agent to take actions based on observations, using respectively a reward function (RL) and a number of supervised learning-like approaches (IL).
Additionally, my research combines RL & IL agents with classic search and retrieval algorithms, such as k-Nearest Neighbors, variational methods such as Variational Autoencoders (VAEs), and World Models (WM), a recently-discovered class of machine learning models that learn to predict the evolution of a system in time, given the current state of the system and a sequence of actions.
The doctoral dissertation of Federico Malato, MSc, entitled Enhancing decision making with retrieval-learning hybrid agents will be examined at the Faculty of Science, Forestry and Technology, Joensuu campus. The opponent will be Senior Lecturer Diego Perez Liebana, Queen Mary University of London, United Kingdom, and the custos will be Associate Professor Ville Hautamäki, University of Eastern Finland. Language of the public defence is English.
For more information, please contact:
Federico Malato, [email protected]
- Public examination
- Dissertation (PDF)
- Photo (coming)
