Sorbonne University - ISIR lab

Autonomous agents require reasoning and planning strategies for performing tasks. We, therefore, believe that the semantics captured by large language models can enhance the decision process at different levels.

Natural language can serve for building and clarifying the planning strategy, and therefore the
actions done by a robot. Several works have addressed instruction identification as abstract representation or natural language expression, but the limited data supervision is often a challenge. To tackle this issue, we propose to develop interactive training processes, which imply asking humans to
label situations with sentences, with strong care on limiting interactions to a few relevant situations, to reduce human effort. The underlying assumption is that the compositionality of language is correlated to compositionality in the agent’s world.
In this internship, we envision working on the generation of natural language instructions and improve currents model. Our objective is to enhance the semantics behind objects to identify the most relevant actions/sub-actions.

To apply for this job please visit mlia.isir.upmc.fr.