At the 9th International Summer School on AI and Big Data, Dr. Danh Le Phuoc (BIFOLD) will talk about Knowledge in Perception Systems. The keynote will take place on Friday, 07.07.2023 from 11 a.m. – 12:30 p.m.

Keynote: Knowledge in Perception Systems

Semantic memory and episodic memory play a critical role in human perception. The semantic memory refers to our brain’s repository of general world knowledge and episodic memory refers to our “episodic memory system”, which encodes, stores, and allows access to “episodic memories”, e.g. recollection of personally experienced events situated within a unique spatial and temporal contexts. This inspired us to introduce the semantic stream, a dynamic knowledge graph, wherein semantic and episodic memories are represented as interconnected graphs. This presentation allows integration and fusion of various kinds of sensory observations, e.g, images, videos and point clouds, into interlinked sub-symbolic and symbolic data streams at different levels of semantic abstractions. 

My talk will delve into the fundamental elements of perception systems, from sensory inputs to high-level cognition, providing a comprehensive overview of how different knowledge types contribute to the whole process of building these systems. Special attention will be given to dynamic knowledge representation, semantic-driven learning, the fusion of sensory data, and the integration of contextual knowledge. Furthermore, I will share my experiences in building perception pipelines for autonomous vehicles and robots via a declarative programming model based on semantic streams. This programming model enables developers to write semantic stream fusion programs, composed of if-then rules associated with stream data fusion operations for both reasoning and learning tasks.

Dr. Danh Le Phuoc

Dr. Danh Le Phuoc is a pioneer in the field of semantic stream processing and reasoning. He built some first systems in semantic sensor/stream middleware and processing,  earning various awards in the Semantic Web and IoT community, including an honorable test-time award and best papers/demos/posters. He is also known for his work as a co-editor of the W3C/OGC standard, Semantic Sensor Network Ontology. These systems and standards have been adopted in various EU, Irish and German projects, e.g COSMO, SMARTER, SMARTEDGE and AIoTwin. Notably, he is the technical coordinator of the EU Horizon project, SMARTEDGE (Semantic Low-Code Programming Tools for  Edge Intelligence) with 15 partners including Bosch, Siemens, Dell and NVIDIA to build emerging cooperative systems such as collaborative robots and V2I/I2V systems. 

Currently, he is a research group leader at TU Berlin and a Junior Fellow at Berlin Institute for Foundation for Data and Learning (BIFOLD). There, he founded the research lab, Pervasive Intelligence and Computing (PICOM.AI). The research conducted at PICOM Lab resolves around the intersection of knowledge graph, semantic communication, semantic computing, neural-symbolic AI, and edge-cloud computing continuum, with the applications for autonomous vehicles, robotics and industrial IoT. Besides, he is also a senior scientist at Fraunhofer FOKUS.

Read more about the 9th International Summer School