A Butterfly-inspired Multisensory Neuromorphic Platform for Integration of Visual and Chemical Cues.

Unisensory cues are often insufficient for animals to effectively engage in foraging, mating, and predatory activities. In contrast, the integration of cues collected from multiple sensory organs enhances the overall perceptual experience and thereby facilitates better decision-making. Despite the importance of multisensory integration in animals, the field of artificial intelligence and neuromorphic computing has primarily focused on processing unisensory information. This lack of emphasis on multisensory integration can be attributed to the absence of a miniaturized hardware platform capable of co-locating multiple sensing modalities and enabling in-sensor and near-sensor processing. In this study, we address this limitation by utilizing the chemo-sensing properties of graphene and the photo-sensing capability of monolayer molybdenum disulfide (MoS 2 ) to create a multisensory platform for visuochemical integration. Additionally, we leverage the in-memory-compute capability of MoS 2 memtransistors to develop neural circuits that facilitate multisensory decision-making. Our visuochemical integration platform is inspired by intricate courtship of Heliconius butterflies, where female species rely on the integration of visual cues (such as wing color) and chemical cues (such as pheromones) generated by the male butterflies for mate selection. These butterflies also utilize visuochemical integration for foraging and predator avoidance. Our butterfly-inspired visuochemical integration platform has significant implications in both robotics and the advancement of neuromorphic computing, going beyond unisensory intelligence and information processing. This article is protected by copyright. All rights reserved.