Design-Space Exploration of Biologically-Inspired SNN Models for Application-Specific Many-Core Systems

In recent years, Spiking Neural Networks (SNNs) have drawn significant attention as a promising route for advancing machine learning models. SNNs are different from traditional neural network architectures in that they replicate the spiking behavior of biological neurons. This research study explores different aspects of SNNs, ranging from model comparison to the development of innovative runtime spiking tools, and demonstrates application in real-time environments. The exploration begins with a background study of different generations of neural network analysis and an investigation of the use of SNN for real-time applications. This research study highlights the strengths of the architectural flow of SNNs, which are event-driven and encode information via discrete spikes and are particularly well-suited for tasks that involve processing spatiotemporal data, such as sensory processing, robotics, and object detection prediction. Moving forward to achieving a complete understanding of the working mechanism of different neuronal models. A further research study was conducted on SNNs and their mathematical neural model comparison. This analysis provides a foundational understanding of the different neuronal models, strengths, and limitations. In addition, this exploration also focused on the challenges, including specifying the most suitable model for classification tasks that require high accuracy and the lowest performance loss. Finally, the results of the proposed study show significant differences in computational efficiency and performance among the models, which highlights the importance of choosing the best model for a specific task. Building upon this groundwork, a standalone AutoML concept in the SNN domain is presented to improve their adaptation of the most suitable model based on the complicated patterns in the specific dataset. The proposed AutoML-SNN algorithm is designed specifically for the utilization of SNN models, and it eliminates the trial-and-error method for the selection of neural models. The experimental results emphasize the importance of dynamic neural network selection. The proposed algorithm also improves the SNN adaptability in neural model selection approaches for obtaining high accuracy while minimizing performance loss. This framework allows for dynamic neural model selection, offering a flexible and efficient approach to optimizing SNN models based on the selective dataset. Additionally, this dissertation introduces a standalone "A Runtime Analysis and Visualization Simulator (RAVSim)", designed to enhance the user’s understanding of SNN behavior. RAVSim is a graphical tool that not only supports SNN design and analysis but also facilitates a comprehensive comparative analysis of various neural models and training of SNN models at a high level without requiring any coding skills. Moreover, the runtime VI in RAVSim introduced the first runtime interactive simulation environment, designed to analyze and dynamically visualize the behavior of SNNs, which allows end-users to interact, observe output concentration reactions, and make changes directly during the simulation at any time. This Vi enables users to explore parametric value balancing, design different connectivity schemes, and analyze using a mixed-signal visualization feature. The RAVSim is an open-source simulator, and it is officially accepted and publicly available on LabVIEW’s official website. Finally, we present a unique architecture that uses the power of SNN in combination with transfer learning to achieve real-time analysis of human presence detection and counting using spiking cameras (DAVIS 346) and perform experiments using edge computing hardware (a series of NVIDIA Jetson). This architecture, which is deployed on edge computing devices, controls a comprehensive pipeline of components and seamlessly integrates various strategies, such as combining object detection with transfer learning, human recognition, counting of detecting objects, localizing and tracking throughout the object visibility as well as architecture execution using the multi-core concept for robust detection and real-time analysis. The application is initiated by detecting objects and monitoring environments for motion events, once the spiking camera is triggered, and sends information to the core algorithm, after that the detection class first interprets the incoming data and analyzes the object using transfer learning (i.e., pre-trained CNN weights to SNNs upon detection), this process enables event-driven processing. The utilization of multi-core processing speeds up the analytical workload while maintaining real-time operations on edge devices, such as NVIDIA Jetson. The architecture also keeps a valuable spike train dataset, which records important information about recognized objects. In addition, a research study is conducted for a comparison between FPGA and NVIDIA Jetson with a basic pattern recognition application. This investigation enables us to choose the hardware platform for our multi-object detection application, which consists of a complex architecture. However, the preference for an edge device is based on important variables such as energy consumption, task execution time, and overall performance efficiency. Through this comparison, we aim to analyze how the series of NVIDIA Jetson devices (i.e, Nano, NX, Orin Nano, and Orin AGX) computation is different from FPGA’s (Basys 3b Board) reconfigurable hardware. Thus, this detailed comparison helped us to identify the costs and benefits of using NVIDIA Jetson in our real-time object detection application as compared to FPGA (i.e., how each platform handles a pattern recognition task) and also to find the best Jetson device in terms of execution of complex algorithms in the runtime environment. Therefore, Jetson devices (particularly the Orin AGX) are most suitable in our case, where applications require rapid prototyping, flexibility, and integration with third-party hardware (such as spiking cameras). The dissertation further investigates the integration of different generations of neural networks by directly combining SNNs and CNNs into a single network architecture. Therefore, we introduced a simple yet effective fully direct integrated hybrid SC-NN model that combines the strengths of SNNs and CNNs to address the advances in ML tasks effectively. By introducing SNNConv layers, the model is capable of capturing both spatial information and temporal dynamics. The experimental findings demonstrate the importance of the hybrid SC-NN model and its ability to outperform state-of-the-art approaches based on the training and validation loss metrics, presenting an effective solution to advance machine learning models. In order to encourage collaboration and knowledge sharing, this thesis concludes by contributing to the open-source community. The open-source simulator RAVSim offers researchers and developers an accessible platform to implement and experiment with SNN models. Therefore, the findings of this and all other research studies presented in this thesis are made public, including open-source code. As SNNs continue to grow, their integration into various applications and collaborative research efforts holds the potential to define the future of Machine Learning (ML) by providing innovative solutions to complex problems and opening the way for the next generation of intelligent systems. Through this exploration, the thesis aims to contribute significantly to the understanding, optimization, and application of SNNs in the field of ML.