SPAD Image Sensors with Embedded Intelligence

Single-photon avalanche diodes (SPADs) are solid-state photodetectors that can detect individual photons with picosecond timing precision, enabling powerful time-resolved imaging across scientific, industrial, and biomedical applications. Despite their unique sensitivity, conventional SPAD imaging workflows passively collect photons, transfer large volumes of raw data off-chip, and reconstruct results through offline post-processing, leading to inefficiencies in photon usage, high latency, and limited adaptability. This thesis explores the potential of embedded artificial intelligence (AI) for efficient, real-time, intelligent processing in SPAD imaging through hardware-software co-design, bringing computation directly to the sensor to process photon data in its native form. Two general frameworks are proposed, each representing a paradigm shift from the conventional process. The first framework is inspired by the power of artificial neural networks (ANNs) in computer vision. It employs recurrent neural networks (RNNs) that operate directly on timestamps of photon arrival, extracting temporal information in an event-driven manner. The RNN is trained and evaluated for fluorescence lifetime estimation, achieving high precision and robustness. Quantization and approximation techniques are explored to enable FPGA implementation. Based on this, an imaging system integrating a SPAD image sensor with an on-FPGA RNN is developed, enabling real-time fluorescence lifetime imaging and demonstrating generalizability to other time-resolved tasks. The second framework is inspired by the human visual system, employing spiking neural networks (SNNs) that operate directly on the asynchronous pulses generated by SPAD avalanche breakdown upon photon arrival, thereby enabling temporal analysis with ultra-low latency and energy-efficient computation. Two hardware-friendly SNN architectures, Transporter SNN and Reversed start-stop SNN are proposed, which transform the phase-coded spike trains into density-coded and inter-spike-interval-coded representations, enabling more efficient training and processing. Dedicated training methods are explored, and both architectures are validated through fluorescence lifetime imaging. Based on the Transporter SNN architecture, the first SPAD image sensor with on-chip spike encoder for active time-resolved imaging is developed. This thesis encompasses a full-stack imaging workflow, spanning SPAD image sensor design, FPGA implementation, software development, neural network training and evaluation, mathematical modeling, fluorescence lifetime imaging, and optical system setup. Together, these contributions establish new paradigms of intelligent SPAD imaging, where sensing and computation are deeply integrated. The proposed frameworks demonstrate significant gains in photon efficiency, processing speed, robustness, and adaptability, illustrating how embedded AI can transform SPAD systems from passive detectors into intelligent, adaptive, and autonomous imaging platforms for next-generation applications.

Nov 11, 2025 5:00 PM — 6:00 PM