Abstract
Extracting information about vehicles or pedestrians detected by vision based systems is vital for smart surveillance applications and traffic monitoring systems. These systems have to have the ability to extract real-time parameters form the observed objects, such as number, location, speed, size, acceleration, temperature, signs, occupants, etc., necessary for next-generation intelligent transportation systems.
Both image sensors and algorithms used in the vision based systems require low data rate (leads to smaller communication bandwidth), low complexity (leads to simple and smalls implementation and lower cost), and low power for fast, smart and efficient operation. These requirements could be addressed by the proposed trapezoid (TZOID) multi-resolution complementary metal oxide semiconductor (CMOS) active pixel sensor (APS) imager that has customized pixels in the pixel array allowing efficient pixel-resolution mapping (PR-M) method.
The focus of this study is to develop techniques for designing multi-resolution CMOS APS imagers and to simplify the feature extraction algorithms with straightforward camera calibration methods while achieving low power, low complexity and low data rate for next generation traffic detection systems. Fundamental problems associated with designing pixel array of a multi-resolution CMOS image sensor, PR-M mapping method, and feature extraction algorithms with straightforward camera calibration method are addressed in this research.
The technical inquiry is divided into four research objectives. First objective is the investigation of video based traffic detection and general camera calibration/mapping methods. The main outcome of this objective is to illustrate the drawbacks of current techniques and to develop multi-resolution CMOS imager sensors and camera systems. The second objective is to model the relationship between pixel’s size and pixel’s spatial resolution in real-world for design and development of trapezoid pixel arrays that could generate low data rate at significantly reduced power consumption. The third objective is to develop camera calibration and mapping methods without specific calibration tools or iterative procedures. The fourth objective is to develop intelligent feature extraction algorithms, specifically for vehicle and pedestrian detection with low complexity and computational requirements.
The intellectual merits include: (1) providing a model for pixel’s physical size and related real-world spatial resolution, (2) understanding the challenges in design of multi-resolution pixel array for traffic detection, (3) providing a general framework on multi-resolution pixel array design for traffic applications with smaller frame memory, (4) understanding the challenges in camera calibration and feature extraction from two-dimensional (2D) images, (5) developing efficient and accurate features extraction algorithm with low complexity camera calibration procedures for vehicle and pedestrian detection.
A TZOID multi-resolution CMOS APS image sensor that extracts real-time parameters from the observed objects without requiring any specific patterns or manmade markings for traffic applications is reported in this work. The TZOID imager sacrifices detection accuracy tolerable for alleviating superfluous data to achieve low communication bandwidth and low computation power. A new P-RM method that deduces the relationship between pixel’s size and its spatial resolution was used for both deriving pixel design equations for the TZOID image sensor and for extracting location and speed of the detected objects. Additionally, P-RM method was also used for height estimation and three dimensional (3D) reconstruction of scenes captured by the imagers.
After computer simulations, both controlled laboratory and actual world experiments were performed with trapezoid and standard (rectangular) CMOS image sensors. The tests on 3D mensuration using proposed P-RM method showed overall accuracy of better than 98% in laboratory environment and better than 96% accuracy in real world pedestrian height estimation cases. It is also shown that the proposed P-RM method provides a general algorithm for 3D information extraction and reconstruction of 2D objects.
The TZOID multi-resolution CMOS APS image sensor integrated circuit (IC) was fabricated in a 0.18µm CMOS process. It was tested with P-RM method and its performance parameters were compared with an off-the-shelf standard CMOS image sensor IC. Tests showed that TZOID generates 98% less data than that of standard CMOS image sensors while trading off 2% detection accuracy.