Associate Professor Fengwei An’s team from the School of Microelectronics (SME) at the Southern University of Science and Technology (SUSTech) has recently made significant progress in the field of circuit design for image processing. Their research results included an optical flow estimation system for high-speed motion tracking and a super-compact hardware architecture for the bilateral filter with a piecewise approximate computing algorithm.

Their findings have been published in the IEEE Transactions on Circuits and Systems II: Express Briefs (TCAS-II), a high-level academic journal that publishes brief papers in the circuit design field.

Optical flow estimation system for high-speed motion tracking

The synchronous double-frame exposure method provides high-speed velocimetry-flow images for motion estimation by selecting adjacent frames with appropriate intervals in high frame rate video. This method has been widely applied for commercial video-based velocity measurements. Nevertheless, no low-cost, high-speed cameras offer the video quality that researchers, scientists, and industry demand.

The first paper published by Prof. An’s group was entitled “Asynchronous Double-Frame-Exposure Binocular-Camera with Pixel-Level Pipeline Architecture for High-speed Motion Tracking”.

In this study, the researchers presented an asynchronous double-frame exposure method for motion tracking to capture two-frame images with adjustable time intervals based on binocular cameras. This time interval can significantly increase the upper limit of the displacement of motion, which is usually solved by expensive high-speed cameras. Based on the proposed method, a dedicated hardware architecture is designed to implement the perspective transformation and Wallis filter to eliminate the position mismatch and brightness variance of the two cameras.

They built a high-speed Lucas-Kanade optical flow estimation system with A-DFE on the XILINX Spartan-7 FPGA platform with MT9V034 CMOS image sensors. The shortest time interval can reach 1.06 ns, which means a significant improvement of the upper limit of the measurement velocity. The compact, fully parallel architecture can attain energy efficiency with 3.23 nJ/pixel on a low-cost FPGA device.

Ruoheng Yao and Huanshihong Deng, undergraduate students of SME at SUSTech, are the co-first authors of this paper. Prof. Fengwei An is the corresponding author, and SUSTech is the sole unit of the paper. This work was supported by the Shenzhen Science and Technology Innovation Commission.

Figure 1. Overall architecture for motion estimation with the asynchronous double-frame-exposure method and the demonstration system on FPGA

Super-compact hardware architecture for the bilateral filter with a piecewise approximate computing algorithm

Image denoising is a traditional topic that aims to remove noise from a noise observation. Among the numerous methods for image denoising, the bilateral filter is a well-known method in image denoising for its edge-preserving while reducing noise. Therefore, it is a hot topic about how to design a more compact hardware architecture for the bilateral filter.

The group’s second paper was entitled “A Compact Hardware Architecture for Bilateral Filter with the Combination of Approximate Computing and Look-up Table”. They designed a super-compact hardware architecture for the BF by the proposed piecewise approximate computing algorithm.

In this study, the range weights are fitted by the least square root method and stored in the look-up table in advance. Thus, the weight storage and logic resource consumption of the piecewise approximation filter are greatly reduced. The noise reduction and edge-preserving performance can also attain equivalent performance to the standard bilateral filter.

The design accelerates the bilateral filter by the parallel pixel-level pipeline architecture and the LUT-based divider for normalization, which can reach real-time denoising at more than 30 frames/second for 8M-pixel videos at the maximum working frequency of 278 MHz with the power dissipation of only 168 mW. The hardware resource consumption reaches the top level of a similar design.

Ruoheng Yao is the first author of this paper. Prof. Fengwei An is the corresponding author, and SUSTech is the sole unit of the paper. This work was supported by the Shenzhen Science and Technology Innovation Commission.

Figure 2. Algorithm framework and example illustration of the piecewise approximation method, and the proposed architecture for the BF with approximate range kernel

Paper links (In order of appearance above):

IEEE Transactions on Circuits and Systems II: Express Briefs (TCAS-II): https://ieeexplore.ieee.org/abstract/document/9755979   

IEEE Transactions on Circuits and Systems II: Express Briefs (TCAS-II): https://ieeexplore.ieee.org/abstract/document/9733932

To read all stories about SUSTech science, subscribe to the monthly SUSTech Newsletter.