An Adaptive Video Acquisition Scheme for Object Tracking and Its Performance Optimization

Srutarshi Banerjee*, Henry H. Chopp, Juan G. Serra, Hao Tian Yang, Oliver Cossairt, Aggelos K. Katsaggelos

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


We present a novel adaptive host-chip modular architecture for video acquisition to optimize an overall objective task constrained under a given bit rate. The chip is a high resolution imaging sensor such as gigapixel focal plane array (FPA) with low computational power deployed on the field remotely, while the host is a server with high computational power. The communication channel data bandwidth between the chip and host is constrained to accommodate transfer of all captured data from the chip. The host performs objective task specific computations and also intelligently guides the chip to optimize (compress) the data sent to host. This proposed system is modular and highly versatile in terms of flexibility in re-orienting the objective task. In this work, object tracking is the objective task. While our architecture supports any form of compression/distortion, in this paper we use quadtree (QT)-segmented video frames. We use Viterbi (Dynamic Programming) algorithm to minimize the area normalized weighted rate-distortion allocation of resources. The host receives only these degraded frames for analysis. An object detector is used to detect objects, and a Kalman Filter based tracker is used to track those objects. Evaluation of system performance is done in terms of Multiple Object Tracking Accuracy (MOTA) metric. In this proposed novel architecture, performance gains in MOTA is obtained by twice training the object detector with different system generated distortions as a novel 2-step process. Additionally, object detector is assisted by tracker to upscore the region proposals in the detector to further improve the performance.

Original languageEnglish (US)
Article number9432960
Pages (from-to)17227-17243
Number of pages17
JournalIEEE Sensors Journal
Issue number15
StatePublished - Aug 1 2021


  • Image acquisition
  • Viterbi algorithm
  • image reconstruction
  • object detection
  • object tracker assisted detection
  • object tracking
  • optimization
  • video signal processing

ASJC Scopus subject areas

  • Instrumentation
  • Electrical and Electronic Engineering


Dive into the research topics of 'An Adaptive Video Acquisition Scheme for Object Tracking and Its Performance Optimization'. Together they form a unique fingerprint.

Cite this