Detection of Text on Road Signs from Video

In this project, we develop technologies for automatically detecting text on road signs from video. Text on road signs carries much useful information necessary for a driver’s safely driving and efficient navigation. Automatically detecting text on road signs can help to keep a driver aware of the traffic situation and surrounding environments. Such a multimedia system can reduce driver’s cognitive load and enhance safety in driving, which is especially useful for elderly drivers with weak visual acuity.

We have proposed a fast and robust framework for incrementally detecting text on road signs from natural scene video. The new framework makes two main contributions. First, the framework applies a Divide-and-Conquer strategy to decompose the original task into two sub-tasks, that is, localization of road signs and detection of text. Corresponding algorithms for the two sub-tasks are proposed and they are smoothly incorporated into a unified framework through a real-time feature tracking algorithm. Second, the framework provides a novel way for text detection from video by integrating 2D features in each video frame (e.g., color, edges, texture) with 3D information available in a video sequence (e.g., object structure). The feasibility of the proposed framework has been evaluated on 22 video sequences captured from a moving vehicle. The new framework gives an overall text detection rate of 88.9% and false hit rate of 9.2%, which makes it possible for it to be applied to a driving assistant system and other tasks of text detection from video.

This research is partially supported by GM Collaborative Research Lab @ Carnegie Mellon University (http://gm.web.cmu.edu/)