intro.tex

\chapter{Introduction}

This dissertation addresses the problem of traffic light detection efficiently using the inertial sensor hints of a smartphone.

Smartphone usage has been growing rapidly over the past several years. 
All smartphones today come with Global Positioning System (GPS), inertial sensors, and cameras. 
GPS provides reasonably high accuracy localization in outdoors. 
Outdoor localization by GPS is fundamental to many higher level applications such as navigation, driving assistance, and autonomous driving.

GPS is not sufficient for many higher level applications such as autonomous driving and pedestrian navigation by visually impaired for two reasons.
First, GPS accuracy is not high enough, especially in urban environments for reliable localization for these applications. 
Second, in addition to high accuracy localization, the perception of the environment is paramount for effective decision making. 

Detection and recognition of traffic lights and signs in public streets is an important part of perception for both autonomous driving and navigation by visually impaired.  
There has a significant research \cite{traffic_google} on traffic light detection in the context of autonomous driving. 
However, in the context of pedestrian navigation by the visually impaired, the existing research is limited. 

Traffic light detection during pedestrian navigation is more challenging compared to autonomous driving because of several reasons. 
First, unlike a car mounted camera, the movement of a handheld or body-mounted camera is shaky due to natural walking gaits. 
%This can result in periodic blurred frames and motion blur. 
Second, the viewpoint of a pedestrian's camera can move a lot compared to the straight viewpoint of a camera on a vehicle.
Third, the traffic lights are mostly perpendicular to a vehicle, where they are angled for a pedestrian. 

In this work, we address these challenges and build a system to detect and recognize traffic light and sign for pedestrian navigation.
Here, we combine sensor fusion and image processing to efficiently detect traffic lights. 
The benefit of sensor fusion is two-fold.
Sensor fusion allows us to process only a subpart of a video frame.
We can predict where the traffic lights will move in a video frame using the estimated motion and orientation of the smartphone and can process only that region of the frame. 
This results in an order of magnitude improvement in image processing time on average. 
Additionally, traffic light detection in a sub-image reduces the false positive rate since spurious detections are eliminated from the regions of a frame where the traffic lights are not located.
In addition to the traffic light detection, walk sign recognition is important for pedestrian navigation.
Here, we use a neural network classifier to recognize walk signs with high accuracy. 

The primary contributions of this work are:

\begin{itemize}
\item Computation time improvement by a factor of 10x (on average) for traffic light detection using the inertial sensor hints. 
\item Accuracy improvement in traffic light detection by processing only the relevant region of a video frame. 
\item Experimental study of the sensor fusion for the reliable estimation of a smartphone's pose. 
\item Efficient heuristic filters for reducing the false positive rate in traffic light detection.
\item High accuracy walk sign recognition at video frames using a neural network classifier.
\item Evaluation of our system with real-world traffic light video in various lighting conditions. 
\item Evaluation of our system with other public datasets.
  
%% \item Our system uses the sensor hint to improve the computation time. We can improve 10x computational time.
%% \item We use heuristic filter to check the traffic bulb in a black box that improve the accuracy.
%% \item We experiment and implement our system in the real time.  

\end{itemize}

The remainder of this dissertation is structured as follows.
We discuss the background of inertial sensors of smartphones, image processing for traffic light detection and neural network classifier for walk sign recognition in chapter \ref{c:background}.
In chapter \ref{c:relw}, we discuss the state-of-the-art work related to our system.
We describe our system architecture in chapter \ref{c:system}.
In chapter \ref{c:evalu}, we present the end-to-end evaluation.