Real-Time 3D Localization

After a month's hard work, I've implemented a fundamental framework of real-time 3D localization system, which combines object recognition, feature tracking, and 3D pose estimation. The whole system grabs video stream from a single USB camera, locates the visual landmark if present in current scene, and calculate 3D pose of camera's trajectory based on landmarks. One of the fantastic characteristics of the system is its real-time performance and tolerable accuracy of positioning.
So far as I know, another similar real-time localization system is Davison's MonoSLAM introduced in the previous articles in my blog, which based on smooth motion model and active vision. It's a great work and integrates many good ideas, such as active search based on information matrix(deduced from correlation matrix of EKF with Shanon's Information Theory), invert depth, and dimensionless recovery of the scene based on SFM. In fact, I've been inspired by Davision and found an alternative to real-time 3D localization, or tracking and recognition.
However, current system is constrained to some propositions and conditions. Much work has to been done before its practical application.
Experimental results will be given in a few days after I clean up the codes and fix up some bugs.

Below are two video demos for this tracking and localization system, running on P4 2.8G, due core CPU, Winxp.
I. Real-time tracking based on object recognition-I
This video demo illustrates the robustness of the recognition system which is invariant to affine transformation and scales, partial occlusion.


II. Real-time tracking based on object recognition-II
This video demo illustrates the robustness and real-time performance of SIFT-based tracking system with a single USB camera. First, the recognition system finds the target of interest; then the tracking system tracks the SIFT features and locates the target in the scene. Targets missed during the tracking phase can be re-found and re-located by the recognition system.