As some other researchers may be interested in Davison's MonoSLAM, and just like me, want to experience the demo under both windows and linux with USB Camera support. So at here, I took down some of the key tips of my porting experience.
1.USB camera support
Davison's original demo only support 1394 camera under linux, but I didn't have one. So first I added USB camera support in order to experience the demo.
1.1 Under linux -- V4L2 Support
The following work was done under ubuntu 6.10 & 7.04(gcc-4.1.2), and you many have to change it for some other linux versions.
Download v4l2 driver according to the kernel version and install it properly. Try spacaview(availabel on the same website) to test the USB camera. A list of supported USB camera can be fond here, which includes almost all the popular USB camera.
Spcaview is a smart but powerful tool to control the USB camera under linux, and it provides a good encapsulation to V4L2 APIs. I extraced some of the source code and built them into a library(libspcav4l) for easy usage in other applications. My transported project uses this library too.
The VW34 library used by MonoSLAM provides an abstraction to the manipulation of image source, either from file sequences or video capture device. Its general mechanism looks like this :
1) A looper runs continuously on the backgournd to read image file or capture video data from the caputre device. Once it gets the image data, it then notify the upper layer(image processing layer) to fetch the data.
2) The upper layer responds to the notification from the lower loop and process the image data in a callback function registered during the initialization phase.
One of the drawback of this mechanism is that, the looper runs continuously and takes up a lot of CPU resources. In fact, the approach adopted by windows' WDM driver is much more reasonable and elegant : whenever the video data arrives, WDM notifiles the upper layer to fetch it. That is a interruption approach, not loop approach.
The related code can be found at VW34/VWGLOW/Interface folder. Refering to VW34/VWFirewire, I added an interface named VWUSBCam, which calls functions in libspcav4l and added VWUSBCam interface to MonoSLAMGlow/monoslamglow*. Now the USB camera works for MonoSLAM!
1.2 Under Windows -- DirectShow Support
DShow provides a routine to easily manipulate the usb camera. Same as under linux, I added usb camera support with DShow to VWUSBCam interface.
2. Poring to Windows
It's usually troublesome to port a linux application to windows, especially when the application uses many libraries under linux. Fortunately, Oxford's VW34 is designed to be platform independent, and that saved me a lot of time. monoslamglow and scenelib are purely coded with c++, which is also cross-platform.
STL can be seen everywhere in Davison's MonoSLAM. This is good, for STL is efficient and elegant. But some fatal bugs lurk in the code due to the misusing of STL, for example vector::erase. Scott Meyer's Effective STL is really a good reference to write both effective and robust code with STL.
As VC6 does not fully support templates, so I chose VS2005 to compile the projects. Some additinal libraries should be prepared first. They are :
glut-3.7.6 : opengl wraper
wxWidgets-2.6.4 : wxWidgets support
Both the two libraries are platform independent and you can get them freely.I compiled and build them into static libraries.
other libraries used in MonoSLAM :
glow_104
scenelib
vw34
All the libraries are compiled into static libraries. The compiler may complaint "unresolved link" or some other errors due to the project setting or syntax problem. Just follow these prompts and fix them one by one.
What I want to mention is that, VW34 wrapps VNL which is part of VXL library, and after successfully compiling of the source code, plenty of "multiple symbols error" prompts may jump out while linking. I checked the VXL's FAQ and found the solution :
change the project setting in VS2005 as following :
Project->Settings->C/C++ Tab->Category: Code Generation->Use run-time library: Multithreaded DLL
Now I have a copy of MonoSLAM runs on windows with USB camera support. Next I fixed some treacherous bugs mainly due to the misusage of STL.
3. Fix bugs
Here I listed several most treacherous bugs of MonoSLAM. The amazing thing is that, these bugs keep quiet under linux, due to difference between gcc & vs2005 compiler, as well as the difference between implementations of SGI's and Microsoft's STL.
But, in my view, they are really bugs, and linux just "hides" them! So I'll list them up and give the solutions respectively.
3.1. Multiple Inheritance
The first problem encountered is that, once the main GUI window redrawed, it crashed.
It is very curisou, and may be cased by the bugs in wxWidget, opengl, glut or glow. And I had on idea of it ~~
As usual, I debuged the code, and looked deep into the whole mechanism of the paint procedure of monoslamglow. Same as other GUI applications, the mechanism is message driven and call a callback function to redraw the window. After carefully tracking all the variables, I found the handler of the callback object changed during the process which crashed the application!
The inheritance tree looks like this :
MonoSLAMGlow
<-- GlowWindow <-- VWEvents::EventHandler <-- GlowPushButtonReceiver <-- GlowCheckBoxReceiver Multiple inheritance is complicated, especially when vitual inheritance is involved. You can get a idea of how complicate it is from Lippman's
All the four base class has virtual functions. The handler to callback object invalidated when converting to VWEvent::EventHandler polymorphicly. Lippman pointed out that the polymorphism of c++ itself is very complicate which involves adjusting pointer's address and the actual implementation is compiler-dependend.
I found that the adjusted address of pointer to the derived class had an addrss offset compared to its actual address. So, I adjusted the inheritance order of the base class, that was, put VWEvent::EventHandler at the first place, and the GUI application worked fine!
3.2. STL's pitfalls : vector::erase
Scott Meyer pointed out some dark corners of the c++ language and STL's pitfalls in
After erasing an object from a vector, all the iterators previously pointing to the object in the vector will be invalidate.
The above rule was emphasized again and again by Meyer. Though simple, yet easy to fall into this pitfall, as MonoSLAM did.
At first, MonoSLAM crashed for the error "incompatible iterator!" after running for a short while. Curious error, isn't it ? Experience told me that something may be wrong with stl's container, such as vector, map, set, list, etc. But locating the error was rather difficult because stl's container appeared everywhere in the code. After a careful debugging and scrutinizing the code, I found following code :
for(vector::iterator feat = feature_init_info_vector.begin();
feat != feature_init_info_vector.end();feat ++)
{
if(feat->...){}
else{
delete_partially_initialised_feature(feat); // call vector::erase inside
}
//...
}
similar codes could be found at other places.
It seemed pretty good from logical sense: go through all the objects in the vector and delete unqualified ones.
But it was wrong, according to the above rule!
Once feature_init_info_vector deleted an object, it would reallocate memory and copy the remaining object into new memory. So the feat iterator invalidated after calling delete_partially_initialised_feature, in other words, feat pointed to a old, invalidated memory object, not that in the new allocated feature_init_info_vector! That's why the application complained for "incompatible iterator"!
The solution is very easy, with Meyer's advice, return a iterator pointing to the new allocated feature_init_info_vector and assign it to feat. Ok, that's done !
4. Conclusion
Above is my experience with porting MonoSLAM to windows with USB camera support. Currently, it works well under both linux and windows. There may be some other hidden bugs, I'll fixed them when they emerges.
