The last two weeks have been an attempt to get PCL doing a good chunk of what I want it to. Sometimes succeeding more than expected! Othertimes, notsomuch. PCL is a powerful tool, but its development focuses more on its tool abilities than on having tutorials for how to get everything working right. Even so, some real progress! And so far we have point clouds individually importing and constructing some surface meshes too.

Above: How I measure trustworthiness.

PCL Installation

I’ll admit: I totally gave up. The proper way to install PCL for Visual Studio 15 is to download the source yourself along with all the dependency binaries and build what you need to locally, and that was my first attempt. I got right to work on that, and found that the third party binaries were also only for older versions of VS, so I had to build those too. I started then with Boost, which required being built in cmake. Then I learned that the Boost-cmake project lost support a while back, and that now there is no “best” way to build Boost with cmake anymore. And… suddenly, the alternative might be more reasonable than the correct approach.

So I did a quick safety check on the background of author who built these prebuilt VS15 binaries, going by the name “カテゴリー” (translated in the comments and sidebar as Tsuksa Sugiura), and found that this programmer actually has a steady background on GitHub contributing to Kinect projects. Given the consistency of their work in the field, publicly over the last year at least, then either this is a very in depth long con - or the binaries should be good. So far, the results are looking good and working, but I wouldn’t take my word on it alone either.

After that, PCL was installed on my computer for VS15, and once all the third party libraries were there and system variables tied in, most of the examples just worked!

Here you can see in PCL_Viewer my PCD, with the “black” curtain hanging behind the blue chest, and the “light blue” rug on the ground.



PCD File Format

First things first, even before integrating the code into a C++ project, I wanted to test out PCL’s example exes. The library really only works with the PCD file format, so even testing things like PCL_Viewer meant converting my .off files. Surprisingly, a fairly simple task! The two are almost the same, with slightly different headers. Unsurprisingly: it still came with complications. Colors are not stored in a traditional 255 r/g/b format in pcd, but rather, converted into a single floating point type that uses smaller variables to compose it. And even then, my attempts at conversion did not seem to fully work. But if you are not looking to use those colors long term, the conversion is fairly quick and easy.

Me demonstrating my excellent abilities in “complaining on the internet” about the colors not working…

PCL Community

So when you have problems, communities usually help! The mailing list archives (forum equivalent) have been fantastic for finding existing answers to questions I have via google, and while signing up to post is a bit spammy since you get a new email for everyone else’s new posts, the community is very alive and welcoming. I got a couple replies within 48hrs to my post complaining about PCD’s color system even, so at the least, it’s more active than any subreddits I’ve seen on the subject matter - not to mention being the hub for PCL itself. Just no entertainment here, it’s more like StackOverflow: all questions and answers.

On the left, Marching Cubes. On the right, Poisson reconstruction.

Surface Reconstruction

The next step then was attempting to create a mesh. Poisson reconstruction is currently succeeding where preliminary attempts with marching cubes have failed. The biggest issue here was switching from PointXYZ clouds to PointNormal clouds - surface reconstruction requires normals (direction the polygon faces) to work. There are tons of papers out there on normal estimation, but somewhere I saw a neat trick in MeshLab that seemed to work here too (for a quick/dirty approach at least) of having all normals point towards/away from the rgbd sensor (aka our Kinect). So I tweaked my off2pcd converter again, and resaved with every point getting a 0,0,-1 normal (positive 1 had the polys facing the wrong way for my needs). After that, my lumpy cloud world version of Beautiful Corner popped into existence, marking the biggest progress point of this post.

When you reconstruct three overlaid clouds, but they failed to align first…

Registration

Registration then is the lowest point of progress, as ICP (iterative closest point, to get multiple clouds aligned) is not working as expected yet. Similarly, posts for this one so far on the community haven’t been as helpful as I’d hoped, but I think I just need to do a deeper dive on the subject matter. Hopefully we can expect something working here next week! But for now I can say that ICP needs a little more than just plug and play. I saw MeshLab videos that similarly suggested roughly aligning clouds before attempting ICP, so now I need to find a way to make that work here too. Currently, trying ICP with inconsistent point clouds and significant differences does not seem to work, but that is a problem for next week.

In Conclusion

Most of my progress these last two weeks came out of following the example of the “tools” folder in PCL. There are an amazing number of example exes in there which all show the various functions of PCL working on your computer, and similarly, they are all open source. This makes them a great reference point if you learn by example, and much of my progress came from mixing and matching various tools with my code. Next week I’ll be diving in after ICP again as I try to automate that alignment (which I know is totally possible) and try to use some estimation functions before ICP. Oh, and maybe I’ll try to break up my blogposts a bit instead of semi-weekly behemoths like this. I can dream, at least?