Well I heard back from my Industry adviser! As always Eric provided me with thoughtful and informative advice. I won't spell it all out here, but basically these are the main points in regards to what a TD is up against.
- Working with a system that is already in place before you get there.
- Facial rigging in relation to facial motion capture.
- Kinect as a motion capture device.
- Lack of tools for common rigging practices.
Armed with this insight I went back to my faculty advisor for some more directing to my study. The following discussion covered topics about pushing the industry, challenges of differentiating oneself in the digital world and how to leverage the tools that are out there in a continually shrinking world. After all this I found myself no closer to starting those python scripts than I was last year but yet inspired to try something risky. Something that interests me but may not be what I originally set out to do.
That something is currently looking into alternate film making and generating 3D assets. Specifically I am now looking at 123D Catch, an automated 3D mesh generation tool from Autodesk, and the RGBD toolkit, an open source software package for capturing 3D footage through a Kinect and a DSLR. While these technologies don't seem to be abel to be easily integrated together, they do share a common thread of 3D asset generation which I want to explore.
My initial attempts at exploring these technologies was met with difficulties. Firstly lets look at 123 Catch.
Being an automated service from Autodesk I assumed that any problems I came across would be my own doing, and boy was I right. 123D offers a number of ways to interface with the service, online, downloaded able app and even comes in Windows and Mac flavours. 123D basically allows for fast 3D asset generation by taking a series of photos of an object on a turntable. The user then uploads these photos and is shortly returned a mesh that they can modify, 3D print or even export a template that allows for cardboard to be cut out and stacked forming a rough 3D shape. Pretty neat. So what did I get when I did it myself? Behold!
So that's me, in the new green room at the CDM. There are a number of problems with this capture and they all stem from my own unfamiliarity with the software. Firstly wrong is that the focus of the camera was less than perfect. This generated fuzzy edges that 123D would translate into those large swaths of extraneous green around the model. The perspective was also problematic as is evident in the legs of the model. Since the position shifted around in the models legs I assume that 123D had difficulty deciding on how to build that whole area.
My initial 123D attempt was not clean and could be way better but it was still a success. With this model I was able to download a .obj file and import it directly into Maya, my favorite 3D software. This opens up a whole realm possibilities by quickly digitizing anybody here at the school and animating them. I would just need to setup a series of automated scripts to rig a bi-ped and perfect my image capturing method. Exciting!
123D is neat, but it's not the most exciting thing I'm trying for this directed study. Larry also turned me onto the RGBD toolkit which would allow for me to generate 3D captures of footage. So I spent some time trying to get that system up and running.
First the good news. RGBD toolkit is open source and runs on both Mac and Windows, giving me options for when things don't work. Next the bad news, RGBD toolkit doesn't want to work right out of the zip, it requires a DSLR glued to a Kinect and requires substantial knowledge of drivers and troubleshooting.
Once I had managed to secure a DSL from the school I then had to get the software running. My first issue was a missing .dll. Fine, that kind of things happens all the time and I tracked down the offending file and got it installed. Great so now RGBD toolkit can see my Kinect device. However I found that once RGBD is talking to my Kinect, Unity lost the ability to. This is because RGBD toolkit wants a particular driver for the Kinect Device while Unity and the Kinect SDK are looking for another. This is a problem because a major project I am also doing at school this term requires me to work with Unity and a Kinect. For now I can solve this problem by rolling back to the previous driver through the device manager, but this is not an elegant solution. I'ved started looking into writing my own drivers so that both pieces of software can see the Kinect at the same time, but this is proving to be another major undertaking since drivers cannot be 'hacked' together and require a deep understanding of low level computer workings. So for now I'll be toggling drivers back and forth.
Great so now I'm thinking to myself "RGBD sees the Kinect, let's get this show on the road!" Then I ran into my next problem, the toolkit crashes when attempting the first capture. That's a big WTF buddy right there. So I started troubleshooting the install, the connections and the method that I was attempting. I found that I may have gotten the wrong kind of DSLR for this experiment, as RGBD toolkit requires one that is able to capture video and not just stills. While this may or may not be related to the crash bug that I'm experiencing it is a factor in the equation that is not aligned to the developers instructions.
After all this trial, error, success and failures I have outlined a new next steps that I will be attempting in the coming weeks;
- Secure a DSLR that supports video capture, try the toolkit again.
- See if I can substitute a video camera for a DSLR if once cannot be found.
- Intall the toolkit on my Mac to see if I can get better results.
- Perfect my image capture techniques and generate another 123D Catch model.
- Clean up said model and attempt to rig it with a basic biped rig
- Ask Eric about the possibility of putting animation keyframes gathered through a Kinect onto a model generated by 123D.
Ok so that's it for now. Much more to come. How I wish I didn't have to eat or sleep or do anything else but plug away at these fascinating problems.
