123D Catch uses several file formats in it's cloud including .obj paired with .mtl and .jpg files, which we can (most likely) convert to be displayed on out holomonitor.

An idea was brought up of being able to take a few pictures of an object from different perspectives and through some automated process projecting a hologram of that object on the holodisplay. After a bit of research we saw a program created by Autodesk called “123D Catch” which is capable of producing 3d models from collections of pictures. I investigated to see if we could understand the files produced by this program and convert them for display on the holomonitor.

123D Catch is mostly on the cloud. The user takes pictures of the object he or she wishes to model from multiple perspectives and then uploads them to Autodesk's servers which then produce the 3d model just from those pictures. Too few pictures, and the model is distorted as seen below:

scanned:

original:

This was done with only around 10 pictures. Others have taken several more than this and obtained much better quality renderings. Requiring so many pictures (and at better angles than I took) is one limitation of this process. Another is the element of time. After uploading the images to the Autodesk servers, it takes time to complete the render. My guess is probably at least 10 minutes for something like what I've done, but more likely about half an hour. Further testing should be done on the time it takes to render. A further limitation lies in the fact that Autodesk forces users to create a free account with them specifically for the 123d suite before allowing you to upload images or download renderings (even of otherc' creations.

After the model is finished rendering, it can be freely downloaded in one of 3 forms.

An stl file is solely the 3d model without any texturing at all. Example:

nice and simple, but not what we want. We want Color!

A proprietary file made by autodesk, this contains a lot of information. Unfortunately, it's completely binary, and would take quite a bit of time to crack enough for use.

An open file type that's become a standard throughout the 3d modeling community. This type is probably our best bet for our use. It comes together with a .mtl file and a few .jpg files.

obj files can vary significantly between locations but those created by autodesk are pretty formulaic. They start with some comments prepended by “#”. Then they declare their material library or reference their mtl file by saying

mtllib mesh.mtl

(Note the obj file will always be named mesh.obj, and the mtl file will always be named mesh.mtl).

After that declaration the file then declares a collection of vertices in x y z format:

v -1.25345 1.75660 200.000003
v some_float some_float some_float
...

When all these vertices are declared the file will go on to declare all the texture vertices in u v w format:

vt .23456336 .4456773 0
vt some_float_between_0_and_1 some_float_between_0_and_1 0
...

These points will be a percentage of the x or y lengths of the texture needed. The w in this case will always be zero if created by 123D Catch

Once the texture vetices are finished, the file then declares faces, but before that it declares what material it will use for the following faces. This references information in the .mtl file.

usemtl Texture_i

where i is a reference to the jpg file (which are all named tex_i.jpg where i is a number from 0 to the number of jpgs created - 1.)

then it declares it's faces:

f 7354/81583 7355/81584 7356/81585
f index_of_vertex/index_of_texture_vertex index_of_vertex/index_of_texture_vertex index_of_vertex/index_of_texture_vertex
...

This binds three vertices previously declared to three texture vertices previously declared using the texture referenced by the preceding material. This essentially creates a triangle using the vertexes and image vertexes that we already defined. When we finish using the images defined here, we move on to the next material by referencing it in the same way as above.

Now some parts of the meshes don't use the images at all and they reference their material as so:

usemtl Untextured

then proceed to define their faces using only the vertices defined initially and NOT the texture vertices.

f 2155 2240 2031
f some_vertex_index some_vertex_index some_vertex_index
...

Untextured is a material declared in the mtl files just as the Texture_i materials are. Also remember that these values used in declaring faces are indices of vertices we already declared.

That ends the OBJ files.

The MTL files are a little bit easier.

MTL files are material library files. They have the same content repeated over and over for each material type except for the change of the jpg which is referenced for the texture.

Each material starts with a declaration:

newmtl Texture_i

Replacing the i with a zero ordered index identical to the desired jpg to be used for the texture of this material. After that each material (except the last “Untextured” material which we'll explain later) has the following lines:

Ka 1.000000 1.000000 1.000000
Kd 1.000000 1.000000 1.000000
Ks 0.000000 0.000000 0.000000
d 1
Ns 0.000000
illum 1
map Kd tex_i.jpg

Again replacing i with that zero ordered index of the jpg files rendered by 123D Catch. Although, I have reason to believe this information won't change, it's good to know what the lines of this file mean: Ka refers to the “ambient color” and requires a color as three floats between 0 and 1 in rgb format. Similarly, Kd refers to the “diffuse color” and requires the same. Ks, however, refers to specular color and has the same requirements but is always set to black in the case of 123D Catch created files. “d” here refers to the dissolve or opposite of transparency. This is always set to 1 or completely opaque. Ns is the specular exponent for specular color and is always set to zero as far as I've seen from 123D Catch. illum refers to the Illumination mode. There's a list of these on wikipedia, but suffice it to say 1 refers to having both color and ambient light I believe. Most importantly, map Kd says to set the diffuse color to the texture in the jpg as seen. This is what allows the OBJ file to reference the points on the jpg files.

This form is repeated for however many texture jpgs were created, and then the final set is for Untextured and takes the absolute form as such:

newmtl Untextured
Ka 0.501961 0.501961 0.501961
Kd 0.501961 0.501961 0.501961
Ks 0.000000 0.000000 0.000000
d 1
Ns 0.000000
illum 1

Notice this sets Ka and Kd to a light-ish grey. And does NOT reference a texture jpg (hence Untextured).

That's it for those file types. Other than the OBJ and MTL files there are a number of jpgs created by 123D Catch as referenced in the MTL file.

As far as programmatically using 123D Catch, I'm uncertain if they have an API of some sort or not . . .