Optimizing Photogrammetry for Sketchfab
Making your 3D scan post look great
Intro
Being a Photogrammetry artist and posting on Sketchfab for quite a few years, I’ve developed a workflow for uploading pieces to Sketchfab and making them look great.
This post covers:
- Cropping your scan to have clean mesh borders.
- Optimizing for Sketchfab upload limits (and performance).
- Adjusting Sketchfab’s 3D settings to make the viewing experience great.
This post assumes that you know the basics of capturing for Photogrammetry and using Photogrammetry processing software. For a more basic guide, refer to my Getting Started with Photogrammetry post.
Gear used in example scan
Camera:
- iPhone X
Software:
- Reality Capture for Photogrammetry processing (Free to use, pay once to export).
- Meshmixer (free) for cleaning up the mesh.
Cropped Mesh Borders
Typically, the raw output from most photogrammetry software will look something like this:
It includes all the floating triangles or the marginal triangles that Photogrammetry software produce during reconstruction (AKA Poisson Surface Reconstruction).
I recommend cropping meshes in an external software like Meshmixer that include a crop+remesh functionality to get beautiful, straight, cropped borders like this:
My workflow here is:
1. Simplify the mesh in the Reality Capture (or your Photogrammetry software of choice) to the final poly count before doing the crop.
In this case, I simplified to 100k triangles for the final output from the initial 15 mil. triangle reconstruction result as the geometry was not very complex.
2. Export from Reality Capture, import the mesh into Meshmixer and crop it to your liking.
I have a video tutorial for this step specifically:
As a side note, the reason I recommend to simplify the mesh before the crop is because if you crop first and then simplify, it leads to results that look like this:
This is a result of Reality Capture not “locking” the border triangles while simplifying a mesh. Therefore, unless you’re using another software for the simplification that can lock border triangles, I recommend simplifying to the end triangle count and then cropping the borders.
Sketchfab upload optimizations
The upload limit per post for a free Sketchfab account is 50 MB, so here are some ways of optimizing your photogrammetry scan to stay under the limit.
The other goal I try and optimize for here is for fast load times and performance on all viewing devices, especially as viewing Sketchfab models is becoming more and more popular on mobile and in AR/VR devices.
It’s worth mentioning that I do recommend the Sketchfab Pro account with a 200 MB upload limit if you’d like to post higher quality scans (both in triangle and texture count/resolution).
OBJ vs FBX
Exporting your scan as an FBX is far better for optimizing for minimal file size. Here’s a comparison of FBX vs OBJ exports for the example scan:
The FBX file format in this case is 5x smaller than OBJ for the same mesh.
Exporting without Vertex Colors and Normals
Not including vertex colors and the normals can further reduce the file size.
If your Sketchfab post will be a textured mesh that is Shadeless (as in, not using any real time lighting), then excluding vertex colors and normals will not affect the visual quality of your post at all.
You can find these settings to excluside vertex colors and normals while exporting from Reality Capture:
Texture file format
From my experience, this will make a massive difference in minimizing the file size of your exports with a minimal difference in visual quality.
Here’s a comparison between the same texture as a PNG (32bit) and JPEG (24bit) file format:
If you’re trying to optimize your photogrammetry post to be under the 50mb limit, going with a JPEG texture becomes crucial. It can also give you the extra space to upload more and higher resolution textures.
In a more professional setting, having the full bit depth of PNG is important. However in this scenario, JPEG textures might be just fine.
Here’s a quick comparison of the same mesh and texture as a PNG vs JPEG. To my eyes, in an shadeless viewing format, I can’t see any meaningful difference.
If I’m missing something important here, please let me know in the comments.
Texture resolution
Typically, your photogrammetry scan at its optimal texture resolution will have way more resolution than you can use. Limiting the texture resolution of your scan (AKA the Texel size) is a compromise that can vary per scan subject.
My rule of thumb for my Sketchfab posts is choosing between one to four 4k textures.
If you have a Sketchfab Pro or Premium account, you have the upload space to have a lot more textures in your post. However, having a high texture count can limit performance on certain devices (on mobile, in WebVR, or on desktop browsers that aren’t used hardware acceleration), so it’s worth keeping that in mind.
Quick note: Reality Capture rarely displays your textured model at its actual texture resolution. A good way to get a preview of what the texture will look like in Sketchfab is by rendering a screenshot within Reality Capture.
3D Settings on Sketchfab
After exporting and uploading your scan to Sketchfab, I recommend having these settings and doing the following operations to your model:
Shading
I always prefer the Shadeless setting for Photogrammetry meshes as it shows the scan in its original lighting condition.
The added benefit of using the Shadeless setting is that it can also hide the underlying mesh topology artefacts as well as hide the low-poly aspect of your model.
It’s worth mentioning that depending on the subject of your scan and its lighting conditions, using the Lit shader might also look good enough. However for my subjects, I always go with Shadeless for a more realistic look.
Background
This setting is entirely dependent on the mood you’d like to set around your model.
For this example scan, I’m using the “Road in Dordogne” environment as the background with the “Ambient Environment” setting checked.
Ground Shadows
This setting is subtle but makes a massive difference visually for “grounding” your model in the scene instead of it floating in a void.
Head over to the “Lighting” tab and expand the “Ground Shadows” setting. Toggle it on and switch it to the Baked AO setting.
Adjust the “Height” setting until it’s just under your model but give it some room so that it looks like it’s floating. You can adjust the other parameters as well to get a look that you’re happy with.
Field of View
This setting will vary per subject and can make a difference in how the model’s size is perceived.
If your scan is of an enclosed space (like this scan of an elevator), then you want your FOV to be quite high (70°–90°). Otherwise, navigating through the scan becomes quite difficult.
If your scan of an object like this car or something similar, I recommend the default FOV (45°) or lower for a more zoomed-in look.
Post-Processing Effects
You can also add post-processing effects. I don’t use them very often but many do.
For example, using the Depth of Field option with a low FOV can give you a Telephoto Bokeh look.
Face Rendering: Double sided vs Single sided
This setting is to toggle whether you can see through a model from its back side.
For the car model, the default option of Double sided makes sense.
However, for a scan of an enclosed space such as this scan of an elevator, I recommend using the Single sided setting to be able to zoom out and see through the obstructing wall.
It can be found in the “Materials” tab and under “Faces Rendering”. If you have multiple textures for a scan, you’ll need to adjust it for every texture individually.
Annotations
Adding annotations is a great way of crafting a narrative flow through your model, even if it doesn’t really have one. It’s also a great way to point the viewer to notice some details that might be missed.
I typically like to use the annotations to give the model viewing a directional flow, like a tour guide walking a visitor through a location.
It’s important to know that each annotation has its own pivot point. As in, when you click on an annotation, it adjust the camera pivot to where it was when the annotation was made.
Knowing that, what I like to do when adding annotations is to make sure the pivot point for each annotation is focused around the annotation itself. What that achieves is that when the viewer is reading an annotation and when they rotate the camera, the camera rotates around the annotation point which helps the viewer understand the annotation even better.
It’s hard to explain it through writing, so watch this video instead:
You can adjust the pivot point by “translating” through the scene (right mouse button click+drag) or by double clicking on the model where you’d like to set the pivot point to. Notice that I do the double clicking outside of the annotation settings in the Post-Processing settings to not create a new annotation instead of adjusting the pivot point.
Save View
I’ve saved the best for last. This setting does a few things:
- Captures a thumbnail for your post
- Sets the initial position of your model when the page loads
- Sets the initial pivot/rotational position of the camera when the page loads
Therefore, this setting can have the biggest impact on the amount of attention your post can draw as well as shapes the experience the viewer can have while looking your model. In the future, I might make a video tutorial for this section specifically as there are many little things that are better explained via video than text.
I typically try a few various views and FOV’s to get to a thumbnail that I’m happy with. In this gif, you can see me iterating through a few perspectives and making micro-adjustments of the composition within the thumbnail.
Hint: you can also rotate the environment behind the model to get a better background using Alt/Command + Left mouse click.
The next step is to save and exit the 3D Settings, and load the post to see how the model loads.
What I’m typically looking at here is:
- From which direction is the camera loading in? Is it conducive to understanding the model and the scale of the scene?
- Is it loading in from a weird angle that doesn’t make sense?
If the you don’t like the angle that it’s loading in from, the way you adjust it is by rotating the model itself in the 3D Settings.
Note: neither the annotations nor the initial loading position rotate along with the model, so you’ll have to set them up again.
- Do a full rotation of the model. Is the camera clipping through anything? Can the viewer get a good sense of the model from a rotation (without zooming in or out)?
These questions help optimize the UX of your model especially if you assume a viewer will only load a model and rotate it without adjusting the pivot point or perspective.
And that’s most of it! Hope this post can help you create and upload beautiful scans to Sketchfab and get eyeballs on your beautiful work.
If you’d like to see the example car scan used above, you can find it here: https://skfb.ly/6YCxY
Check out more of my Photogrammetry work on Sketchfab and follow me via Twitter.
