Photogrammetry for AR: Capturing Real Objects for Mixed Reality

Your AR app needs 3D assets. You could hire a 3D artist to model everything from scratch, or you could photograph an object and let photogrammetry do the heavy lifting. The second approach is faster, more accurate, and increasingly essential for AR experiences that demand photorealistic content.

Photogrammetry—the process of extracting 3D geometry from photographs—has matured significantly. What once required expensive equipment and specialized expertise is now accessible to most development teams. This shift opens new possibilities for AR applications that need to incorporate real-world objects with precision and speed.

The Photogrammetry Workflow

Image Capture

Start with a strong foundation: quality images. You need multiple overlapping photographs taken from different angles around your subject. The exact number depends on complexity and desired accuracy, but typically 30–100 images suffice for small objects.

Key principles:

• Consistent lighting: Avoid harsh shadows and reflections. Cloudy days or diffuse studio lighting work best.
• High resolution: Use your best camera or smartphone. Modern phones capture sufficient detail for most AR use cases.
• Fixed focal length: Zoom in or move closer rather than changing focal length during capture.
• Overlapping coverage: Each image should overlap with adjacent shots by at least 30%.

Processing and Mesh Generation

Photogrammetry software analyzes overlapping images to identify corresponding points and reconstruct 3D geometry. Tools like Agisoft Metashape, Reality Capture, or open-source alternatives like COLMAP process your image set into a point cloud, then convert it to a textured mesh.

The output is typically a

.obj

.ply

bashCopy
# Example using COLMAP (open-source)
colmap feature_extractor \
  --database_path database.db \
  --image_path images

colmap exhaustive_matcher \
  --database_path database.db

colmap mapper \
  --database_path database.db \
  --image_path images \
  --output_path sparse

Optimization for AR

A raw photogrammetry mesh can contain hundreds of thousands of polygons. AR applications running on mobile devices need models under 50k–100k triangles. Use decimation tools to reduce geometry while preserving visual fidelity, and bake ambient occlusion into textures to maintain perceived detail.

pythonCopy
import trimesh

# Load mesh and reduce polygon count
mesh = trimesh.load('raw_model.obj')
reduced = mesh.simplify_quadratic_mesh_decimation(target_count=50000)

# Export optimized version
reduced.export('optimized_model.glb')

Integration into AR Frameworks

Once optimized, your model integrates into AR experiences using standard frameworks. Here's a minimal example with Three.js for web-based AR:

typescriptCopy
import * as THREE from 'three';
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader.js';

const loader = new GLTFLoader();
loader.load('optimized_model.glb', (gltf) => {
  const model = gltf.scene;
  model.scale.set(0.01, 0.01, 0.01); // Scale to appropriate size
  scene.add(model);
});

For native mobile AR (ARKit or ARCore), the workflow is similar—load your optimized

.glb

.usdz

Practical Considerations

Photogrammetry excels at capturing organic shapes, textured surfaces, and objects with fine detail. It struggles with shiny surfaces, transparent materials, and thin features. For those cases, hybrid approaches combining photogrammetry with traditional 3D modeling work well.

At LavaPi, we've found that photogrammetry accelerates asset production significantly when capture is planned thoughtfully. The time saved in modeling can be redirected toward AR interaction design and optimization.

The Bottom Line

Photogrammetry bridges the gap between physical reality and digital AR experiences efficiently. With proper capture technique, realistic optimization, and standard integration workflows, you can build AR applications that showcase real objects at scale. The investment in getting the capture right pays dividends in production speed and visual authenticity.