Roadmap

Points of interest

A point of interest (POI) is a named place in a scene: a door, a window, a bed, a wardrobe. We place POIs on the same 10 cm grid that already drives collision and the live map, so each one has a position the viewer can stand at and a label it can show. The live map and the 3D view will mark them. Clicking a POI will move or guide the viewer to it.

A* route finding

With POIs on the grid, a route between two of them is a shortest-path search over the walkable cells. We use A*, because the grid is small, the cost of a step is uniform and a straight-line distance gives a good heuristic. The route is drawn on the live map and as a line on the floor in the 3D view. The same grid that stops the viewer at walls also keeps the route inside walkable space, so a route never goes through furniture.

Stronger collision

Collision today is a 2D grid built from two sources: the camera path and the trained splat. Where the two disagree, the splat map can only add obstacles. We want to treat disagreement as a signal, and to record ceiling and floor height so that low overhangs and stairs can be represented.

Known limitations

• Ceilings and unseen areas are rough or dark because the camera barely saw them. A second pass pointing at the ceiling is one way to fix this, but it has to be merged with the first walk as a separate experiment.
• Collision is two-dimensional. It stops the viewer at walls and furniture but does not model stairs or overhangs.
• Scale is assumed from a 1.4 m phone height rather than measured from a reference object.
• Quality depends on the capture. A fast or non-continuous walk reduces the number of frames COLMAP can place.
• Training needs a large GPU, about 16 GB at full 1080p. We did not train on a smaller card.

Other open items

• Re-capture in 4K for sharper frames after downscaling.
• A demo video of the pipeline and the viewer.
• Running the one-machine path, python run_all.py --test, on a clean GPU machine.