Choosing Between MFnMesh::boundingBox + MItGeometry and MBoundingBox + Custom Octree for Paginated Large Mesh Queries

Answer

For a mesh that is static or changes infrequently but is queried many times with varying world‑space boxes, the MBoundingBox‑based spatial index (e.g., an octree) combined with a pagination wrapper yields lower per‑query latency and deterministic paging, provided you can afford the one‑time build cost and extra memory.

Confirmed facts

• MFnMesh::boundingBox returns the object‑space AABB of the whole mesh; to test a world‑space box you must multiply that AABB by the mesh’s inclusive transformation matrix on every query.
• MItGeometry walks vertices, edges, faces or UVs with a fixed stride; each traversal is O(N) and does not expose a page number or offset, so the caller must count results and sort them if a deterministic order is required.
• MBoundingBox is an incremental accumulator; when used to build a custom octree (or kd‑tree) the construction is O(N) and once built a spatial query is O(log N + k) where k is the number of reported vertices.
• The octree duplicates vertex references or stores additional index arrays, increasing memory usage roughly by a factor of 1.2‑2× depending on depth.
• If the mesh is deformed, the octree must be rebuilt or updated; otherwise the MFnMesh+MItGeometry path needs no extra structures.

Steps for the recommended approach (octree + pagination)

1. Build the octree once from the mesh’s vertex positions in object space (store the vertex index in each leaf).
2. For each world‑space query box, transform the box into object space using the inverse of the mesh’s inclusive matrix (or keep the octree in world space and update it when the transform changes).
3. Traverse the octree: if a node’s AABB is completely inside the query box, add all vertex indices in that node to the result list; if it is outside, skip; if it partially overlaps, recurse to children.
4. Implement pagination by limiting the number of indices added per call (e.g., pageSize) and storing the traversal state (a stack of node iterators) between pages so the next call continues where it left off.
5. When the mesh undergoes a deformation that exceeds a tolerance, rebuild the octree; otherwise reuse the existing structure.

When to stay with MFnMesh::boundingBox + MItGeometry

• Query frequency is low (≲5 per frame) and memory is constrained.
• The mesh changes every frame or you cannot afford the build time.
• You accept O(N) per‑query cost and can manage pagination by counting results and sorting them after each traversal.

Missing diagnostic detail

To finalize the recommendation, please specify the expected query frequency (e.g., average number of bounding‑box queries per frame or per second). This determines whether the amortized build cost of the octree is outweighed by per‑query savings.