Unspecified tensor ordering in torch.distributed.all_gather causes silent aggregation failures

PyTorch's torch.distributed.all_gather collective returns tensors from all ranks, but the order of the returned list is unspecified. This is a documented failure condition for any workflow that assumes a fixed rank-to-index mapping, including deterministic gradient aggregation and parameter synchronization across processes.

Because the ordering is not guaranteed, two identical distributed runs may produce logically equivalent but differently ordered outputs. The unresolved design decision forces developers to choose between relying on implicit backend behavior, attaching explicit rank metadata to each tensor, or restructuring collectives to avoid order-dependent logic entirely.

Given that the function does not promise a stable ordering, what is the safest pattern for interpreting gathered tensors deterministically? Do specific distributed backends or PyTorch versions provide stronger ordering guarantees, or must applications always treat the output as an unordered collection?