Update / I’m sad

I am sad.

This is because I think I need three variants of every data structure.

The first is the normal variant – pointers are what they are, absolute values (virtual or physical as they may be).

The second is the single segment variant – pointers are offsets, and all data must be within a contiguous range which is half the address range; the only way the user can guarantee this is to make a single contiguous allocation. So here we’re looking at shared memory, or buffers passed down into or up from the kernel, that sort of thing.

The third is the multi segment variant – pointers are offsets, and we keep track of the base address of each segment, and that has to be done on a per-address range basis (so each process needs to register each new block of shared memory, to indicate what the base address is in its virtual address range). However, since all offsets are positive, we can use unsigned (and we have to anyway, since we borrow some MS bits for a segment number), and so we support the entire address range. Downside is the per-address range state, which the user has to set up and add segments to when they are made, and the overheads (array scan) in the data structure to figure out which segment a given pointer belongs to. Oh also actually since we borrow MS bits for the segment number, the more segments you want to support, the smaller the maximum address range for each segment. Again, this means users have to allocate blocks.

The third variant supports what the second variant does, but the third variant I think it primarily for NUMA on Windows – not for everyone and their dog – and the second variant is *so* much easier to use and the single segment use case is going to be common, shared memory between processes, so it’s worth having that variant.