offsets rather than absolute pointers

So, bugger me, been off work for a week and a half and only now finally had time to think a bit about something which has been on my mind for agggges – using offsets rather than absolute pointers.

The idea is to make it possible to share data structure instances across processes without having to have the same virtual address base, and between the kernel and user-mode.

So, having thought it over, there’s an obvious limitation; we can only use entities (structures, for state and elements) which are in the same single contiguous memory block.

Consider; say you allocate one block of memory, for a freelist state and a hundred freelist elements.

You share this between two processes. They each now have a different virtual address (first is at say 10,000, the second and 10 million), but the whole block is contiguous from that address.

The freelist internally uses offsets; so both processes can push and pop from the freelist and everything is fine.

Now let’s say the freelist elements each contain a btree element.

We then share the btree state between the processes.

Now we pop a freelist element and we want to insert it into the btree.

The problem is the btree is also using offsets; but the offset from the btree state to the freelist element *is different in each process*, because the freelists have different virtual memory addresses. (Also, ptrdiff_t is only allowed in the spec to show the difference between pointers in the same block of memory – in part most likely because of situations like this!)

So we simply *cannot* use anything other than those things which are in the same conitiguous block of memory.

So if we had one block of memory which contained the btree state, the freelist state and the freelist elements, then we’d be fine – so it *is* still useful.

But one thing we do lose is the ability when a freelist is empty to do a malloc and throw some more elements on there – because they will be in a different, non-conitiguous block.

One thing to also consider is the void pointer of user data. What this is set to is entirely up to the user, but the user faces the same problems as the data structure, so they will also need to use values which make sense across processes and/or kernel/user-mode.

So I’m working now on an experimental freelist with offsets.

Pretty cool if it works – you can take an instance of a data stucture, and use it concurrently over multiple kernel threads at the same time as you use it with multiple threads *in multiple different processes*.

So much to kernel/user-mode isolation… =-)