Update

Well, just done a ton of doc work.

1. ringbuffer APi docs first pass
2. building guide (lfds) first pass
3. building guide (test) first pass
4. porting guide (lfds) first pass
5. porting guide (test) first pass – but realised the in-page examples are too long, have to come at this in a different way

All the API docs need a second pass now to correct mistakes and make them uniform, then a third pass to actually wrote notes and content – right now it’s just prototypes and arguments, basically, the framework of the page.

Got a long list of minor code/doc fixes/improvements to make, which has built up as I’ve been going along. Have to do all those and reflect them in the docs.

Then go over the non-API docs for a second pass.

Then I can bring all the build files up to date and run test on all platforms.

Then I can release – and then immediately start working on the next release, to get the SMR code back in there and out into the field. I’d like to get the benchmark programme out too.

ringbuffer

Moved over to new API design.

Made the tests pass.

I’ve updated the ringbuffer API page to the new API design. Will write the per-API docs tomoz.

ringbuffer API design

I’ve been unhappy with the ringbuffer API. It’s totally unclear how it’s supposed to be used.

The existing API looks something like this;

get_read_element()
put_read_element()

get_write_element()
put_write_element()

Great, huh? obvious – not.

Naively, I think the expectation is for something like this;

ringbuffer_read()
ringbuffer_write()

read() get the oldest unread element, write() writes to the ringbuffer, overwriting the oldest element if there are no free elements.

The problem though, which led to the existing API, is that the ringbuffer is lock-free and so multi-reader and multi-writer; as such. if we read an element, someone else can completely over-write its contents WHILE WE’RE READING IT.

As such, the existing API acts when reading to *detach* the element from the ringbuffer, so no other thread can touch it, and the same when writing; we detach an element, write to it, then place it back into the ringbuffer.

In fact I’ve realised this problem/behaviour all turns upon whether or not the ringbuffer is beig used to store transient or permanent data. What I mean by this is that we arrange the ringbuffer either so that each element is pointing to some permanently allocated store for that element (like an audio buffer) or we arrange the ringbufer so that each element is holding a void pointer, which is valid for one read (so we have say a freelist of audio buffers, and we write into the ringbuffer a pointer to a filled audio buffer we’ve obtained from the freelist).

If each element has some store permanent associated with it (this being done typically when the ringbuffer is initialized) then we have the problem earlier described; if we read by just getting a pointer, which is to say obtain a pointer to a filled audio buffer, another thread could perfectly well write into that buffer as we’re still using it. So we need to detach the element from the ringbuffer, which gives us the get_read_element() / put_read_element() style semantics.

OTOH, if we have a freelist of audio buffers, and we take a buffer from there, fill it, then put a void pointer to it into the ringbuffer, then the act of reading is the act of taking that void pointer and the reading thread can then do something with it and when it’s done, put it back on the freelist. With this arrangement, we can have simply read() and write() – write will as in the previous paragraph overwrite the existing void pointer, but that’s fine; that’s not going to interfere with anyone who’s already obtained a copy of that void pointer. (One thing we do need to look out for is when a write to the ringbuffer overwrites an exiting element, where the ringbuffer is full; the writer must get back the void pointer he has overwritten, so that the user can do something with it – such as returning it to a freelist – rather than it being lost).

In the spirit of having one thing perform one task, the ringbuffer shall now have read() / write() semantics, and the user must look after his pointers.

Update

liblfds and test compile again, libcommon has been deleted. Will need to update all the build configuration of course. I’ve been building up a list of small things in the code which I need to fix/improve. Will get them all done before updating the build configurations on other platforms.

The build/usage/porting doc layout is now healthy – the layout makes obvious sense (I hope anyway!) to people coming to it for the first time.

Wrote the first pass of the btree docs.

Finish first pass of the usage guides for liblfds and test.

Todo;

1. ringbuffer docs (and possible API changes – not sure I like how it is now)
2. finish first pass of the building and porting guides for liblfds and test
3. implement the ‘small fixes’ list
4. update all the build config and have test pass on all platforms
5. second pass of all docs

and back round we go

So, I made libpal and moved the liblfds and test porting layers into it. The liblfds porting layer consists of header files only, so libpal does not need to be compiled for liblfds – but it does need to *be there* and that was *odd*.

It meant liblfds now dependend upon some header files elsewhere, so you had to make sure to copy not just the library other but this dependency.

This is not going to be expected.

I think in fact the problem was that I added libcommmon and libbenchmark. I like having them, they make it easier to code, but then I need to document well there’s test, and then there’s libcommon, and you need to build this and this for that and so on…

So what I’m going to do is make liblfds, test and benchmark independent (well, test and benchmark both require liblfds of course :-)

They will have their own fully independent porting layers, and I’ve got rid of libcommon/libbenchmark. It will mean duplicating some abstraction and utility code between test and benchmark, but it’s worth it for the simplification of documentation.

thoughts

I’ve been looking at the porting documentation.

Right now it’s wrong, because it talks about porting *test*, but you don’t port test – test has no porting stuff in it. You need to port libcommon; it’s that which has the abstraction layer for test.

It’s called libcommon because although it’s not in this release, there’s also a benchmark programme, and there’s also a libbenchmark, which has another, fatter abstraction layer needed by the benchmark programme.

So it’s a complex mess.

What I’m thinking now is to remove the abstraction layer from liblfds, get rid of libcommon and libbenchmark and have libporting (porting abstraction layer). This will contain all of the abstraction layers – you implement as much as you need to get what you want working, i.e. there’s a minimal layer for liblfds, a bit more on top of that for test, then quite a bit more for benchmark.

libcommon and libbenchmark also contain quite a bit of utility code used by test and benchmark, but I can move that code into the programmes themselves – just to get it out of the way from the user.

Thing is, I really do NOT want to go off into major surgury land at this point. I want to get a release out.

Update

So far today;

1. hash docs first pass
2. first pass for both usage guides
3. introduction second pass

Next;

1. btree docs
2. building guides

Update

So, today;

1. list bifuricated (library and test)
2. first pass list docs written (for both lists)
3. first pass liblfds porting guide written
4. added liblfds to the API docs

Outstanding;

1. binary tree docs
2. hash docs
3. liblfds docs
4. ringbuffer docs
5. porting guide for test
6. complete the building guides
7. complete the user guides
8. bring all the build configurations up to date and run test on all platforms