7.1.0 is out

Finally, I can get some sleep 🙂

And a haircut.

Some exercise!

A social life.

And I can start applying for jobs.

benchmarking earlier liblfds releases

So, problem.

Consider – as time goes by, liblfds supports more platforms. 7.1.0 supports MIPS. 6.1.1 did not. If benchmark benchmarks all version of liblfds, well, then it will try to include the earlier version’s header file and it will try to link to its library.

The header file is fine – it will work, and it can be modified (I can make bugfix releases for earlier versions) to make public which data structures are available. The library file *isn’t* fine, because right now it has to be hard coded in the makefile…!

It’s the libnuma problem all over again. With libnuma, I have a second build type. Adopting that solution for liblfds versions would mean one of two things, depending on whether the assumption is made platform support only increases over time. If as releases go by, platform support extends, then we’d end up with one extra build variant per release, i.e.;

gcc_gnumake_liblfds611_liblfds700_liblfds711
gcc_gnumake_liblfds700_liblfds711
gcc_gnumake_liblfds711

This would be doubled, because of libnuma.

However, if platform support can simply *vary*, then this approach becomes awkward.

I am however strongly inclined to the view it will only increase over time.

However however, this then leaves the user with what on the face of it is an awkward problem – when the user comes to build, he sees these build variant directories, and to select the correct variant, he has to know what releases of liblfds he can build.

OTOH that’s not so hard – he can just try to build them, and see if they build okay.

Update / benchmarking 7.0.0

Documentation first-pass finished.

No missing articles.

I need to fix up the “enum flag” type use in the codebase – there’s a type in liblfds itself, and another in the test_and_benchmark code, and they’re intermixed in that code. It needs to be one or the other.

Changing subject, the code for each release of liblfds is wholly indpendent and occupies different namespaces (by means of prefixes to the names of all public entities). As such, every version of the library can be compiled against and linked to, concurrently.

As such, the benchmark app can benchmark *the different versions of the same data structure*.

I’ve just spent an hour adding in a benchmark for the 7.0.0 btree!

The performance is really bad, though – and I never remembered it being like this; it was also pretty good. So I’m wondering if I’ve flubbed up the implementation somewhere along the line.

Update

Public APIs which are too complex or too big to be documented are totally worthless.

When you come to write your docs, you have to rewrite all that code.

I’ve spent the last half-day rewriting the libbenchmark abstraction API for enumerating processer/memory topology.

The problem has been that the function was directly using a bunch of topology_node API stuff, which then would all needed to be documented.

I’ve written a small set of helper functions, which mask all of the topology_node stuff.

Getting there!

One more set of API function docs to write – about a dozen or so pages.

Then a bunch of pre-release prep, then release!

Update

So, read the spec, return with an expression from void is a warning/error.

I fixed it by doing something I’ve thought for quite a while I should do – not use brackets with return. It’s not a function, don’t use brackets. Turns out now to matter, since it allows me to mask windows kernel threads in a readable way.

So, the code now compiles on WDK, all platforms.

This means I now have successful compilation on all platforms with all build files sets – all twenty-one of them…!

I”m not going to go round for the second pass and make sure everything still compiles on everything else until I’ve done the docs, since they will prolly lead to changes.

I’m now writing the docs.

Learned something new

So, Windows kernel has no return type for its thread functions.

To abstract this, I have a macro, which is for Windows kernel defined like this;

#define LIBSHARED_THREAD_RETURN_TYPE( return_value )

So at the end of the thread function when I come to return, I have this;

return( LIBSHARED_THREAD_RETURN_TYPE(return_value) );

(The type of return_value is “int” for Windows kernel, just so the assignments in the code compile).

So the code ends up post-pre-processor like this;

return();

Which I thought would be fine. Return is not a function call, it’s a keyword, so the brackets just disappear.

Guess what?

Of *COURSE* it doesn’t work.

The compiler doesn’t like it. “error C2059: syntax error : ‘)'”

Linux kernel source code

Just found out Linux kernel source code base has lower case #defines.

I found out ’cause they’re colliding with my VARIABLE NAMES…

Linux may not be the super-awesome-power code base we’re thinking it is =-)

Going nuts

Been working non-stop, mininal sleep, to get the release out.

There’s SO MUCH WORK to get builds working across what in fact in the end is 17 different sets of build files, which run on six different plaforms, especially when you’ve largely rewritten most of them.

I in fact hadn’t consciously realised it, but I had not implemented some of the abstraction layers – I’ve just implemented (really a re-implementation, mainly, I’d done a lot of in a long time ago, but removed the code since then) the Windows kernel abstraction layer and I’ve just right now been implementing the Linux kernel abstraction layer for libshared…

…and, now, finally, after all this time, I have discovered the Linux kernel does not offer a “wait for thread to terminate” function, which totally and utterly fucks the entire abstraction mechanism for threads.

*truly -epic- facepalm*

Linux does has a function to create threads in the kernel and to put them on a given node and CPU, but you have to manually implement signalling between the thread(s) and the issuing thread, to sync between them.

This means I now need to implement this style of mechanism across all platforms, which means rewriting a metric fucking ton of code. It also means making an abstraction layer for an inter-thread communication mechanism. Jesus, headache, headache, headache. WHY NOT JUST OFFER WAIT-ON-THREAD-HANDLE FFS EVERYONE ELSE DOES.

It also means 7.1.0 is going to be released without Linux kernel support for test and benchmark. You make your choices, you get what you get. Linux made its choice, and so it doesn’t get test and benchmark.

makefile whack-a-mole

I finally found the gnumake recommended practises documentation and I’ve been making the makefiles comformant.

When you have fourteen makefiles, over four platforms, it takes a while to get all of them happy everywhere…