Maybe the end of the line for GCC

So, I’ve been running my GCC build script on ARM64, just for 4.9.2. That script works on Debian on x64. It doesn’t work on ARM64. There’s a wierd error, fairly far into the build process, that it can’t find pthreads.h (which is present, and in the usual place).

Googling just leads to a bunch of other people, over the course of many years, saying they’ve found the same problem and are just about at their wits end (because there’s no apparent cause, and so no apparent fix).

I’ve spent six weeks getting that build script to the point it’s at on x64, and now I run it on ARM64, it doesn’t work.

My line of thought now runs like this : it is not possible to build GCC.

Because GCC cannot be built, it is only possible to test code with the GCC version on the build platforms you have. You can support no other versions, because you have to access to them, so you can’t compile with them.

That’s completely intolerable. No serious software package can be presented to users in this way – where the supported compiler versions are not under the control of the package developers.

I had a look at the build instructions for clang, and they look normal and sane. Thank God for choice!

Update

So, I started writing a HTTP server. Single thread, async I/O for network and disk. Simple, right? wrong. Linux async I/O support is a fabulous mess. I *think* I can just about get away with it, with two threads, one for epoll and one for io_getevents (kernel AIO). I *think* the kernel version of AIO will work on regular files (but I’m not certain, actually).

The HTTP 1.0 spec is easy to implement – but there’s no pipelining, and pipelining matters. I looked at the 2.0 spec, and it looks complicated. One of the beautiful things about 1.0 was that it was *simple*. I think 1.1 allows clients to send say chunked encoding, so already implementation is a headache. So we’ll see where I go with this.

So, given that I’ve got to read the HTTP specs, what I’m actually working on now is compiling all the GCC versions on the various build platforms. It’s a lot of work – GCCs seem to come in tranches, so there’s going to be a number of separate sets of build problems to deal with.

Right now I’m compiling 4.9.2 on ARM64. Will make sure -march=native works. Then it’ll be time to build all the versions and finding out about the build problems.

And then doing it all again on ARM32 and MIPS32 (but it should be pretty easy by then, hopefully they’ll run through without any extra work).

build system heaven

Absolutely bloody beautiful.

I have now two scripts, both run on Linux but one targets Linux and the other Windows. I’ve spent the day getting the Windows build working, and now it does; the VM starts up, the build iterates through all variations (IA64, x64, debug, release, coverage, debug, release, etc, etc, etc, etc), and runs the test on release builds. No benchmarks, because it’s a VM.

*Beautiful*.

I produced a 7.0.1 and 7.1.2 with the liblfds build updated to match the 7.2.0 build – necessary for libbenchmark to build, since it has an embedded copy of 7.0.x and 7.1.x.

The Linux version runs over all the Linux hosts – x64, MIPS32, ARM32 and ARM64, and does the same. Gnuplots made by the benchmarks are brought back to the host.

The only problem now is that the GCC on ARM64 doesn’t understand -march=native, so the make fails.

This actually touches upon me needing to compile every released version of GCC On each platform, and having the build system use them all.

That’s the next work.

After that, I need to get Bugzilla working again, which means doing something about the web-server, since nginx doesn’t support CGI.

Unexpected behaviour from Gnuwin32 gnumake and tar on Windows

Gnuwin32 gnumake does not on Windows support single backlashes as path seperators when being called with -C from inside a makefile.

So – if you have a makefile, and from inside that makefile you call another makefile, and you are in that call using -C (change directory), that inner make call will fail because it does not understand the path to -C. I think it’s probably treating the single backslashes as escapes. Single forward slashes work, as do escaped backslashes.

However, if you run that inner make directly from the command line, with single backslashes, it works.

Gnuwin32 tar seems to be randomly dropping file suffixes on Windows. The tar is made on Linux, sftp’d to Windows, then if I list it, about one in ten files is missing it’s suffix, which for liblfds means usually .c or .h, which means of course that the makefile falls over because it can’t find the files.

Going to try using zip instead. Compression sucks, but hopefully it’ll work.

Build system update

More good stuff.

Have implemented the python functions (using vboxmanage) to start and stop VMs (where starting means getting the guest IP and then waiting until you can see the SSH server is responding).

At this point I’ve discovered a design flaw. I had thought I could now use SSH, SFTP and make on all platforms – which is to say, including Windows (which is what I’m using the VMs for).

This is not in fact the case. Windows needs a batch file run before the C build environment is ready. What’s worse is that it takes a bunch of args which vary depending on the target processor and operating systems – and you’ll note in that I didn’t think about cross-platform builds. The build system supports processor types, but it doesn’t (currently) support multiple *operating system type targets*.

Windows being a complete PITA again.

So too sleepy now to do more tonight. Did what I wanted to do, which is get VM startup/shutdown going.

Build system update

It’s going very well.

I had to do a fair bit of work on the library itself to get it back into fully compiling shape, for the tests and benchmark, and the two SMR methods.

So I now have a build system which will build on every Linux platform, over every build variant, run the tests and the benchmarks, and pull the gnuplots back to the originating system and convert them to PNGs.

What I need to do now is;

1. get all the version of GCC compiled on all platforms and add support for changing GCC version (so then all the Linux verions will build over every released version of GCC)

2. set up the Windows VMs with an SSH server, then figure out how to use vagrant to start/stop the VMs on demand, and then figure out how to get builds going on the Windows platforms

Build system progress

Looking good.

Have the three dev boards hooked up and ready to roll (arm32, arm64, mips32).

The build system knows the platforms it has, and how to get files to them and build on them; and it knows the targets for liblfds720. I’ve been building successfully on arm32 but I ran into a problem on the arm64 platform, the GCC (4.9.2) there doesn’t understand “-march=native”. This was a surprise.

In fact, “native” hasn’t always been present historically, even though I expect it with 4.9.2, so it’s a problem I need to address anyway.

I think what I need to do now is build every GCC version on each platform. I haven’t yet mastered building glibc, but even if it’s every GCC with current glibc (hmm, assuming that *can* work) then at least I should get past this wierd (?) GCC compiler on arm64.