So, building GCC and glibc.
In short, I have given up trying to build GCC and glibc.
I spent ten weeks trying. It can’t be done out-of-the-box, because the build system is much more complex than its documentation, and it may also be the build system is actually does not work out-of-the-box, and requires significant source code / build system modification to work, where those modifications most likely vary by GCC and glibc version, and are themselves undocumented.
The benefit of building glibc is that it allows the building of the various versions of pthreads, where the benchmark app benchmarks the locking mechanisms in pthreads.
I can however build GCC now, on its own, so I’ve been getting on with this. This allows the makefile to be checked (I’ve already seen I’m now using GCC options which early GCCs don’t know about) and will allow me to run test and benchmark.
Building GCC on the dev boards is quite time consuming. They only have 1 GB of RAM, so I an only run make -j 1, because the boards although they have more cores run out of memory with more parallel builds (well, at least with later GCCs, which have more complex builds; earlier GCCs get away with more).
It took twelve hours to build GCC 6.2.0 on my PINE64 :-)
I had quite a few problems with the CI20 dev board - but in part I think this was due to running out of store (I was running with -j 4, then -j 2, and now -j 1, same as the other boards) and in part I suspect because if the CPUs are running flat out, the board can freeze up - adding “nice” to the make for configure seemed to help…)
So this leaves me now building GCCs.
The baseline is a pre 4.7.3 and a post 4.7.3 on each platform. The native compiler is 4.9.2 (they’re all running debian 8, oddly enough) but on the PINE64 this compiler doens’t understand -march=native - but I want the same GCC, from the same build process, on each machine anyway.
So far, I have achieved this on x86_64 (my laptop). 4.2.0 is built, as is 4.9.2 and 5.2.0.
On the PINE64, aarch64, support for the platform arrived with GCC 4.8.0. I’ve built 4.9.2 and 6.2.0 successfully.
On the Pi 2, running arm, complete failure so far. I’ve tried building the latest point release for every GCC prior to 4.7.3 and they ALL fail, for different reasons.
4.9.2 also fails, as does 5.2.0, because of a bug in libgmp.
It looks like I’m going to have to do some real hacking on the arm platform to make a GCC, even on its own.
On the CI20, MIPS32, I’ve tried 4.2.0 and 4.2.4 and they both failed, I think due to changes in signal handling, such that later GCCs can no longer compiler these earlier versions (header file incompatibilities).
Another three builds are currently running - let’s see how it goes.
Oh - I emailed (together) the GCC and glibc help mailing lists, explaining what I wanted to do, and asked for help.
In glibc 2.25 there seems now to be a script for building glibc, which may help - but I’m too emotionally antagonistic toward this task now to continue down that route.
One other guy pointed out about NetBSD “pkgsrc”, a way to get hold of working portable source.
But that was it. No other help. People either didn’t know, or were wisely steering clear of someone else’s tar-pit which would be problematic to help with from the outside.
Building the CI20 is problematic.
The device/OS appears unstable.
It will freeze up. It seems to do okay with earlier GCC versions, but now I’m up to 4.7.4 it’s dying during the build. I’m running “nice make -j 1” and I’ve added 1 GB of swap (1 GB of RAM), and the Pi 2 and PINE64 both make 4.7.4 fine with 1 GB of RAM and no swap.
I can reboot the device and restart the make - but because it freezes up, the file which was being written at the time of the freeze comes to exist but is truncated - which breaks the build when (typically object) file is used.
So I’m trying this now, but taking care to try to find out which file was broken by the freeze.
So, I spent an age figuring out which versions of GCC had which version of its depedency libraries.
Then turns out GCC is really quite conversative, and uses quite old versions of those libraries, only bumping up a version when it needs to.
So I’ve been figuring out the correct versions all over again, and although it’s clear back to 4.5.0, it’s not so clear after that - but I’m about to go out for a couple of hours, so I thought I’d kick off a 4.5.0 build on all platforms.
So I fix up the build script, fix up the source dirs (soft links in each GCC dir to its dependencies) and kick off the build.
It fails (it’s just failed on aarch64 too).
Configure bug in the libgmp GCC has used since 4.5.0 onwards, so that the check for flex fails in the presence of non-standard signal handling - such as configure being run from a Python script.
facepalm
Srsly. GCC build is a minefield. Take a step, lose a leg. Accurate description, no lie.
Then also a day or two ago I finally bought an enclosure for my external 2TB 2.5 inch drive, which is 15mm, so it was hard to find an enclosure.
Plugged it in, works, that’s great, but I need to copy all my blu-rays to an old 3.5” drive I’ve kept for this, so I can reform the 2.5 drive to veracrypt (it’s truecrypt right now, and NTFS on that, too!)
So I kick off that copy too.
Which fails. It copies about 500mb of Blade Runner and then stops copying.
Makes it look like one or other of the disks has a problem.
So now my happy hopes of having all these things buzzing along while I head off to the gym are replaced by clanging leaden bells of failure and bloody awkward problems.
(It looks like the source drive has the problem, which is a bugger - that’s the drive I intended to use and keep using. A reformat might help, maybe, but my Blade Runner blu-ray is in storage, so I don’t have it with me to re-rip).
So…
Looks like the 3.5” drive I have is broken - I held on to it to be able to reformat the 2 TB 2.5” drive I have which has all my blu-rays on. I could prolly do a bad block scan… …only it would literally take days, and I don’t want that much noise 24/7 in my room next to me while I work! Still, it’s far better THIS than the other drive, ’cause then I would have lost data. Most of the blu-rays are in storage and so not accessable.
libmpfr 2.4.2, the recommended version for GCC, is broken on MIPS. Go figure. There’s a patch, I’ve applied it.
I discovered after playing around with a USB voltage/amps meter that the four-port synwire USB charger I use to power my dev boards and wifi router instead of having four 2.4 amp ports, as I had thought, in fact has two 2.4 amp ports and two 1 amp ports. This is why the ci20 kept bloody crashing. I’ve ordered a decent Anker unit, with six ports. Anker are right now AFAIK the best USB equipment manufacturer.
The PINE64 turns out to have major ethernet issues. I downloaded a benchmark app. It normally runs at about 90mbit (the hub is 100mbit) but every few seconds drops down to zero for three or four seconds. Googling finds a lot of people complaining about this. However, I also now find it’s just playing up completely, and right now, I can’t connect to it at all. Wierd.
GMP 4.3.2 (the version recommended by GCC) has a subtle bug such that one of the GCC tests will spuriously fail. Allegedly this is for 4.8.0 and later, but I seem to have it with 4.5.4. What this means in fact is that make check on GCC ALWAYS FAILS.
I’ve been reflecting a bit about the GCC build system.
It seems to me there is a clear and fundamental difference between black box and white box systems. A black box system is operable only by its controls; you cannot look inside. A white box system can be operated by its controls, but you can also modify whats inside.
A build system should be a black box. It should be a working black box. The user operates it by the controls on the outside. The user has no need to modify - and so to have to learn HOW to modify - what’s inside the box.
If the user in fact must modify the insides of the box to make it work, then it is not a black box - in fact, the user is performing developmemt work. The system has been published in a state such that the controls on the outside are inadequate, for whatever reason, to make the system work. The user to use the system MUST learn how it works, so he can perform the necessary development work to make it work.
In the case of GCC, where the white box is extraordinarily complicated, depends on external libraries with their own issues, is barely documented and is buggy and the problems vary by platform, having to learn how to modify the system is a very high burden, in fact, almost insurmountable, and I would think often not sensible - the cost is much more than the benefit.