Benchmark update

Good news!

On what is I think now the fifth attempt (over the course of about two years) I finally have the design of the benchmark programme correct.

It’s a complex beasty – you have the data structures, the benchmarks, the many different types of locks (and the lock-free as well of course), where the locks vary by platform, CAS and DWCAS backoffs need to vary, where each benchmark runs over a range of logical processor sets (e.g. say on every logical core, then on one logical core per NUMA node, etc, etc – the full meainngful set of sets), and where you want to be able to produce arbitrary gnuplots afterwards.

It used to be more complex – there were NUMA and non-NUMA variants to consider, and also before the SMR variants of data structures became a separate data structure, you also had to consider the SMR and non-SMR variants.

I remember spending two weeks in particular Tunis working on this – it’s a deadly trap. The problem is you can just about fully optimize the code layout; abstract the lock types, loop over the benchmarks, loop over the data structures, etc, etc, etc – you end up with a perfect factorization. The problem is, it’s at the limit of what you can manage… so when you come back to it, you can’t remember how it works, and, more painfully, if *ANY* new requirement comes along, it can’t fit – but you’re already at your limit *even when you were fresh to the code and had it all in your mind*.

A simpler approach was needed.

What I’ve done is basically two things – first, I’ve removed the per-lock abstraction. Each data structure now has an implementation for each lock. This was actually needed because having one template data structure which used function pointers for locks added the overhead to the locking benchmark runs of following that function pointer. Probably the cost is very small – even negligible – but it’s unwise to assume, so it would have to be investigated to be sure, and of course adding overhead to the locking benchmarks favoured liblfds. Second, I’ve used an abstraction layer, as with liblfds; the abstraction layer implements the locks – all of themm, for all platforms, dummying them up if they’re not actually available – and all the code above that is always fully compiled on all platforms.

I’m still thinking about what to do with the output though, since there can be many thousands of output records, and I want to access them arbitrarily to make gnuplots – I could run into a performance problem!

Anyways, I’ve just made the full basic structure of the benchmark programme compile. There’s a lot of code thrown in there from the previous attempts – it all needs a lot of work on quality – but it compiles and the fundamental design is correct, so we’re on the right track.

Hopefully will tomorrow have some gnuplots for freelist.

My recent encounters with airport security

I want to write about my recent travel experiences with airport security.

I flew from Newark, New Jersey (EWR) to Schipol, Amsterdam (AMS) and then a second flight, a few days later, from AMS onwards.

In EWR, I as I always do declined the body scanner and asked for a pat-down.

One of the operators tried to persuade me the scanner was safe by saying it was “like an ultrasound”, which is a shocking claim – utterly untrue. An ultrasound is like sonar. It uses sound waves which pass physically into the body and bounce around. A body scanner is like an x-ray machine. It uses ionizing radiation, which rams into the body and damages cells and contributes (in what is hopefully a tiny fashion, but the research is not clear) to the risk of cancer.

No one working with these machines should EVER say that. It was disgraceful.

So then I was “patted-down”.

Except it wasn’t a pat-down. That phrase is for when you stand up and someone pats you. I was “patted down” with such physical force I was actually staggering backwards and forwards – I was unable to maintain my balance standing. This went on for about five minutes.

So that was EWR.

Then in AMS, when I departed, again, I declined the scanner and was patted-down – actually patted-down; but this time, I was also taken into a privacy booth, and asked to unbutton and lower my jeans, which I did, he visually checked the buldge in my underwear and that was it.

That hasn’t happened before. I’m wondering if there is a connection between these two (EWR and AMS) events. Has there been globally a directive to make non-scanner checks more… more, what? more effective? presumably effective? while at the same time of course making them more intrusive, more intimate, more and more unpleasent – and remember here, there’s no evidence I am hostile; these are just routine, generalized checks.

The TSA recently stated passsengers no longer have the right to decline the body scanner – they can, at the TSA’s discretion, be required to go through, or, of course, be refused their flight (and what else? then coming to the attention of the FBI? who have the power to make a warrentless entry in all kinds of situations – including, shockingly, living within one hundred miles of the US border, which includes of course all of say NYC, which is where I used to live. Congress and the Senate are useless – these kinds of powers should not exist, but they do).

So, what do I come away from this with?

If EWR pat-downs are like that normally, I am not longer going to fly through EWR. I’d have to go through EWR again to find out if it is so, and the experience was unpleasent enough I have no wish to experiment to find out – so that kind looks like EWR is now on *my* no-fly list.

I already no longer fly through Gatwick, not for that reason, but because of the awful, awful experience of the cameras as you queue for immigration.

If US airpot pat-downs are like this in general, then – believe it or not – I’m no longer flying to the US. I’ll see if I can get in by ship in a more peaceful manner.

What of Schipol? I don’t know what to make of that. The TSA (the American airport security organisation) being abusive I know about. They are culturally like that and it’s not being fixed – indeed, I’d say it’s getting worse. (They’re also expensive – eight billion a year – and no good; the detection rate for explosives and guns is about 8%) – but Schipol? I thought the Dutch would be better. I will give them a second chance, and see what happens.

Benchmark progress

I decided on the source code layout, which is a huge issue in this programme – what’s being done (benchmarking many variations) lends itself to a highly factorized layout, but it ends up being unmaintainable, and also suffers of course inherently from inflexibility; when you come up with another test type which doesn’t fit in the factorization you have, your code is problematic.

I spent yesterday and now today so far getting the topology code back on its feet, which seems to have worked, first pass (by first pass I mean hours of compiling and fixing up the code for 7.0.0 – but first pass as it, it wasn’t broken once this was done!)

Sample output from my Core i5 on Linux;

M  = Memory
N  = NUMA node
S  = Socket (physical package)
Ln = Level n cache
D  = Data cache
I  = Instruction cache
U  = Unified cache
P  = Physical core
L  = Logical core

 P   P 

I’m now going to see if I can get release 6 (six, not 6.0.0, etc) compiling reilability and consistently on Raspberry Pi – it seems to compile with some versions of GCC but not others, and the problem is the in-line assembly for DWCAS. There’s a better way to do this, which is to cast to an int long long unsigned and use the GCC instrinsic, which I hope will side-step the problem.

Edward Hugh died

I’ve just found out – I was Googling to see where he’d gone; he’d not posted anything for about six months.

He died on the 29th December, 2015, his birthday.

Every hand-over-eyes-I-see-no-problem politican in Europe will breathe an involuntary sigh of relief.


Cor, finally.

Finished settling in, more or less, and now about to actually do some real actual genuine honest-to-God *coding* again at last.

Next release will be 7.1.0, will be identical to 7.0.0, except it will contain the benchmark programme.

Thales, no, make that Fails =-)

I was looking over job postings for remote (home based) work and came by something from Thales, in the UK, so, this morning, I went over to their site to see what they have.

What they have is the electronic equivelent of a pair of rubber gloves, which you must wear while performing brain surgury.

First problem is real simple : I’m staring at their careers page, and I can see tons of stuff about how amazing Thales is, and I can’t see a single thing which shows the list of, you know, JOBS.

After about five minutes, I find it. Scroll down to the bottom of the page, then head over to the right rail, then there’s a small heading “apply for a job” and a drop-down list of countries. It takes a while to grasp that *this* is *it*. It’s like discovering the power button on your laptop is on the underside and looks like a screw head.

Next problem of course is that I’m interested in a number of different countries, and now it gets wierd.

You see, I first picked “UK” and discovered an unusable job listing/search interface. There are lots of jobs, you must be able to search, but the search functionality provided just isn’t viable. You end up having to perform hundreds of search permutations, because it’s not possible to select multiple entries in any given catagory or field. So for example, there are four catagories all of which involve writing software. I’m interested in all of them – but I can only select one of them. Any search must be repeated four times. There are keywords I want to search for – but there’s no logic functionality; the site searches for ALL the keywords and they must ALL match. Etc, etc. There’s also no way to look for home based work.

However, I’ve just gone to the other countries sites, and they all seem to be using Taleo (which I’ve come across before and roundly detest).

All the sites list all the jobs in all countries.

So it looks like… the country list in “apply for job” is actually a way to select which job interface you want. For every country except the UK, you’ll get Taleo – in your own language – but if you choose the UK, you get a customm interface which is not useable.

So I wanted to send Thales some feedback. This is just about impossible. This is usual for large companies; it’s bloody obvious that all sites need to be able to get feedback from their users, and equally common for it to be impossible. Of course a busy site like Thales will receive so many emails only a fraction can be read – but a sampling is all you need to get feedback on what you’ve messed up with your site.

Thales also on their feedback page are using and only using old, insecure cipher suites. I had to switch to an insecure browser (RC4, etc) to be able to send them anything. Not a good thing for a company selling defence and security.

The usual

So, where to begin.

I’m in a new country.

The usual problems.

1. Bank has blocked my card, on the second use. I did tell them, of course. This is usual – not right, but usual. Use your card in another country, and it’s blocked. Simple as. Banks are like cats. They can’t resist pouncing, no matter what.

2. My previous apartment, in a condo, can only send me my deposit in the form of a cheque. I am now outside of the USA and indeed in a country which has no branches from my EU bank. The condo tells me they must do so to adhere to banking regulations in their state. If this is true, then the state is preventing bank to bank transfers… in 2016. I look forward to their adoption of the quill and ink dipping.

3. I’m looking to open a local bank account anyway, to get hold of a euro denominated VISA card, because I need this to order shopping on-line here. To do so, I need to be able to make landline calls, because I need to get a reference from my EU bank, as they have no on-line chat support. To get landline calls, well, I’ve finally figured out what’s going on with that, and I’m now buying a SIP account to use with LinPhone. Thank God I finally get away from Google Voice – I wish I’d been able to figure this out much, much earlier. I hate having my calls recorded by Google.

4. I’ve just tried creating a SIP account with a particular provider. First problem – no cipher overlap. They’re only using older, insecure cipher suites on their site, RC4 and the like. Doesn’t fill me with confidence. I switched to a standard browser and tried again. Turns out my passphrase is… too long. More than 15 chars. My email address is two characters longer than my passphrase. You see the problem? some idiot when designing their SQL schema allocated (say) 128 characters for email addresses… but only 15 for passwords. Did I mention the lack of filling with confidence?

Chase Bank

Today is the 4th January. I leave the USA for the EU on the 7th. I need to transfer funds to my EU bank account, so I have money when I arrive. Today is the last day upon which I can make this transfer and be sure the money will be there when I arrive – I am so late in this because of AirBnB, the next story.

I logged into Chase. I added a new international wire recipient.

This was mildly agonizing – kind of a prelude, a warm up, a small filling without anaesthetic, just to get you in the mood for the following agony.

It’s a series of fiddly, complex, error prone, poor documented forms and after the fifteen minutes or so it took me to fill them in, I realised I had where I should have entered my address, entered the destination bank’s address. There was however a link by that address, “change”, so I clicked, to edit that address.

It does not in fact edit that address. It takes you to the beginning of the adding process, and you must fill out the entire form from scratch a second time.

You can imagine how I felt about that little booby-trap. Clearly Chase are using their web-site to prepare future soldiers for IEDs. Never trust that link, soldier!

Chase then requires me to enter a verification code, which can only be sent by SMS. I do not keep a phone, for privacy reasons, so I cannot receive the code. The site says “click cancel if you can’t get a code” and the recipient will be added in a pending state (great UI design, eh? cancel means next).

I do this, and then phone Chase to find out what to do.

Here comes the tooth extraction, sans anaesthetic.

Because I have already created the new recipient, they cannot give me an authorization code. In fact, one is not needed as such – because now, at this point, the only way for that recipient to become valid is to wait for a day or two.

(Remember – this is the final day for me to make this transfer, so the money is in the EU when I arrive).

The only thing which could be done would be for the new recipient TO BE DELETED, so she could give am auth code, which I must enter BEFORE I make a new recipient, and then I could enter the entire, fiddly, complex, poorly documented form a third time – assuming of course I make no mistakes.

However, since it was now past the end of day for international transfers (and I do not comprehend why, in 2016, banks only issue these transfers between the hours of about 9am and 4pm), she could not delete the account – and would not be able to delete it at all – and so the only way was to wait for it to activate in time.

I was however given an auth code so I could try entering the account a second time, and see if this worked. I duly tried, and it did not.

I could not imagine, in a fantasy, a more twisted, bizzare, broken, unexpected, screwed-up, insane, stupid, dumb, idiotic arrangement.


Do you even know it exists?

Total time entering accounts : about 40 minutes. Total time on the phone : about 25 minutes. Total time : 65 minutes.

I will tomorrow have to go down to a local branch, and pay an extra five dollars to do so, and make a transfer there. This I know from experience will take an hour.

Total time to transfer some money from A to B? 2.5 hours.

Here’s a question. All banks are appalling. Why is this?

I think it is not natural that all banks should be appalling. There should be a range, from good to bad – but there is not. They are *all* bad. I think there are mechanisms at play which are generating this outcome. What are they?

In my life, there are four sources of stress;

1. banks
2. telcos
3. gyms
4. any major on-line service which dominates its market

I have more or less eliminated by #2 by no longer keeping a phone number.

I’m kinda thinking now of only living in warmer countries and taking up running, which would deal with #3.

I’ve yet to find a way around #1.

Usually with #4 there’s actually enough choice you can just about use other services. AirBnB is the most recent example of a catagoricially stupid near-monopoly which I now on an emotional level reject due to their truly staggering incompetence – no, more than incompetence, imbecility, because what’s being can only happen when there is no actual conscious, rational thought occurring at all.

Slowly leaving the web

I find I’m turning away from the web more and more these days, because most sites are so badly written and run that trying to use on-line services is usually profoundly frustrating and often impossible.

There are trivial examples – WordPress, for example, if you flub a login, wipes the username field, so you have to type it in again.

ARM, the processor company, I came yesterday to register on their developer forum. The username field was hobbled by javascript to be alphanumeric only, so the usernames could not have special characters. This code also prevented delete from working (so it was impossible to delete a typo – the page had to be reloaded), and tab too (you have to click in the next field with the mouse). Then, once I’d figured it out (reloading the form more than once, I must say) and filled the form in, registration didn’t work. I’ve emailed ARM, to try to get that working.

Craigslist, I’ve just come to post some adverts. The all-too-usual bait’n’switch. Fill in the entire advert, THEN they put you through the account registration work, and then I find out a phone number is mandatory, for an authentication phone call or SMS. I also found out you can’t post to the help forum without a Craigslist account, which I can’t get because I lack a phone number. There is a direct contact form, I’ve used that.

AirBnB – well, they’ve always been bad, but there’s new bad. Now identity verification is mandatory. To do this, you have two options – the first is to upload a copy of your passport. This is so far beyond the unutterable pale of totally stupendous ignorant blind idiotic stupidity I don’t know what to say. Have they not noticed the daily news of major companies being hacked? the second option is to fill out a form with some personal info – address, social security number, etc. This also is wrong, because SSNs should not be used for ID – the problem is that they are static (you cannot easily change them) and to a first approximation, all such static IDs are now public knowledge. It’s like giving someone a personal magic phrase, like “purple pumpkin”, which cannot be changed, and then accepting anyone who knows the magic phrase *is* that person, where that phrase is more or less common knowledge. Getting back to the form, it doesn’t work. Hit submit, and it just takes you back to the start page for ID verification. I emailed AirBnB about this problem and received a pro forma reply. I presume someone glanced at my email, saw it was about ID, and hit a button to send the pro forma. I replied, no reply yet. I doubt I’ll get anywhere with them on this, AirBnB are not functional with regard to customer interaction or problems.

Amazon. If you buy through Tor, some time *after* you’ve paid, they wipe all your open orders and blank you password so you have to reset it. If you do this twice, they lock your account, and it takes a few days to get it open again. During this time, you cannot log into your AWS account. Any VMs you have running are costing you money, because you cannot turn them off. When your account is unlocked, your address book has been wiped.

Kayak. Pretty good site, actually, but has a very annoying habit, as is now extraordinarily widespread, of constantly popping up dialogs – many of which are modal-like, greying out the whole screen and being placed front and center – asking you to set up email reminders and the like. AirBnB does this a lot too and in fact a *lot* of commerical web-sites when you go to them will all start with a modal “mailing list / special offer / register now” dialog – forcing an advert on a user who is coming to to their site to purchase from them. It’s insane. Do you really want to harrass, discourage and annoy customers? If cookies are off, this occurs on every page.

Hugo Boss. An example of geoIP lookup gone bad. You actually cannot access a specific country site, as you will *always* be redirected to the site for the country your IP address is listed as being in. GeoIP lookup accuracy is about, what, 80%? but I don’t know how accurate lookup is at country level resolution. Maybe it’s higher. However, customers who actually want to buy in a given country, so they can post something to a friend, cannot. Also, as is often the way, Hugo Boss will only post to the country of purchase, and only accept cards from that country. Not useful, presumably to stop fraud, but I don’t know that for sure – it may just be bad design; I see lots of other sites (Amazon, etc) who do not have this restriction.