crypt() function potential insecurities with invalid salts

So, yesterday I discovered quite a serious vulnerability in an application using the crypt() function (it was in perl, but perl just calls through to the system library so this application design flaw may be found in anything using the crypt() function for authentication).

Firstly why use crypt() at all? It’s well known that the DES-style crypt is very weak, for example the salt is only 2 characters and it only takes the first 8 characters of the password and ignores everything after that. However the modern glibc implementation of crypt() include a number of very secure hashing functions prefixed with $, particularly $6$ which is hashed SHA-512 and I’d advise everyone to use.

Anyway, back to the issue at hand. Your standard crypt() password check would look like this (taken from perl’s Catalyst::Authentication::Credential::Password module):

ie you use as the salt the encrypted version of the password to encrypt the user specified password, and then check that against the crypted password itself. If they match it means the password was the same.

However in this application for certain pre-created accounts or accounts that had only been logged in to using an oauth mechanism (eg facebook login) the user’s password field was an empty string. This seems reasonable enough – crypt() of a blank password should always return something non-blank (unless you’re using an older version of mysql that has a crypt() inconsistency that I reported 2 years ago). eg

Unless that is, that you specify an invalid (or blank) salt:

D’oh. This basically means that if you are using a blank password column to specify no password login allowed in reality someone can log in with ANY password! So in the case of the app in question if you knew the registered email address of someone who had a precreated (but locked out) account, or the address that someone who logged in using oauth you could do a login with any password. I’m guessing that because this is not particularly widely known amongst developers there are probably a number of apps where this is possible today but no-one has tested it.

Some workarounds/mitigations:

  • Set your database password field to default to something that is non-blank (eg ‘a’) – even if crypt() classes the salt as invalid it will return blank which will not match this field.
  • Use something that overrides crypt() to auto-generate a salt if none is specified (eg Crypt::Password::Util’s crypt function). This won’t cover you on the case where the specified salt is invalid and so crypt() just returns a blank.
  • Assert that the user’s password field is not blank before allowing login (but you need to make sure you do this everywhere in your application).
  • For language designers: Either make sure that your crypt() function doesn’t ever return blank (throws an error on invalid output for example), OR that it automatically generates a valid salt regardless of whether or not a salt is specified (including the case where a salt is specified but is invalid).

Extracting old weird format audio files

So, I had a friend who has a load of recordings from about 10 years ago which were done on a weird dictophone. The files had the extension .FC4 which according to the internet is a legacy Amiga audio format with no more support. Great.

First thing was to run file on it:

Great. Let’s see if we can do a better job looking at a hex dump (with xxd):

So, looks like at offset 0x100 (256) we have something that is a RIFF/WAV file, then the stuff that shows as U is probably a chunk-size block or somesuch. Given the blocks of data afterwards it could probably be 16-bit single channel at a guess. Perhaps something can read that if we cut the initial header off and re-save:

Ah-ha looks like file has a clue now. Let’s try to play it:

D’oh. Opening as a raw file in audacity shows pretty much white-noise (whereas you’d have expected it to be something vaguely like speach but with blips in every so often if it was any sort of valid PCM or wave type encoding).

After searching around for a long time I discovered this post which talked about a very similar looking header and especially WAV encoding 0x350. This linked to an mplayer plugin with an acm and inf file however the ubuntu version of mplayer doesn’t support w32codecs. I tried installing this in several different ways in a windows 7 vm but couldn’t get it to work.

I then tried compiling mplayer from source only to be greeted with:

D’oh. Rather than mess around with trying a 32-bit compile or hacking the assembly I remembered I had a 10-year old laptop lying around with a very old 32-bit install of gentoo. Power it up, install the codec files and it plays them!

I then try to extract some proper PCM WAV file from the FC4 file using mencoder. But mencoder doesn’t support audio-only. I also try using the -dumpstream option in mplayer but that just dumps the encoded audio. So finally I come across the -ao pcm option which puts out a nice plain wav file that I can encode into mp3 or any other format.

Extracting images from word documents and turning into PDF

So someone recently sent me a load of Word 2007 documents that had musical scores embedded as pictures. Nice and easy to convert these (losslessly) into PDF’s with one image per page using this img2pdf script

Migrating Drupal to new server breaks login

So I just cloned a drupal site onto a new server. It all worked fine but then I couldn’t log in. I found this post which said you need to perhaps change the $cookie_domain variable but that didn’t make any difference. Finally I found the root cause of the problem – mod_rewrite wasn’t active so even though the user/login page was displaying (it was status 404 which redirects through to index.php hence into drupal) it wasn’t accepting POST requests.

Job done.

Add placeholder to all input fields in Drupal

Although this blog is wordpress I’d typically choose Drupal for doing any proper content-based website as it’s so easily flexible and supports multiple languages really easily out of the box. I was doing a site recently where we wanted to have mostly placeholders everywhere rather than form labels. This was unfortunately a bit tricky so I wrote the following code for the theme’s template.php to automatically put labels into the placeholders of input elements:

Ultra-high performance with PowerDNS

I love PowerDNS, it’s so flexible through the multiple backends that you can do pretty much whatever you want with it. When we had a nightmare weekend at 123-reg several years ago, I designed and built the next generation of the DNS infrastructure on PowerDNS because it would connect straight into our main DNS databases and we could easily change schema etc without having issues.

However, because of this flexibility sometimes PowerDNS has some performance issues. When designing the second generation of high-performance DNS servers for the HostEurope group I tested a number of other opensource servers. The highest performing by far was nsd however it used a lot of memory and wasn’t able to give us the flexibility that we required.

Looking at PowerDNS it had a number of scaling issues – up to 4 or 6 cores it was fine but after that it just couldn’t scale any further. So, working with a number of tools such as valgrind and perf I identified a number of scaling issues with the distributor code and created a series of patches which were rolled in to PowerDNS Authoritative 3.3 and 3.4 and enable PowerDNS to scale easily to 32 or 64 cores. I also came across a Linux kernel issue with locking (actually an improvement in the packet handling code) which meant that the boxes couldn’t handle more than about 250k PPS before the thread contention on the socket lock hit a limit. Fortunately with Linux 3.9 the patches that the Google guys had produced a few years ago finally got incorporated and so the SO_REUSEPORT patch was also added to PowerDNS.

Finally, I found a very high performance key/value database (lmdb) which is very reliable and fast (sqlite was limited to about 3 or 4 cores, BDB always corrupts itself, KoyotoDB is nice but updates cause a lot of contention and I’m not sure if it’s being actively developed any more). And the result is a high performance backend for PowerDNS (lmdbbackend) which can scale to over 1m QPS/server with instant updates and low response times. The best way to avoid an embarrassing DNS DDOS is to have enough capacity to respond to all queries that are thrown at the server, after that you can try to filter the big hitters. Unfortunately many anti-DDOS appliances that we tested turned out to fail quite badly on UDP-based attacks, or attacks at > 1m PPS so it’s often best to not put them in front of infrastructure that may sustain such an attack.

If you want professional consulting and assistance with setting up a high-performance resilient DNS infrastructure please contact me through my consultancy company dns-consultants.com

The little-known toJSON javascript function

I say little-known, perhaps it is not but at least I struggled for a while to figure out how to do a really simple thing – change the format that dates are sent out via a JSON call. Typically a date will be sent in ISO string format eg “2015-01-19T10:30:33.732Z” which is not ideal for my very thin backend service to dump into a database – I usually use a unix timestamp. Rather than having to add a whole load of parsing code and figuring out which was meant to be a date field and which is just a normal string I really wanted any dates to automatically serialize to a unix timestamp (or javascript timestamp which is unixtime*1000). The solution?

Thanks to Javascript’s great prototype concept (this must be a good day – usually it winds me up quite a bit!) this means that any date objects will now get serialized into the format that I wanted. To send them all as unix-times you can just do: