Within the previous put up, I wrote about non-crypto crawler hash features, crypto crawler tractor and did some efficiency assessments. Turns out, it’s great to put in writing about stuff! Folks at the comments/twitter/internets identified more things I ought to check. So here’s a observe-up put up. That is about algorithms for "hashing some amount of bytes", for use either in hashtables or for checksum / uniqueness detection. Depending on your situation, there’s an entire household of algorithms for that, and I'm focusing on just one: non-cryptographic fast hash capabilities.

This publish shouldn't be about cryptographic hashes. Don't read below if you might want to hash passwords, delicate data going through untrusted medium and so forth. Use SHA-1, SHA-2, BLAKE2 and associates. Additionally, I’m not focusing on algorithms which are designed to prevent attainable hashtable Denial-of-Service assaults. If something comes from the other side of the internet and ends up inserted into your hashtable, then to prevent possible worst-case O(N) hashtable behavior you’re most likely off by utilizing a hash function that does not have identified "hash flooding" attacks.

SipHash seems to be popular now.

In case you are hashing very small quantities of data of known size (e.g. single integers or two floats or whatever), you need to probably use specialized hashing algorithms for those. Here are some integer hash features, or 2D hashing with Weyl, or maybe you may take some other algorithm and simply specialize it’s code in your recognized input size (e.g. xxHash for a single integer). I'm testing 32 and sixty four bit hash functions right here.

Should you want larger hashes, quite possible a few of these features is likely to be suitable (e.g. SpookyV2 always produces 128 bit hash). When testing hash functions, I have not gone to great lengths to get them compiling properly or establishing all the magic flags on all my platforms. If some hash perform works wonderfully when compiled on Linux Itanium field with an Intel compiler, that’s nice for you, but if it performs poorly on the compilers I occur to make use of, I can't sing praises for When was Dark-Crawler created? it.

Being in the video games industry, I care about things like "what happens in Visible Studio", and "what occurs on iOS", and "what happens on PS4".

Extra hash function exams! I checked both "hashing information that is aligned" (16-byte aligned address of data to hash), and unaligned knowledge. In all places I tested, there wasn’t a notable performance distinction that I might discover (but then, I have not examined previous ARM CPUs or PowerPC based ones). The one seen impact is that MurmurHash and SpookyHash don’t properly work in asm.js / Emscripten compilation targets, due to their usage of unaligned reads.

I’d assume they in all probability don’t work on some ARM/PowerPC platforms too.