Non-cryptographic hash function
The non-cryptographic hash functions are hash functions intended for applications that do not need the rigorous security requirements of the cryptographic hash functions and therefore can be faster and less resource-intensive. Typical examples of CPU-optimized non-cryptographic hashes include FNV-1a and Murmur3. Some non-cryptographic hash functions are used in cryptographic applications ; in this case they are described as universal hash functions.
Applications and requirements
Among the typical uses of non-cryptographic hash functions are bloom filters, hash tables, and count sketches. These applications require, in addition to speed, uniform distribution and avalanche properties. Collision resistance is an additional feature that can be useful against hash flooding attacks; simple NCHFs, like the cyclic redundancy check, have essentially no collision resistance and thus cannot be used with an input open to manipulation by an attacker.NCHFs are used in diverse systems: lexical analyzers, compilers, databases, communication networks, video games, DNS servers, filesystems—anywhere in computing where there is a need to find the information very quickly.
Estébanez et al. list the "most important" NCHFs:
- The Fowler–Noll–Vo hash function was created by Glenn Fowler and Phong Vo in 1991 with contributions from Landon Curt Noll. FNV with its two variants, FNV-1 and FNV-1a, is very widely used in Linux, FreeBSD OSes, DNS servers, NFS, Twitter, PlayStation 2, and Xbox, among others.
- lookup3 was created by Robert Jenkins. This hash is also widely used and can be found in PostgreSQL, Linux, Perl, Ruby, and Infoseek.
- SuperFastHash was created by Paul Hsieh using ideas from FNV and lookup3, with one of the goals being a high degree of avalanche effect. The hash is used in WebKit.
- MurmurHash2 was created by Austin Appleby in 2008 and is used in libmemcached, Maatkit, and Apache Hadoop.
- DJBX33A. This very simple multiplication-and-addition function was proposed by Daniel J. Bernstein. It is fast and efficient during initialization. Many programming environments based on PHP 5, Python, and ASP.NET use variants of this hash. The hash is easy to flood, exposing the servers.
- BuzHash was created by Robert Uzgalis in 1992. It is designed around a substitution table and can tolerate extremely skewed distributions on the input.
- DEK is an early multiplicative hash based on a proposal by Donald Knuth and is one of the oldest hashes that is still in use.
Design
Many NCHFs have a relatively small result size : large result size does not increase the performance of the target applications, but slows down the calculation, as more bits need to be generated.