Module hash4j

Package com.dynatrace.hash4j.similarity

Interface SimilarityHashing

public interface SimilarityHashing

Various implementations of similarity hash algorithms.

Method Summary

Static Methods

Modifier and Type	Method	Description
static SimilarityHashPolicy	fastSimHash(int numberOfComponents)	Returns a SimilarityHashPolicy for FastSimHash, which is a fast implementation of the SimHash algorithm as introduced in Moses S.
static SimilarityHashPolicy	fastSimHash(int numberOfComponents, FastSimHashVersion fastSimHashVersion)	Returns a SimilarityHashPolicy for FastSimHash, which is a fast implementation of the SimHash algorithm as introduced in Moses S.
static SimilarityHashPolicy	minHash(int numberOfComponents, int bitsPerComponent)	Returns a SimilarityHashPolicy for a modified version of b-bit minwise hashing described in Ping Li and Christian König, B-Bit minwise hashing, 2010..
static SimilarityHashPolicy	minHash(int numberOfComponents, int bitsPerComponent, MinHashVersion minHashVersion)	Returns a SimilarityHashPolicy for a modified version of b-bit minwise hashing described in Ping Li and Christian König, B-Bit minwise hashing, 2010..
static SimilarityHashPolicy	simHash(int numberOfComponents)	Returns a SimilarityHashPolicy for SimHash as introduced in Moses S.
static SimilarityHashPolicy	simHash(int numberOfComponents, SimHashVersion simHashVersion)	Returns a SimilarityHashPolicy for SimHash as introduced in Moses S.
static SimilarityHashPolicy	superMinHash(int numberOfComponents, int bitsPerComponent)	Returns a SimilarityHashPolicy for SuperMinHash described in Otmar Ertl, SuperMinHash - A New Minwise Hashing Algorithm for Jaccard Similarity Estimation, 2017..
static SimilarityHashPolicy	superMinHash(int numberOfComponents, int bitsPerComponent, SuperMinHashVersion superMinHashVersion)	Returns a SimilarityHashPolicy for SuperMinHash described in Otmar Ertl, SuperMinHash - A New Minwise Hashing Algorithm for Jaccard Similarity Estimation, 2017..

Method Details

minHash

static SimilarityHashPolicy minHash(int numberOfComponents,
 int bitsPerComponent)

Returns a SimilarityHashPolicy for a modified version of b-bit minwise hashing described in Ping Li and Christian König, B-Bit minwise hashing, 2010..

Parameters:

numberOfComponents - the number of components of the signature

bitsPerComponent - the number of bits per component in the range [1, 64]

Returns:

a policy

minHash

static SimilarityHashPolicy minHash(int numberOfComponents,
 int bitsPerComponent,
 MinHashVersion minHashVersion)

Returns a SimilarityHashPolicy for a modified version of b-bit minwise hashing described in Ping Li and Christian König, B-Bit minwise hashing, 2010..

Specifying the version of the implementation ensures compatibility with later hash4j versions that may change the default implementation. This is especially important if the signatures are persisted.

Parameters:

numberOfComponents - the number of components of the signature

bitsPerComponent - the number of bits per component in the range [1, 64]

minHashVersion - the version of the implementation

Returns:

a policy

superMinHash

static SimilarityHashPolicy superMinHash(int numberOfComponents,
 int bitsPerComponent)

Returns a SimilarityHashPolicy for SuperMinHash described in Otmar Ertl, SuperMinHash - A New Minwise Hashing Algorithm for Jaccard Similarity Estimation, 2017..

Parameters:

numberOfComponents - the number of components of the similarity hash

bitsPerComponent - the number of bits per component in the range [1, 64]

Returns:

a policy

superMinHash

static SimilarityHashPolicy superMinHash(int numberOfComponents,
 int bitsPerComponent,
 SuperMinHashVersion superMinHashVersion)

Returns a SimilarityHashPolicy for SuperMinHash described in Otmar Ertl, SuperMinHash - A New Minwise Hashing Algorithm for Jaccard Similarity Estimation, 2017..

Specifying the version of the implementation ensures compatibility with later hash4j versions that may change the default implementation. This is especially important if the signatures are persisted.

Parameters:

numberOfComponents - the number of components of the similarity hash

bitsPerComponent - the number of bits per component in the range [1, 64]

superMinHashVersion - the version of the implementation

Returns:

a policy

fastSimHash

static SimilarityHashPolicy fastSimHash(int numberOfComponents)

Returns a SimilarityHashPolicy for FastSimHash, which is a fast implementation of the SimHash algorithm as introduced in Moses S. Charikar, Similarity estimation techniques from rounding algorithms, 2002.

To compute the SimHash signature, a counter is used for each signature component to count how many times a corresponding bit is set in the hash values of all elements of the given set. Unlike other SimHash implementations that iterate over all the individual bits of all the hash values of the elements, FastSimHash processes 8 bits at once thanks to some bit tricks, which results in a significant speedup.

Parameters:

numberOfComponents - the number of components of the similarity hash

Returns:

a policy

fastSimHash

static SimilarityHashPolicy fastSimHash(int numberOfComponents,
 FastSimHashVersion fastSimHashVersion)

Returns a SimilarityHashPolicy for FastSimHash, which is a fast implementation of the SimHash algorithm as introduced in Moses S. Charikar, Similarity estimation techniques from rounding algorithms, 2002.

To compute the SimHash signature, a counter is used for each signature component to count how many times a corresponding bit is set in the hash values of all elements of the given set. Unlike other SimHash implementations that iterate over all the individual bits of all the hash values of the elements, FastSimHash processes 8 bits at once thanks to some bit tricks, which results in a significant speedup.

Specifying the version of the implementation ensures compatibility with later hash4j versions that may change the default implementation. This is especially important if the signatures are persisted.

Parameters:

numberOfComponents - the number of components of the similarity hash

fastSimHashVersion - the version of the implementation

Returns:

a policy

simHash

static SimilarityHashPolicy simHash(int numberOfComponents)

Returns a SimilarityHashPolicy for SimHash as introduced in Moses S. Charikar, Similarity estimation techniques from rounding algorithms, 2002.

As SimHash is significantly slower than FastSimHash, prefer using fastSimHash(int) instead!

Parameters:

numberOfComponents - the number of components of the similarity hash

Returns:

a policy

simHash

static SimilarityHashPolicy simHash(int numberOfComponents,
 SimHashVersion simHashVersion)

Returns a SimilarityHashPolicy for SimHash as introduced in Moses S. Charikar, Similarity estimation techniques from rounding algorithms, 2002.

As SimHash is significantly slower than FastSimHash, prefer using fastSimHash(int, FastSimHashVersion) instead!

Specifying the version of the implementation ensures compatibility with later hash4j versions that may change the default implementation. This is especially important if the signatures are persisted.

Parameters:

numberOfComponents - the number of components of the similarity hash

simHashVersion - the version of the implementation

Returns:

a policy