Persistent Hashing and Persistent Dictionaries

This module contains functionality that allows hashing with keys that remain valid across interpreter invocations, unlike Python’s built-in hashes.

This module also provides a disk-backed dictionary that uses persistent hashing.

exception pytools.persistent_dict.NoSuchEntryError

exception pytools.persistent_dict.ReadOnlyEntryError

exception pytools.persistent_dict.CollisionWarning

class pytools.persistent_dict.KeyBuilder

class pytools.persistent_dict.PersistentDict(identifier, key_builder=None, container_dir=None)

A concurrent disk-backed dictionary.

__init__(identifier, key_builder=None, container_dir=None)

Parameters

• identifier – a file-name-compatible string identifying this dictionary

• key_builder – a subclass of KeyBuilder

__getitem__(key)

__setitem__(key, value)

__delitem__(key)

clear()

store(key, value, _skip_if_present=False, _stacklevel=0)

store_if_not_present(key, value, _stacklevel=0)

fetch(key, _stacklevel=0)

remove(key, _stacklevel=0)

class pytools.persistent_dict.WriteOncePersistentDict(identifier, key_builder=None, container_dir=None, in_mem_cache_size=256)

A concurrent disk-backed dictionary that disallows overwriting/deletion.

Compared with PersistentDict, this class has faster retrieval times.

__init__(identifier, key_builder=None, container_dir=None, in_mem_cache_size=256)

Parameters

• identifier – a file-name-compatible string identifying this dictionary

• key_builder – a subclass of KeyBuilder

• in_mem_cache_size – retain an in-memory cache of up to in_mem_cache_size items

__getitem__(key)

__setitem__(key, value)

clear()

store(key, value, _skip_if_present=False, _stacklevel=0)

store_if_not_present(key, value, _stacklevel=0)

fetch(key, _stacklevel=0)