Persistent Hashing and Persistent Dictionaries

This module contains functionality that allows hashing with keys that remain valid across interpreter invocations, unlike Python’s built-in hashes.

This module also provides a disk-backed dictionary that uses persistent hashing.

exception pytools.persistent_dict.NoSuchEntryError[source]
exception pytools.persistent_dict.ReadOnlyEntryError[source]
exception pytools.persistent_dict.CollisionWarning[source]
class pytools.persistent_dict.KeyBuilder[source]
class pytools.persistent_dict.PersistentDict(identifier, key_builder=None, container_dir=None)[source]

A concurrent disk-backed dictionary.

__init__(identifier, key_builder=None, container_dir=None)[source]
Parameters:
  • identifier – a file-name-compatible string identifying this dictionary
  • key_builder – a subclass of KeyBuilder
__getitem__(key)
__setitem__(key, value)
__delitem__(key)[source]
clear()
store(key, value, _skip_if_present=False, _stacklevel=0)[source]
store_if_not_present(key, value, _stacklevel=0)
fetch(key, _stacklevel=0)[source]
remove(key, _stacklevel=0)[source]
class pytools.persistent_dict.WriteOncePersistentDict(identifier, key_builder=None, container_dir=None, in_mem_cache_size=256)[source]

A concurrent disk-backed dictionary that disallows overwriting/deletion.

Compared with PersistentDict, this class has faster retrieval times.

__init__(identifier, key_builder=None, container_dir=None, in_mem_cache_size=256)[source]
Parameters:
  • identifier – a file-name-compatible string identifying this dictionary
  • key_builder – a subclass of KeyBuilder
  • in_mem_cache_size – retain an in-memory cache of up to in_mem_cache_size items
__getitem__(key)
__setitem__(key, value)
clear()[source]
store(key, value, _skip_if_present=False, _stacklevel=0)[source]
store_if_not_present(key, value, _stacklevel=0)
fetch(key, _stacklevel=0)[source]