Skip to main content

​
mellea.backends.cache

Caching strategies.

​
Classes

​
Cache

A Cache for storing model state (e.g., kv cache). Methods:

​
put

put(self, key: str, value: Any)
Inserts into the cache. May result in eviction of other cached values.

​
get

get(self, key: str) -> Any | None
Retrieves a value from the cache. Returns None if the id has no cached value. May impact which cache values are evicted.

​
current_size

current_size(self) -> int
Returns the number of things currently in the cache. Mostly useful for debugging.

​
SimpleLRUCache

A simple LRU cache. Methods:

​
current_size

current_size(self)
Just return the size of the key set. This isn’t necessarily safe.

​
get

get(self, key: str) -> Any | None
Gets a value from the cache.

​
put

put(self, key: str, value: Any)
Put a value into the cache.