Skip to content

Latest commit

 

History

History
125 lines (92 loc) · 6.89 KB

default.md

File metadata and controls

125 lines (92 loc) · 6.89 KB

Home › Serializers › StandardSerializer

Default Serializer (MessagePack)

The StandardSerializer is cachekit's general-purpose serializer. It is used automatically when no serializer is specified on a @cache decorator. It combines MessagePack encoding with optional LZ4 compression and xxHash3-64 integrity checksums via cachekit's Rust ByteStorage layer.

The registry aliases for this serializer are "default" and "std" ("auto" is AutoSerializer). The class name is StandardSerializer.

Overview

Best for:

  • General Python objects (dicts, lists, tuples)
  • Mixed data types
  • Scalar values and nested structures
  • Small to medium-sized data
  • Binary data (bytes)

Performance characteristics:

  • Serialization and deserialization: no published timing; the instruction budgets count their instructions
  • Memory overhead: Low
  • Network overhead: Compact binary format

Basic Usage

StandardSerializer is used automatically — no configuration needed:

from cachekit import cache

# StandardSerializer is used automatically (no configuration needed)
@cache
def get_user_data(user_id: int):
    return {
        "id": user_id,
        "name": "Alice",
        "scores": [95, 87, 91],
        "metadata": {"tier": "premium"}
    }

Registration Aliases

StandardSerializer can be referenced by alias when configuring serializers:

Alias Resolves To
"default" StandardSerializer — language-agnostic MessagePack
"std" StandardSerializer — explicit alias
"auto" AutoSerializer — Python-specific types (NumPy, pandas, set, UUID)

Note

"auto" resolves to AutoSerializer, which adds Python-specific type detection (NumPy arrays, pandas DataFrames, sets, UUID). "default" / "std" resolves to StandardSerializer, a language-agnostic MessagePack variant designed for cross-language interoperability. Both are based on MessagePack + Rust ByteStorage.

Type Support Matrix

❌ means serialize() raises TypeError when you call the serializer directly. A @cache-decorated call with a backend does not raise: it returns the value, caches nothing, and logs the failure (Troubleshooting → Serialization Failures).

Type Supported Notes
dict ✅ Nested structures
list ✅ Any element types
tuple ✅ Round-trips as list (MessagePack has no tuple type)
str ✅ Unicode
int ✅ Arbitrary precision
float ✅ 64-bit
bool ✅
None ✅
bytes ✅ Binary data — only serializer that handles raw bytes
datetime, date, time ✅ ISO-8601 string in a sentinel map, e.g. {"__datetime__": true, "value": "<ISO-8601>"} (protocol spec); revival of these maps varies across SDKs
numpy.ndarray ❌ Use AutoSerializer (serializer="auto")
pandas.DataFrame ❌ Use ArrowSerializer or AutoSerializer
pandas.Series ❌ Use AutoSerializer
Pydantic models ❌ See Caching Pydantic Models
set / frozenset ❌ Convert to list first
Custom classes ❌ Convert to a dict (dataclasses.asdict, model_dump) or use a custom serializer

Compression and Integrity

StandardSerializer automatically handles:

  • LZ4 compression — shrinks repetitive or record-heavy payloads of 64 bytes or more; a smaller value is stored uncompressed and grows by the envelope's fixed overhead (about 25 to 30 bytes)
  • xxHash3-64 checksums — integrity verification on deserialization

Both are handled by the Rust ByteStorage layer and are on by default. With integrity_checking=False (as @cache.minimal sets) the serializer writes plain MessagePack instead: no compression, no checksum.

Cross-config reads. A reader with integrity checking off does not unwrap envelopes. Handed an entry written with integrity checking on, it raises SerializationError (Cache entry was written with integrity checking on but this reader has integrity checking disabled) instead of returning the envelope's internal fields as your value. Under default key generation this never happens through @cache, because the integrity flag is part of the key. A custom key=, fast_mode or the direct serializer API can cross the two configs. Through @cache the entry's header records the envelope, so it is refused at any size. Without that header, as in a direct deserialize(data) call, the reader verifies the envelope's checksum instead, and never judges by the value's shape. So a cached value that merely looks like an envelope, such as [b"\x89PNG", [255, 0, 0, 255, 0, 255, 0, 255], 4096, "rgb"], still round-trips. Verifying means decompressing, so the reader only verifies entries that declare at most 256 KiB. A larger integrity-on entry read with no metadata comes back as its fields, and so does one whose checksum no longer matches. An encrypted read checks only for the legacy envelope layout. The entry's compressed flag is bound into the encryption AAD, and the only writer that ever sealed an envelope with it false was AutoSerializer before 0.12.0, which used the legacy layout. So such an old entry is refused and recomputed when it declares at most 256 KiB (a larger one comes back as its fields), and any other decrypted value comes back whatever its shape.

Corruption detection, not tamper resistance. xxHash3-64 is non-cryptographic: an attacker with backend write access can forge a valid checksum for arbitrary bytes. The checksum catches bit rot and storage bugs. For tamper resistance, use encryption (@cache.secure, or an explicit encryption= option), which authenticates every byte with AES-256-GCM.

@cache
def get_large_dict():
    return {"large": "data" * 1000}  # Automatically compressed

Performance Optimization Tips

  1. Use appropriate TTL to balance freshness vs cache hit rate:

    @cache(ttl=3600)  # 1 hour
    def get_cached_data():
        return expensive_computation()
  2. For DataFrames, use ArrowSerializer: StandardSerializer rejects them.


See Also