| name | pst-save-pipeline |
| description | Deep internals of the palsav save serialization engine + palooz Oodle bindings, and exactly how PST (palworld_aio GUI) consumes them. Load when touching any save parse/roundtrip/compression code, debugging roundtrip drift, editing rawdata decoders, or tracing the SAV<->GVAS<->JSON flow. |
PST Save Pipeline — palsav + palooz internals
palsav (pkg palsav-flex v0.2.0, MIT) = the save serialization engine. Fork of oMaN-Rod/palworld-save-tools, retuned with Oodle compression. palooz (separate sub-pkg under src/palsav/palooz/, v0.2.0, GPL-3.0) = compiled C++ Python bindings to the Oodle decompressor (Kraken-only compress + universal Oodle decompress), x64 + ARM64 via SIMDe.
Layered architecture (bottom = bytes, top = JSON)
palooz (C++ ext) raw Oodle compress/decompress
palsav.compressor OozLib + Zlib; .sav container header parse/build
├ enums.py SaveType{CNK=48,PLM=49,PLZ=50}, MagicBytes{PlZ,PlM,CNK}
├ __init__.py Compressor: _parse_sav_header, check_sav_format, build_sav
├ oozlib.py OozLib: wraps palooz, Kraken/Normal level, loads platform lib
└ zlib.py Zlib: PLZ=double-zlib, single for CNK
palsav.core decompress_sav_to_gvas / compress_gvas_to_sav (public entry, dispatches on SaveType)
palsav.archive FArchiveReader/FArchiveWriter (UE binary format, 809 lines — biggest file)
palsav.gvas GvasHeader + GvasFile (UE GVAS container)
palsav.paltypes PALWORLD_TYPE_HINTS + PALWORLD_CUSTOM_PROPERTIES + DISABLED_PROPERTIES
palsav.rawdata.* per-entity decode/encode (character, group, base_camp, work, map_object, ...)
palsav.json_tools orjson dump/load + base64 byte-tag roundtripping
palsav.commands.* CLI subcommands (convert, backup, diag, roundtrip_validation)
palsav.cli `palsav` CLI dispatcher (PYTHONHASHSEED=0 forced)
The .sav container header
uncompressed_len(4,LE) | compressed_len(4,LE) | magic(3) | save_type(1) | payload
- CNK (0x30) has a NESTED header: first 12 bytes are outer, then a second (uncompressed_len, compressed_len, magic, type) at offset 12-24, data_offset=24. Handled in
Compressor._parse_sav_header.
- Three formats: PLZ=double-zlib (original Steam), PLM=Oodle (newer), CNK=chunked/zlib.
The GVAS container (gvas.py)
GvasHeader: magic 1396790855 (validated), save_game_version must be 3, custom_version_format must be 3, engine version tuple, custom_versions array, save_game_class_name (used to pick compression on write). GvasFile: header + properties dict + trailer (must be 4 zero bytes; logged if not — means incomplete parse). read/load/write/dump for both binary<->object and object<->dict.
PALWORLD_CUSTOM_PROPERTIES — the dispatch table (paltypes.py)
Maps a dotted property PATH -> (decode_fn, encode_fn) pair from rawdata modules. When FArchiveReader hits that path while walking the property tree, it calls the custom decoder instead of the generic one. 17 registered paths, e.g.:
.worldSaveData.GroupSaveDataMap → group.worldSaveData.CharacterSaveParameterMap.Value.RawData → character.worldSaveData.ItemContainerSaveData.Value.RawData → item_container.worldSaveData.BaseCampSaveData.Value.RawData → base_camp.worldSaveData.WorkSaveData → work.worldSaveData.MapObjectSaveData → map_object- etc.
PALWORLD_TYPE_HINTS = path->type overrides for structurally ambiguous properties (disambiguates StructProperty vs Guid at specific paths). DISABLED_PROPERTIES = {'.worldSaveData.BaseCampSaveData.Value.ModuleMap'} — registered but skipped (likely broken for current game version). FArchiveReader checks this set before dispatching.
The rawdata decode/encode pattern (character.py is canonical)
def decode(reader, type_name, size, path):
# 1. read the raw ArrayProperty
value = reader.property(type_name, size, path, nested_caller_path=path)
# 2. extract the byte array
char_bytes = value['value']['values']
# 3. parse the entity-specific structure via an internal_copy reader
value['value'] = decode_bytes(reader, char_bytes)
return value
def decode_bytes(parent_reader, char_bytes):
reader = parent_reader.internal_copy(bytes(char_bytes), debug=False)
char_data = {'object': reader.properties_until_end(),
'unknown_bytes': reader.byte_list(4),
'group_id': reader.guid(),
'trailing_bytes': reader.byte_list(4)}
if not reader.eof():
char_data['trailing_unknown_bytes'] = reader.read_to_end() # roundtrip-critical
return char_data
Roundtrip rule (sacred): every trailing/unknown byte MUST be captured and re-emitted verbatim. The trailing_unknown_bytes field exists precisely so forward-compatible saves (new game-version fields appended) survive a roundtrip. This is the #1 source of roundtrip drift — see history.
How PST (palworld_aio GUI) consumes palsav — THREE paths
1. Direct import (clean): from palsav import json_tools (bootup.py, i18n), from palsav.gvas import GvasFile, from palsav.core import decompress_sav_to_gvas (save_manager.py).
2. Via src/import_libs.py (star-import re-export shim): brings palsav.archive.*, palsav.paltypes.*, palsav.gvas.*, palsav.json_tools.*, palsav.rawdata.group into app namespace. Most palworld_aio modules inherit these symbols through import_libs.
3. Via palworld_aio/utils.py wrappers (the cleanest seam — use this):
sav_to_gvas_wrapper(path) → decompress + GvasFile.read → wraps in GvasFileWrapper (dict-like adapter so the app edits .properties as a plain dict). Uses mmap for >100MB files.wrapper_to_sav(wrapper, path) → extracts underlying GvasFile → compresses → writes.sav_to_gvasfile(path) → returns raw GvasFile (no wrapper).sav_to_json(path) / json_to_sav(j, path) → full dict roundtrip.gvasfile_to_sav(gvas_file, path) → write a GvasFile back to .sav.
constants.loaded_level_json is a GvasFileWrapper — the entire app edits it in place, then wrapper_to_sav writes it back.
SKP_PALWORLD_CUSTOM_PROPERTIES — the GUI's performance override (palobject.py:232)
PST clones PALWORLD_CUSTOM_PROPERTIES then OVERRIDES 6 paths with (skip_decode, skip_encode) NO-OPS (raw bytes pass through opaque):
- MapObject: WorldLocation, WorldRotation, WorldScale3D, Model.EffectMap
- FoliageGridSaveDataMap (entire map)
- MapObjectSpawnerInStageSaveData (entire)
Why: these are huge blobs the GUI never edits — skipping decode saves significant time/memory. Consequence: the GUI cannot edit those fields (they're opaque). The CLI (palsav convert) uses the FULL property table and decodes everything. This is a deliberate CLI≠GUI divergence. If someone needs to edit foliage/spawner/location data, they must use the CLI or remove the skip override.
Compression selection on write (utils.py:70)
t = 50 if 'Pal.PalworldSaveGame' in g.header.save_game_class_name else 49
50=PLZ(double-zlib), 49=PLM(Oodle). So world-class saves → zlib, others → Oodle. NOTE the lowercase 'w' in 'PalworldSaveGame' — verify actual UE class names are lowercase-matching (UE class names are often PascalCase like PalWorldSaveGame); if mismatched, format selection silently falls back to Oodle.
Booth concrete model schema — map_concrete_model.py (Jun 25)
RE'd by comparing locked vs unlocked Level.sav bytes for both booth types:
ItemBooth (old: trailing_bytes 20B blob)
Unknown structure within the 20 trailing bytes. Byte-level diff of locked/unlocked ATest saves revealed only byte[12] differs (1=locked, 0=unlocked). Key finding: private_lock_player_uid is identically non-zero in both states → lock is NOT the GUID, it's is_private_lock at byte[12].
| New field | Size | Description |
|---|
unknown_before_lock | 12B | opaque prefix before lock flag |
is_private_lock | 1B u8 | 0=unlocked, 1=locked |
unknown_after_lock | 7B | opaque suffix |
PalBooth (old: unknown_bytes 236B blob)
Byte[224] differs (1=locked, 0=unlocked). No private_lock_player_uid field exists in the model at all.
| New field | Size | Description |
|---|
unknown_prefix | 224B | everything before the lock flag |
is_private_lock | 1B u8 | 0=unlocked, 1=locked |
unknown_suffix | 11B | remaining bytes |
Roundtrip verified
Locked save → decode → set is_private_lock=0 → encode → reload → confirmed is_private_lock=0. Only the lock flag byte changes; everything else byte-identical. Binary blob manipulation (base64 patch) is gone — lock is set through the typed field.
unlock_all_private_chests (func_manager.py)
deep_unlock skips both booth types (no GUID zeroing)- After deep_unlock, iterates
MapObjectSaveData.values, sets is_private_lock=0 on booth entries
- No base64/binary patch, no
private_lock_player_uid zeroing on booths
History (the sacred-roundtrip theme)
- palsav began bundled with pyooz (Oodle submodule), then locally bundled.
ee362bb (cyrix) — THE pivotal commit: replaced palsav core with "proven 1:1-roundtrip palworld_save_tools code" (copied 40 files). An earlier "modernized" core had BROKEN byte-exact roundtrip; this reverted to the proven code. Verified: player save 30,921 bytes 1:1, Level 4,215,854 bytes 1:1.1e3eff6 (cyrix) — renamed pyooz→palooz, stripped to Kraken-only compress + universal Oodle decompress, removed ~288 unused SIMDe headers.- Recent commits (1a9c328 guild integrity, 5152beb use_u8 fallback, 9ac23cc dynamic item IDs) are ALL about defending roundtrip fidelity against game-format drift.
- Lesson: any change to archive.py, a rawdata decoder, or the property dispatch MUST be validated with
pytest -m slow (the 3.2MB V1 beta roundtrip) AND ideally a real-save byte diff. Roundtrip drift is the project's recurring failure mode.
- Jun 22 —
group.py:_u8_flag: Pre-V1_MARKER guild format has NO _u8_flag bytes between player entries. Old code always read a flag byte when not at EOF, consuming first byte of next player's GUID → 1-byte shift per player → corruption or _raw_tail fallback. Fix: if group_data.get('_has_v1_marker') and not sub.eof().
Validation tooling (palsav.commands)
roundtrip_validation.py — SAV→JSON→SAV byte comparison.diag.py — read-only save diagnostics (header, format, property inventory).resave_test.py — resave + compare.palsav validate <save> / palsav diag <save> from CLI.
Working rules
- To add a new game property: write a rawdata module with decode/encode, register in PALWORLD_CUSTOM_PROPERTIES (paltypes.py). Add type hints to PALWORLD_TYPE_HINTS if structurally ambiguous.
- To edit a skipped property in the GUI: remove/override its entry in SKP_PALWORLD_CUSTOM_PROPERTIES (palobject.py).
- ALWAYS run
pytest -m slow after touching archive.py / rawdata / paltypes.
- palooz is a compiled C++ extension — platform-specific (win/linux-x64/linux-arm64/mac-x64/mac-arm64). Built via setup.py with -O3 -flto. Source in src/palsav/palooz/ooz/.
