Show HN: Ldump – serialize any Lua data

2025-01-319:299641github.com

Serialize any lua data. Contribute to girvel/ldump development by creating an account on GitHub.

API | Overloading serialization | Development

ldump is a flexible serializer, able to serialize any data, starting with circular references, tables as keys, functions with upvalues, metatables and ending with coroutines, threads and userdata (by defining how they should be serialized). It outputs valid Lua code that recreates the original object, doing the deserialization through load(data)(). It aims for functionality and flexibility instead of speed and size, allowing full serialization of complex data, such as video game saves. The output is large, but can be drastically reduced with modern compression algorithms.

Inspired by Ser. Supports Lua 5.1, 5.2, 5.3, 5.4 and LuaJIT. Tested for edge cases, such as joined upvalues and _ENV redefinition. Fully annotated in compatibility with LuaLS.

Type Support
nil, boolean, number, string full
function full
userdata user-defined
thread user-defined
table full
metatables* full
local ldump = require("ldump") local upvalue = 42
local world = { name = "New world", get_answer = function() return upvalue end,
} local serialized_data = ldump(world) -- serialize to a string
local loaded_world = load(serialized_data)() -- deserialize the string

See as a test at /tests/test_use_case.lua:7

local ldump = require("ldump") -- basic tables
local game_state = { player = {name = "Player"}, boss = {name = "Boss"},
} -- circular references & tables as keys
game_state.deleted_entities = { [game_state.boss] = true,
} -- functions even with upvalues
local upvalue = 42
game_state.get_answer = function() return upvalue end -- fundamentally non-serializable types if overriden
local create_coroutine = function() return coroutine.wrap(function() coroutine.yield(1337) coroutine.yield(420) end)
end -- override serialization
game_state.coroutine = create_coroutine()
ldump.serializer.handlers[game_state.coroutine] = create_coroutine local serialized_data = ldump(game_state) -- serialize
local loaded_game_state = load(serialized_data)() -- deserialize

See as a test at /tests/test_use_case.lua:23

Copy the raw contents of init.lua from the latest release into your lib/ldump.lua or git clone -b v1.2.0 https://github.com/girvel/ldump inside the lib/ — you still would be able to do require("ldump").


Read the original article

Comments

  • By lifthrasiir 2025-01-3110:541 reply

    Maybe I'm too pedantic but allowing anything to be "deserialized", which equals to "evaluated" here, is not secure. I think it only has to accept a very limited subset of Lua anyway, so you may switch to a non-Lua format which is made easy to parse. That way the library has a total control over what is being evaluated.

    • By girvel 2025-01-3111:133 reply

      This is an interesting thought. Currently, it is unsafe and intended to load only the files you trust. I should definitely include a warning into README.

      Overall, it would be nice to make it safer. I don't think switching to non-Lua format would make it safer, because it is intended to serialize functions too, which can have arbitrary code even if everything else would be stored as data. Maybe it is possible to make a function like `ldump.safe_load` restricting `load`'s environment, so it wouldn't have access to debug/os/io modules.

      • By gvx 2025-01-3111:46

        You could take a look at SELÖVE, a (severely out of date) fork of LÖVE that is intended to make it safe to run arbitrary .love games. (It used to be on bitbucket, but it looks like it's gone? I'm not sure if I have the repo locally :/)

        Running arbitrary code was such a problem that I just completely ruled it out for bitser. Instead of serializing functions, you can register safe functions as resources. This doesn't solve the upvalue problem, though.

      • By girvel 2025-01-3115:021 reply

        I looked into it, and Lua allows limiting the environment when `load`ing -- through `env` argument since 5.2 or through setfenv before. I will add a helper function to produce a minimal needed environment for safe loading and a documentation page about safety.

      • By lifthrasiir 2025-01-3111:28

        Yeah, you would need an allowlist for functions. Using bytecode would make it much harder, I haven't given deep thought yet.

  • By ithkuil 2025-01-3110:151 reply

    "lump" would have been a nice name

  • By JourneyJourney 2025-01-3111:283 reply

    I'm afraid I spent too much time with LUA lately and fell in love with its simplicity. Kinda hard to go back to OOP after that.

    • By nicoloren 2025-01-3111:401 reply

      Same for me, I used Lua for a desktop software for a client and I enjoyed it a lot!

      I'm thinking of starting to dev a game with LOVE2D just to have an excuse to use Lua.

      • By girvel 2025-01-3114:54

        LOVE2D is a great gamedev framework, I can not recommend it enough. It is so pleasant to work with.

    • By je42 2025-02-0110:08

      I find that most LUA code sits between functional, imperative and OOP.

    • By aldanor 2025-01-3113:33

      It's simple until you dig deep into meta tables lol

HackerNews