Lua memory leak in C process
c, lua, memory-leaks
Solution
Lua faithfully frees unreachable objects by calling the supplied deallocation function (by default `realloc(block, 0)`). It looks like `libc` allocator is struggling to return unused memory, possibly due to high fragmentation. Looking at the `strace` output (I've got roughly the same numbers with Lua 5.1.4 on 64-bit Debian 6), C runtime chooses to allocate using `brk` with small increments, but no deallocation (calling `brk` with lower value) follows. However, if you insert `malloc_trim(M_TOP_PAD)` before entering infinite loop, you'll see in the `top` output that resident size drops drastically to ~5M and `strace` reveals that data segment was indeed trimmed with `brk`. Using custom allocator (e.g. pool-based) or tuning `malloc` parameters will probably help in this situation.
Problem
I have a C program running with Lua. Although I try to use lua_gc() to get and control memory usage of Lua, C process memory usage is still high. The C process uses over 150MB memory even though Lua say that it uses only 4MB memory. I also try to use my l_alloc() function to track Lua memory allocation, but the result is the same with the memory usage Lua told by calling lua_gc(LUA_GCCOUNT) and lua_gc(LUA_GCCOUNTB). After calling lua_close() to close the Lua environment, the process memory is down and looks fine. Therefore, I think the 'lost memory' is still controlled by Lua not C program. Here is the sample C code. It creates a Lua environment, calls Lua function to purge data and then check memory usage. ``` int main() { int rc; uint64_t gc_mem_usage; lua_State* Lua = luaL_newstate(); luaL_openlibs(Lua); lua_gc(Lua, LUA_GCSTOP, 0); luaL_dofile(Lua, "test.lua"); gc_mem_usage = ((uint64_t)lua_gc(Lua, LUA_GCCOUNT, 0) << 10) + lua_gc(Lua, LUA_GCCOUNTB, 0); printf("Lua mem usage: [%" PRIu64 "] Bytes\n", gc_mem_usage); lua_getglobal(Lua, "command_handler"); lua_pushstring(Lua, "CC"); rc = lua_pcall(Lua, 1, 0, 0); if (rc != 0 ) { printf("function error\n"); return; } lua_settop(Lua, 0); // do full gc lua_gc(Lua, LUA_GCCOLLECT, 0); lua_gc(Lua, LUA_GCCOLLECT, 0); // I don't know why it has different result by calling full gc twice sleep(1); printf("-------------After GC ----------------------\n"); gc_mem_usage = ((uint64_t)lua_gc(Lua, LUA_GCCOUNT, 0) << 10) + lua_gc(Lua, LUA_GCCOUNTB, 0); printf("Lua mem usage: [%" PRIu64 "] Bytes\n", gc_mem_usage); // infinite-loop while(1); } ``` Lua sample code: ``` local abc = {} function command_handler(cmd) if (cmd == "CC") then abc = {} end end for i =0, 2000000 do abc[i] = "ABC" .. i .. "DEF" end ``` output: ``` Lua mem usage: [204913817] Bytes -------------After GC ---------------------- Lua mem usage: [4219342] Bytes ``` The output told me that Lua memory usage is down after GC, but the memory usage of this C process is still very high (193.7MB) by checking atop continuously. ``` PID MINFLT MAJFLT VSTEXT VSIZE RSIZE VGROW RGROW MEM CMD 1/1 4622 1 0 3K 193.7M 183.9M 0K 4K 18% a.out ``` Is there any solution to reduce the C process memory usage? My environment is Lua 5.1.4 running in Ubuntu/CentOS.