Need multi-threading memory manager

delphi, delphi-xe

Solution

Our SynScaleMM is still experimental.

EDIT: Take a look at the more stable ScaleMM2 and the brand new SAPMM. But my remarks below are still worth following: the less allocation you do, the better you scale!

But it worked as expected in a multi-threaded server environment. Scaling is much better than FastMM4, for some critical tests.

But the Memory Manager is perhaps not the bigger bottleneck in Multi-Threaded applications. FastMM4 could work well, if you don't stress it.

Here are some (not dogmatic, just from experiment and knowledge of low-level Delphi RTL) advice if you want to write FAST multi-threaded application in Delphi:

- Always use `const` for string or dynamic array parameters like in `MyFunc(const aString: String)` to avoid allocating a temporary string per each call;

- Avoid using string concatenation (`s := s+'Blabla'+IntToStr(i)`) , but rely on a buffered writing such as `TStringBuilder` available in latest versions of Delphi;

- `TStringBuilder` is not perfect either: for instance, it will create a lot of temporary strings for appending some numerical data, and will use the awfully slow `SysUtils.IntToStr()` function when you add some `integer` value - I had to rewrite a lot of low-level functions to avoid most string allocation in our `TTextWriter` class as defined in SynCommons.pas;

- Don't abuse on critical sections, let them be as small as possible, but rely on some atomic modifiers if you need some concurrent access - see e.g. `InterlockedIncrement / InterlockedExchangeAdd`;

- `InterlockedExchange` (from SysUtils.pas) is a good way of updating a buffer or a shared object. You create an updated version of of some content in your thread, then you exchange a shared pointer to the data (e.g. a `TObject` instance) in one low-level CPU operation. It will notify the change to the other threads, with very good multi-thread scaling. You'll have to take care of the data integrity, but it works very well in practice.

- Don't share data between threads, but rather make your own private copy or rely on some read-only buffers (the RCU pattern is the better for scaling);

- Don't use indexed access to string characters, but rely on some optimized functions like `PosEx()` for instance;

- Don't mix `AnsiString/UnicodeString` kind of variables/functions, and check the generated asm code via Alt-F2 to track any hidden unwanted conversion (e.g. `call UStrFromPCharLen`);

- Rather use `var` parameters in a `procedure` instead of `function` returning a string (a function returning a `string` will add an `UStrAsg/LStrAsg` call which has a LOCK which will flush all CPU cores);

- If you can, for your data or text parsing, use pointers and some static stack-allocated buffers instead of temporary strings or dynamic arrays;

- Don't create a `TMemoryStream` each time you need one, but rely on a private instance in your class, already sized in enough memory, in which you will write data using `Position` to retrieve the end of data and not changing its `Size` (which will be the memory block allocated by the MM);

- Limit the number of class instances you create: try to reuse the same instance, and if you can, use some `record/object` pointers on already allocated memory buffers, mapping the data without copying it into temporary memory;

- Always use test-driven development, with dedicated multi-threaded test, trying to reach the worse-case limit (increase number of threads, data content, add some incoherent data, pause at random, try to stress network or disk access, benchmark with timing on real data...);

- Never trust your instinct, but use accurate timing on real data and process.

I tried to follow those rules in our Open Source framework, and if you take a look at our code, you'll find out a lot of real-world sample code.

Problem

Original source

Related problems

• Locks around memory manipulation via inline assembly