CoCoA » CoCoALib

Triggered by the example from issue #1514.
The SEGV seems to arise during destruction at the end of the program!

Current situation: (running on my linux box with 32Gbytes RAM)

• get SEGV reading 4500 polys (each about 500 terms) in ZZ/(2)[x[1..1000]], so will be using RingDistrMPolyInlPPFp. SmallExponent_t was unsigned int (took about 112s)
• No SEGV with SmallExponents_t being unsigned char (took about 58s);
• SEGV with SmallExponents_t being unsigned short (took about 75s)
• No SEGV with SmallExponents_t being unsigned short and NO debugging

I think I have found the cause... it is in "invisible code" written by the compiler. The invisible code is correct, but can trigger deep recursion.

I think there are (at least) two aspects to the problem:

1. a loaf is an element of a linked list with an explicit (raw) pointer to the next loaf; destruction is a recursive process, so if the linked list becomes long then the recursion can become too deep --> SEGV
2. the linked list will become long if there are many "active" allocated slots, and the max loaf size is small (currently 64k bytes)
3. the MemPool and loaf mechanism works well only if the individual slots are not too large; for larger slots it is probably better just to less the system allocator handle the requests.

In the example which triggered the problem, the linked list probably contained over 100000 entries... hardly surprising that the stack overflowed!

I have made a first design improvement:
There is now MaxSliceSize (in bytes); if the requested slice size is greater than this limit then all alloc/free calls are forwarded to the system mem mgr (namely, ::operator new and ::operator delete).

NOTE: both MemPoolFast and MemPoolDebug have been updated.

Currently the limit is set to 128 bytes -- no idea if this is really a good value to use.

This does not solve the original problem, but should make it less likely to occur.

I have just run the test from issue #1337.
With MemPool the program took 473s; disabling MemPool by setting the max slice size very low (to 12), the test took 489s.

I suppose I'm slightly disappointed that MemPool does not make a bigger difference, but probably memory allocation was only a small proportion of the total computation time.

Further test:

/**/ use R ::= ZZ/(101)[x[1..30]];
/**/ L := [RandomLinearForm(R) | i in 1..2000000];  TimeFrom(0);
46.460
/**/ t0 := CpuTime(); L := []; TimeFrom(t0);
1.713

If I reduce the number of indets to 24 (which should allow MemPool to operate) then the times become 33s and 0.76s.
Also memory consumption was considerably greater with 30 indets: 8.4Gbytes against 5.3Gbytes with just 24 indets.

This makes me inclined to increase max slice size to 256bytes.

Still, the system allocator is not bad at all (and is presumably even thread-safe).

I'm not convinced the test in comment 12 is really a good test.
Anyway, i have reproduced results which suggest that 256 is a reasonable limit.

Propose closing this issue (even though it is not truly resolved... but I don't want to mess about inside MemPools).