CoCoA » CoCoALib

Strictly this is not CoCoALib but CoCoA-5.
I ran the CoCoA-5 tests with about 0.5GiB memory: there was 1 failure, namely test-radical.cocoa5

Also tried with a limit of about 100Mbytes; again only test-radical failed (but it produced a SEGV?!?)

The tests test-RadicalMembership1 and test-SparsePolyRing2 are the only ones where memory usage considerably worsened with version 0.99850:

~10 MiB to ~200 MiB (test-RadicalMembership1),
~6 MiB to ~3.2 GiB (test-SparsePolyRing2).

The other tests' memory needs are under 10 MiB, with only two exceptions: test-DynamicBitset1 (~130 MiB) and test-RingFq1 (~30 MiB).

The test test-SparsePolyRing2 didn't change. The 500-fold increased memory consumption is a regression in version 0.99850.

CoCoALib | Test                      | Wall clock | Max. res. set size (kbytes)
-------------------------------------------------------------------------------
0.99818    test-RadicalMembership1-a   0:00.68         8728
0.99824    test-RadicalMembership1-b   0:01.08        10972
0.99850    test-RadicalMembership1-b   0:01.26       207820

0.99818    test-SparsePolyRing1-a      0:00.00         6108
0.99824    test-SparsePolyRing1-a      0:00.00         6224
0.99850    test-SparsePolyRing1-b      0:00.24         6496

0.99818    test-SparsePolyRing2        0:01.24         6284
0.99824    test-SparsePolyRing2        0:01.19         6288
0.99850    test-SparsePolyRing2        0:04.73      3438404

Appended -a/-b to discriminate between tests that have changed between versions. Those changes in the tests might explain some minor differences.

Changed test do NOT explain the big differences in memory usage coming with 0.99850 in the first and third test, as those tests have not changed between 0.99824 (a prerelease version dated March 18) and 0.99850 (the release, March 29).

PS: Data obtained with /usr/bin/time -v.

CoCoALib | Test                   | Wall clock | Max. res. set size (kbytes)
----------------------------------------------------------------------------
0.99818    test-SparsePolyRing1-b   0:00.10      6160
0.99824    test-SparsePolyRing1-b   0:00.10      6156
0.99850    test-SparsePolyRing1-a   0:00.01      6184

Thus for test-SparsePolyRing1 it seems to be a combination: The new test takes more time than the old test (also with older CoCoALib versions) and the new CoCoALib version is slower than the older versions for this new test (and also for the older test).

To be clear:
test-SparsePolyRing1-a: test-SparsePolyRing1.C from CoCoALib-0.99818
test-SparsePolyRing1-b: test-SparsePolyRing1.C from CoCoALib-0.99850

I realize that this issue was intended to be about something else. Feel free to move my posts as you see fit -- sorry for the inconvenience. I think it is more urgent to fix this severe memory usage and performance regression than to introduce checks to prevent similar issues in the future.

It is not clear to me why test-SparsePolyRing1-b.C should have become slower by a factor of 2.
I'm not surprised by the change in resources needed for test-RadicalMembership1.C since the test has changed considerably (and a bug in the impl was fixed, making the code slower but correct).
As I wrote before, I think I know what the problem with test-SparsePolyRing2.C is; maybe I have an idea how to fix it... It is in the new code Anna introduced for computing radicals (and following a suggestion of mine).

My idea did not produce much gain (neither regarding time nor memory requirement).
There's a loop somewhere which is creating lots of rings, but I don't know where/why.
Taking a break.

John Abbott wrote:

I'm not surprised by the change in resources needed for test-RadicalMembership1.C since the test has changed considerably (and a bug in the impl was fixed, making the code slower but correct).

test-RadicalMembership1.C did not change from 0.99824 to 0.99850. Do you think the 20-fold increase in memory usage between these versions is due to the fixed bug?

• there was a bug in IsInRadical which meant that it always returned true IIRC
• I discussed with Anna about speculatively computing radicals of generators when computing radicals of ideals -- this is a good strategy provided that the radical of a polynomial can be computed quickly, but this may not be the case if the characteristic is less than the degree (because the GCD is computed via a syzygy computation)

So, I'm not entirely sure that these change could cause an increase in time and memory requirement... I'm a bit tired this eevening to undertake an investigation -- I did try with valgrind and kcachegrind but that was less helpful than hoped. I hope to discuss with Anna soon, and resolve the matter within a week.

John Abbott wrote:

• there was a bug in IsInRadical which meant that it always returned true IIRC

test-RadicalMembership1-b.C (from 0.99824/50) fails when compiled against CoCoALib-0.99718 -- due to a wrong result returned by IsInRadical.

Since the same test succeeds against 0.99824, it seems that that bug is already fixed in 0.99824 (the source code also seems to support this).

Since 0.99824 does not suffer the memory/speed issue, I'd suspect that this issue is NOT due to the bugfix for IsInRadical.

Obviously I'm guessing here which bug you are referring to -- it might be a different one than what the new test checks.

Ulrich von der Ohe wrote:

test-RadicalMembership1.C did not change from 0.99824 to 0.99850. Do you think the 20-fold increase in memory usage between these versions is due to the fixed bug?

I'll check. I changed the algorithm for the 0-dimensional case: it computes some powers and checks if that is in the ideal. That might be the reason of the increase in memory.
I'll check whether there is also a decent gain in time.

Anna plans to investigate today