Project

General

Profile

Design #649

Make SmallFpImpl safer to use

Added by John Abbott over 9 years ago. Updated over 8 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Safety
Start date:
11 Nov 2014
Due date:
% Done:

100%

Estimated time:
19.90 h
Spent time:

Description

After having wasted an hour or two tracking down a bug (in code which looked "obviously correct"), I want to make SmallFpImpl safer to use -- esp. harder to use incorrectly. Analogously for SmallFpLogImpl and SmallFpDoubleImpl.

Note: The problem was an "ambiguity" between myReduce and myNormalize.


Related issues

Related to CoCoALib - Bug #135: Revise interface to SmallFpImpl & friendsClosed2012-04-20

Related to CoCoALib - Feature #795: Add new fn InvModNoCheckClosed2015-11-05

History

#1 Updated by John Abbott over 9 years ago

The problem is that all the member functions accept any integral type, but most should accept only SmallFpImpl::value_t.

The function myReduce is designed to accept any integral type, while the "seemingly identical" function myNormalize expects only a SmallFpImpl::value_t. Of course, my buggy code called the wrong one... and the compiler issued no warning or error.

Ideally the interface to SmallFpImpl should be designed so that type-incorrect code does not compile.

#2 Updated by John Abbott over 9 years ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 20
Here are 2 candidate solutions:
  1. use a new type for the values (e.g. a class or struct containing the currently used integral type)
  2. define dummy mem fns which accept another integral type (e.g. bool), so that a type-incorrect call to the mem fn will produce an ambiguity (& a compilation error).

My current "instinctive" preference lies with (2), but the dyadic memfns need two dummies each: e.g. void myAdd(bool, value_t) and void myAdd(value_t, bool). Note that it is not enough to have just void myAdd(bool,bool) because then a call with one arg of type SmallFpImpl::value_t will have a unique best match for the standard myAdd memfn. What I don't like with this idea is that it is "a bit of trick" (perhaps the meaning of the dummy functions would not be obvious to someone reading the SmallFpImpl source code; but we can add documentation).

My current "objection" to using a new type is that I wanted (for better readability) to allow the integer constants 0 and 1 to be used as valid values in calls to the memfns (and also in assignments to variables of type SmallFpImpl::value_t). To be able to do this with a new type would imply having automatic conversion from some integral type to the representation type; and this would seem to defeat the extra safety deriving from having a new type.

#3 Updated by John Abbott over 9 years ago

It is not true that approach (1) allows the user to use 0 and 1 as arguments to the memfns. In fact, the only use in CoCoALib was to compute reciprocals; so I have added a myRecip memfn.

At this point I believe that approach (2) is more sensible, clearer and safer.

#4 Updated by John Abbott over 9 years ago

  • Assignee set to John Abbott

I have implemented both (1) and (2). A quick comparison suggests that they are roughly equal in speed (but I'm still investigating one anomaly).

#5 Updated by John Abbott over 9 years ago

  • % Done changed from 20 to 50

I've just wasted a good hour trying to figure out why my new code was about 10% slower than the old code.

The reason? Crummy old compiler on my main machine -- MacOSX 10.5.8 with Apple's version of gcc-4.2.

The same identical code running on a virtual machine (on the same MacBook Pro) showed that the new and old versions were about the same speed (new code about 1% slower); moreover, the code ran significantly faster in the virtual machine than natively!?! (gcc-4.3.2)

I then tried compiling with gcc-4.3.3 on my netbook, and execution speed was essentially identical for both approaches (but 3 times slower than on the MacBook).

What to do?
Grrr -- quite frustrated at having wasted so much time on this. Now I trust this MacBook Pro even less than ever!

NOTE I suppose now I know what to ask Santa for for Xmas...

#6 Updated by John Abbott about 9 years ago

  • Target version changed from CoCoALib-0.99536 June 2015 to CoCoALib-0.99540 Feb 2016

#7 Updated by John Abbott over 8 years ago

Why did I interfere with working code? Well, it's not working now; it's not even compiling! :-(

Well, SmallFpImpl compiles OK, but a main user (DUPFp) does not.

One significant problem is that I have frequently made comparisons of the form c = 0 and c = 1 where c is a SmallFpImpl::value. I have also used both 0 and 1 as values to assign to SmallFpImpl::value.

#8 Updated by John Abbott over 8 years ago

  • % Done changed from 50 to 70

After speaking to Anna, I have added IsZero and IsOne, and also the new calls zero(SmallFp) and one(SmallFp)... note that SmallFp here is actually an enum member.

I did not want to offer a public ctor for SmallFpImpl::value from a machine integer; while this would be very handy for 0 and 1, there is no way to limit it just to these values, but other values are a problem: either the caller must guarantee that they are already "reduced/normalized" or the values have to be reduced/normalized. I did not like the second option (since it will probably incur significant unexpected overhead), and the first seems too risky.

The only solution seems to be to have special functions for 0 and 1; hence the tests IsZero and IsOne (Anna assures me they are quite readable), and the pseudo-ctors zero(SmallFp) and one(SmallFp).

Note that all four functions are independent of the modulus since the current implementation guarantees that the internal reprs of 0 and 1 are indeed just the values 0 and 1. This is a bit of a "dirty trick"; the functions should really depend on the SmallFpImpl object. I hope I won't regret making the short-cut.

I have modified DUPFp and DistrMInlFpPP; the latter is still a bit dodgy (because I used some "dirty tricks" in the past).

#9 Updated by Anna Maria Bigatti over 8 years ago

I think this is a good choice: it is exactly what we do for ring, so the code is overall more coherent

#10 Updated by John Abbott over 8 years ago

I now disagree with my comment number 3. I think that the impl using new types (classes) to represent the values is clearer and safer.

There are two classes: one for reduced values and one for non-reduced values (e.g. for computing inner products more quickly). After an initial period learning the two classes and their meanings, the rest of the code should be easy to understand.

There is a question about names:
  • SmallFpImpl::value is the class for reduced values
  • SmallFpImpl::uvalue is the class for non-reduced values.

I am reasonably happy with value, but less convinced about uvalue.

Does anyone have any better suggestions for the names?

#11 Updated by John Abbott over 8 years ago

I like all types in CoCoALib to be printable; naturally this includes value and uvalue.

SmallFpImpl offers the choice of "export convention" between SymmResidues and NonNegResidues; these are heeded by the mem fn myExport. However, they cannot be heeded by direct printing since the modulus and chosen convention are in the SmallFpImpl object, which is not present in direct printing.

What to do?
  • (A) disallow direct printing; the user must call myExport explicitly?
  • (B) allow direct printing, but print in a manner indicating that the value printed is an internal representation?

Currently, a uvalue is printed between exclamation marks, but a value is printed out as a plain integer.

Right now my preference is to allow direct printing as that makes debugging simpler; if it were disallowed then the poor debugger fellow would probably have to look up in the doc why the value cannot be printed directly -- probably not a welcome hindrance when tracking down a bug!

Another, more verbose possibility would be to print value(n) and uvalue(n); perhaps that would be too verbose?

#12 Updated by John Abbott over 8 years ago

I am not entirely happy with the length of the names for the export conventions:
  • GlobalSettings::SymmResidues
  • GlobalSettings::NonNegResidues

What about removing the GlobalSettings prefix?
Is it desirable? Is it technically feasible? (pros and cons?)

#13 Updated by John Abbott over 8 years ago

When a SmallFpImpl is printed currently it appears as SmallFpImpl(p) where p is the prime. Should the export convention also appear in the print out?

#14 Updated by Anna Maria Bigatti over 8 years ago

John Abbott wrote:

There is a question about names:
  • SmallFpImpl::value is the class for reduced values
  • SmallFpImpl::uvalue is the class for non-reduced values.

I am reasonably happy with value, but less convinced about uvalue.

NonRedValue
CrudeValue
RoughValue
?
u makes me think of unsigned.

#15 Updated by John Abbott over 8 years ago

I quite like NonRedValue; it is quite clear, and a bit long (to discourage "casual use"). I'll try changing the code to see how I feel about it :-)

#16 Updated by John Abbott over 8 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 70 to 80

I have implemented NonRedValue.
Checked everything last night; revised examples, doc, etc.
Checked in today.

The only remaining point is why exgcd (or InvMod for ulong) is about 10% slower when placed in NumTheory.C rather than SmallFpImpl.C. I cut-and-pasted the code, then changed the name to exgcd in SmallFpImpl.

A simple test program shows that exgcd is consistently faster than InvMod. Why?
I have tried 3 different versions of g++...

#17 Updated by John Abbott over 8 years ago

I have resolved the mystery of the two identical functions running at different speeds: my test program was giving (very slightly) different inputs... sigh!

#18 Updated by John Abbott over 8 years ago

In NumTheory.H there is a fn called InvMod which expects two MachineInt; internally it calls the fn InvMod_private that actually does the work (on two unsigned long). The function SmallFpImpl::myRecip needs to call InvMod but I already know that the inputs are good: they are unsigned long and satisfy all the "safety" criteria.

If I call InvMod, the test program takes about 7.1s, but if I call InvMod_private it takes about 6.4s -- approx 10% faster. Anyway, the "correct" implementation should call InvMod_private. But how to do this?

I want InvMod_private to be implemented just once, but also to be available to (at least) 2 different .C files... which implies that it should be declared in some .H file. But I'm not sure how publicly visible InvMod_private should be: it is less safe than most public CoCoALib fns.

Of course, if I find it useful to have access to the "unsafe" function (because it's faster), perhaps some other implementer would also want access...?

So maybe the fn should be public but with a special name?

#19 Updated by John Abbott over 8 years ago

I am thinking of the name InvModNoCheck; it is pretty clear; it is a bit long, so that ought to discourage "casual users".

What do you think?

#20 Updated by Anna Maria Bigatti over 8 years ago

John Abbott wrote:

I am thinking of the name InvModNoCheck; it is pretty clear; it is a bit long, so that ought to discourage "casual users".

What do you think?

yes! go for it!
I thought we had used NoCheck somewhere else, but cannot find it. I only found RootBoundNoChecks in REalRoots.cpkg5)

#21 Updated by John Abbott over 8 years ago

  • Status changed from Resolved to Feedback
  • % Done changed from 80 to 90
  • Estimated time changed from 2.00 h to 19.90 h

[oops cut-and-paste did the wrong thing]
Changing to feedback; implemented the fn mentioned in comments 18-20. See #795

#22 Updated by John Abbott over 8 years ago

  • Status changed from Feedback to Closed
  • % Done changed from 90 to 100

I have just checked the documentation, and it seems to be complete. So I'm marking this issue as closed.

Also available in: Atom PDF