Project

General

Profile

Bug #593

Temporary directories used during configuration

Added by John Abbott over 9 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
High
Assignee:
Category:
Tidying
Target version:
Start date:
20 Jul 2014
Due date:
% Done:

100%

Estimated time:
11.11 h
Spent time:

Description

Improvements needed for the TMP_DIRs used during configuration:
  • perhaps use std prefix so that they can all be removed easily by make clean
  • decide where the TMP_DIRs should be made (I suggest configuration/)
  • perhaps make the scripts produce log files in the TMP_DIRs (which can be inspected in case of failure, but which are automatically removed when the TMP_DIR is removed)

Related issues

Related to CoCoALib - Feature #759: Configuration: where to keep source for test compilations?Closed2015-08-02

Related to CoCoALib - Slug #792: configure: search for libgmp too slowClosed2015-11-02

Related to CoCoALib - Bug #975: configure: possibly ignore GMP compilation flagsClosed2016-11-17

Related to CoCoALib - Support #976: configure: auxiliary scripts should be consistent about error messagesClosed2016-11-18

History

#1 Updated by Anna Maria Bigatti over 9 years ago

  • Target version set to CoCoALib-1.0

#2 Updated by John Abbott over 9 years ago

  • Category set to Tidying

I'll probably also need a script to tidy up temp dirs left by the current scripts... :-(

#3 Updated by John Abbott over 9 years ago

  • Status changed from New to In Progress
  • % Done changed from 0 to 10

I now think that a good solution would be to put tmpdirs in configuration/TMP/. That way they can all be "cleaned" simply by /bin/rm -rf TMP. I hope to implement this in the near future.

I am also tempted to say that tmpdirs are removed only upon successful exit; if we have an easy way of cleaning up tmpdirs then it is not so harmful to leave them "lying around" (which is handy for debugging).

#4 Updated by John Abbott over 9 years ago

  • Assignee set to John Abbott

Christof reported problems with my one of my shell scripts which uses the trap facility to guarantee proper cleaning up of temporary files/directories. The fault was largely mine -- I have very little experience in using trap. After reading some comments about using trap (on appropriate forums, e.g. see the trap example on page http://en.wikibooks.org/wiki/Bourne_Shell_Scripting/Debugging_and_signal_handling) I think it is probably simpler/better not to use trap for automatic cleaning up.

In other words leaving temporary directories/files intact when an error occurs probably makes the scripts a bit simpler (and helps with debugging).

#5 Updated by John Abbott over 9 years ago

  • Priority changed from Normal to High
  • Target version changed from CoCoALib-1.0 to CoCoALib-0.99536 June 2015

I have had further problems with trap in the script gmp-try-default.sh on some Linux boxes. This is further motivation for avoiding using the feature.

#6 Updated by John Abbott almost 9 years ago

  • Target version changed from CoCoALib-0.99536 June 2015 to CoCoALib-0.99540 Feb 2016

#7 Updated by John Abbott about 8 years ago

  • Target version changed from CoCoALib-0.99540 Feb 2016 to CoCoALib-1.0

#8 Updated by John Abbott almost 8 years ago

  • Related to Slug #792: configure: search for libgmp too slow added

#9 Updated by John Abbott almost 8 years ago

After waiting for several configure-make cycles to complete on the fixed machine here in Kassel, I have concluded that it would be much faster if the temporary directories used during configuration were somewhere below /tmp (the system-wide temporary space, which is automatically cleaned out during rebooting). The filesystem /tmp is local, and very much faster than anywhere in my home directory.

The main complication of using /tmp is avoiding clashes if several people try to configure CoCoALib simultaneously, but that ought to be easy to resolve just by using the shell PID somewhere in the directory name.

#10 Updated by John Abbott almost 8 years ago

Another advantage of using /tmp is that scripts no longer need to test whether they are being run from the CoCoA root directory (or else receive as an argument the path to a writable directory with a safe name (i.e. not containing awkward chars which have to be escaped)).

#11 Updated by John Abbott over 7 years ago

Using /tmp for temporary directories is very tempting, but must be done safely (e.g. cannot have two users trying to access the same directory at the same time).

What assumptions may I make about /tmp?
Following KISS, I shall assume that it is local to a single machine, so at any one time all running processes have different PIDs. To help with debugging the dir name should contain the user's name, and probably also some reference to "CoCoA".

It may even be helpful to put the date+time in the name (e.g. if someone is having trouble, and is making many attempts to configure). This may produce "funny" results if someone is configuring CoCoALib around midnight... :-/

How about /tmp/CoCoALib-config-abbott-20161117/gmp-cxx-flags-2135/?
It's long but should be fairly easy to comprehend.

#12 Updated by John Abbott over 7 years ago

Some scripts need to know where the ExternalLibs directory is located.
I think the simplest way to pass in this information is via an environment variable (this avoids having to pass it as an explicit argument).
What should the environment variable be called? I have two proposals:
  • COCOA_EXTLIB_DIR
  • COCOALIB_EXTLIB_DIR

I have a mild preference for the first (as it does not have a repeated LIB in the name, and it is shorter).

#13 Updated by John Abbott over 7 years ago

  • % Done changed from 10 to 30

I have changed the gmp-XYZ.sh scripts... configuration still works on my machine (I think).

#14 Updated by John Abbott over 7 years ago

  • % Done changed from 30 to 40

I have now adopted the following naming convention for the tmp dirs:
/tmp/CoCoALib-config-abbott-2016-11-18/gmp-cxx-flags-11:06:21-8443/

That is /tmp/CoCoALib-config-USER-DATE/SCRIPT-TIME-PID

The component USER will help a user identify directories related to himself.
The component DATE will help identify the current ones (in case old ones are still there).
The component SCRIPT says which script created the directory.
The component TIME again helps to identify the most recent directories.
The component PID avoids clashes should a single user attempt to configure several copies of CoCoALib at the same time.

#15 Updated by John Abbott over 7 years ago

  • % Done changed from 40 to 50

Also done the gfan scripts (and fixed some more tiny problems in the gmp scripts).

#16 Updated by John Abbott over 7 years ago

  • Status changed from In Progress to Feedback
  • % Done changed from 50 to 90

I have updated all the auxiliary scripts to use temp dirs in /tmp following the naming convention suggested above. There is one slight defect: the upper level directory (/tmp/CoCoALib-config-USER-DATE) is never removed, but that is not so important.

I have made several changes (improvements, of course!) to the scripts too.

There is still some inconsistency about whether error messages are printed out /dev/stdout or /dev/stderr; I'll add a new issue about this.

Gereon still has a problem with a "rogue" GMP header file: the compilation options in the file do not correspond to that platform. This causes problems in CoCoA: compilation succeeds (with g++) but then several executables, but not all, give "illegal instruction" errors. I'll add a new issue.

#17 Updated by John Abbott over 7 years ago

  • Related to Bug #975: configure: possibly ignore GMP compilation flags added

#18 Updated by John Abbott over 7 years ago

  • Related to Support #976: configure: auxiliary scripts should be consistent about error messages added

#19 Updated by Redmine Admin almost 7 years ago

  • Target version changed from CoCoALib-1.0 to CoCoALib-0.99560

#20 Updated by John Abbott over 6 years ago

  • Status changed from Feedback to Closed
  • % Done changed from 90 to 100
  • Estimated time changed from 5.00 h to 11.11 h

The scripts seems to work fine now.

Closing.

Also available in: Atom PDF