Return to README CVS log

Up to [Local Repository] / badi / public_scripts / parallelstarter

Annotation of badi/public_scripts/parallelstarter/README, revision 1.1

1.1     ! adi         1: parallelstarter quick how-to
        !             2: ----------------------------
        !             3: 
        !             4: parallelstarter is a script to start a process multiple times in parallel. It controls
        !             5: the number of processes that run simultaniousely and it can call custom scripts to set
        !             6: up an environment for each process. There are three examples included, one for the matsim
        !             7: benchmark test, one for biogeme and one for a database backup script called db_backup.
        !             8: The processes you start get their command line arguments from a file. Each line in this
        !             9: file contains the arguments for one process to start; thus the number of lines give the 
        !            10: number of processes to start in total.
        !            11: 
        !            12: System requirements: GNU/Linux
        !            13: 
        !            14: 
        !            15: Installation/ Setup:
        !            16: - check out the latest parallelstarter from IVT CVS
        !            17: - cd into the checked out directory
        !            18: - run ./install_parallelstarter
        !            19: - cd to your working direcotory
        !            20: - run setup_parallelstarter in the working directory you want to collect the output of
        !            21:   your called processes. The script creates some softlinks to the predefined processes
        !            22:   matsim, biogme or db_backup, depending the argument you pass to it.
        !            23: - edit parallelstarter.setup to fit your needs:
        !            24:   In this file you control the number of simultaniousely running processes,
        !            25:   the command to start one and the command line arguments that are identical for all processes.
        !            26: - you are now ready to call parallelstarter in this directory
        !            27: 
        !            28: For your own process setup, do the following:
        !            29: - edit parallelstarter.prepare_process_start:
        !            30:   This file is a script to copy or link the files or directories needed to run one process. Predefined matsim 
        !            31:   for example expects a template directory, where the matsim Benchmark.jar lies (get ist from 
        !            32:   http://matsim.org/benchmark). This is the working directory you would use normally to start the process 
        !            33:   without parallelstarter. 
        !            34:   This script should create a working directory in the directory you call parallelstarter for each process, change into
        !            35:   it if the process writes to the current dir, it should set up links to libraries, copy or link config files etc. See
        !            36:   the example for the matsim benchmark test by calling "setup_parallelstarter matsim".
        !            37: - edit parallelstarter.cleanup:
        !            38:   This file is a script for postprocessing, after all runs have finished.
        !            39: - The example of biogeme generates its argument list file automatically, see the code for more details.
        !            40:   
        !            41: Using parallelstarter:
        !            42: - Create a file with the differing command line arguments for your runs. biogeme and db_backup does this on its own, they 
        !            43:   need a different config file from which they create the argument list file automatically. For matsim benchmark, you can
        !            44:   test different JVM parameters for example by listing them in this file, each combination of switches on one line.
        !            45:   It's name is configured in parallelstarter.setup.
        !            46:   Each line in this file will trigger one run to start. The full command line for a process is put together as follows:
        !            47: 
        !            48:        $RUN_BIN $COMMON_LEADING_ARGS <one line from $ARGLIST_FILE> $COMMON_POST_ARGS
        !            49:        ^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^                ^^^^^^^^^^^^^  ^^^^^^^^^^^^^^^^^
        !            50:        (the underlined options are set up in parallelstarter.setup)
        !            51: 
        !            52: - start your runs: parallelstarter  [--quiet --quiet-run --rundir-prefix --runid-offset]
        !            53:   where the options have the following effect:
        !            54:        --quiet                 suppress all output but errors
        !            55:        --quiet-run             suppress output from the started process
        !            56:        --rundir-prefix         prefix each run's working directory with this string
        !            57:        --runid-offset          start numbering each run's working directory with this number
        !            58:        --help                  all switches shortly explained