Return to README CVS log

Up to [Local Repository] / badi / public_scripts / parallelstarter

Annotation of badi/public_scripts/parallelstarter/README, revision 1.2

1.1       adi         1: parallelstarter quick how-to
                      2: ----------------------------
                      3: 
                      4: parallelstarter is a script to start a process multiple times in parallel. It controls
                      5: the number of processes that run simultaniousely and it can call custom scripts to set
                      6: up an environment for each process. There are three examples included, one for the matsim
                      7: benchmark test, one for biogeme and one for a database backup script called db_backup.
                      8: The processes you start get their command line arguments from a file. Each line in this
                      9: file contains the arguments for one process to start; thus the number of lines give the 
                     10: number of processes to start in total.
                     11: 
                     12: System requirements: GNU/Linux
                     13: 
                     14: 
                     15: Installation/ Setup:
                     16: - check out the latest parallelstarter from IVT CVS
                     17: - cd into the checked out directory
                     18: - run ./install_parallelstarter
                     19: - cd to your working direcotory
                     20: - run setup_parallelstarter in the working directory you want to collect the output of
                     21:   your called processes. The script creates some softlinks to the predefined processes
                     22:   matsim, biogme or db_backup, depending the argument you pass to it.
                     23: - edit parallelstarter.setup to fit your needs:
                     24:   In this file you control the number of simultaniousely running processes,
                     25:   the command to start one and the command line arguments that are identical for all processes.
                     26: - you are now ready to call parallelstarter in this directory
1.2     ! adi        27: - You may get db_backup from http://ente.limmat.ch/cvs/badi/public_scripts/db_backup/.
1.1       adi        28: 
                     29: For your own process setup, do the following:
                     30: - edit parallelstarter.prepare_process_start:
                     31:   This file is a script to copy or link the files or directories needed to run one process. Predefined matsim 
                     32:   for example expects a template directory, where the matsim Benchmark.jar lies (get ist from 
                     33:   http://matsim.org/benchmark). This is the working directory you would use normally to start the process 
                     34:   without parallelstarter. 
                     35:   This script should create a working directory in the directory you call parallelstarter for each process, change into
                     36:   it if the process writes to the current dir, it should set up links to libraries, copy or link config files etc. See
                     37:   the example for the matsim benchmark test by calling "setup_parallelstarter matsim".
                     38: - edit parallelstarter.cleanup:
                     39:   This file is a script for postprocessing, after all runs have finished.
                     40: - The example of biogeme generates its argument list file automatically, see the code for more details.
                     41:   
                     42: Using parallelstarter:
                     43: - Create a file with the differing command line arguments for your runs. biogeme and db_backup does this on its own, they 
                     44:   need a different config file from which they create the argument list file automatically. For matsim benchmark, you can
                     45:   test different JVM parameters for example by listing them in this file, each combination of switches on one line.
                     46:   It's name is configured in parallelstarter.setup.
                     47:   Each line in this file will trigger one run to start. The full command line for a process is put together as follows:
                     48: 
                     49:        $RUN_BIN $COMMON_LEADING_ARGS <one line from $ARGLIST_FILE> $COMMON_POST_ARGS
                     50:        ^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^                ^^^^^^^^^^^^^  ^^^^^^^^^^^^^^^^^
                     51:        (the underlined options are set up in parallelstarter.setup)
                     52: 
                     53: - start your runs: parallelstarter  [--quiet --quiet-run --rundir-prefix --runid-offset]
                     54:   where the options have the following effect:
                     55:        --quiet                 suppress all output but errors
                     56:        --quiet-run             suppress output from the started process
                     57:        --rundir-prefix         prefix each run's working directory with this string
                     58:        --runid-offset          start numbering each run's working directory with this number
                     59:        --help                  all switches shortly explained