Return to README CVS log

Up to [Local Repository] / badi / public_scripts / parallelstarter

Hint to db_backup download.

    1: parallelstarter quick how-to
    2: ----------------------------
    3: 
    4: parallelstarter is a script to start a process multiple times in parallel. It controls
    5: the number of processes that run simultaniousely and it can call custom scripts to set
    6: up an environment for each process. There are three examples included, one for the matsim
    7: benchmark test, one for biogeme and one for a database backup script called db_backup.
    8: The processes you start get their command line arguments from a file. Each line in this
    9: file contains the arguments for one process to start; thus the number of lines give the 
   10: number of processes to start in total.
   11: 
   12: System requirements: GNU/Linux
   13: 
   14: 
   15: Installation/ Setup:
   16: - check out the latest parallelstarter from IVT CVS
   17: - cd into the checked out directory
   18: - run ./install_parallelstarter
   19: - cd to your working direcotory
   20: - run setup_parallelstarter in the working directory you want to collect the output of
   21:   your called processes. The script creates some softlinks to the predefined processes
   22:   matsim, biogme or db_backup, depending the argument you pass to it.
   23: - edit parallelstarter.setup to fit your needs:
   24:   In this file you control the number of simultaniousely running processes,
   25:   the command to start one and the command line arguments that are identical for all processes.
   26: - you are now ready to call parallelstarter in this directory
   27: - You may get db_backup from http://ente.limmat.ch/cvs/badi/public_scripts/db_backup/.
   28: 
   29: For your own process setup, do the following:
   30: - edit parallelstarter.prepare_process_start:
   31:   This file is a script to copy or link the files or directories needed to run one process. Predefined matsim 
   32:   for example expects a template directory, where the matsim Benchmark.jar lies (get ist from 
   33:   http://matsim.org/benchmark). This is the working directory you would use normally to start the process 
   34:   without parallelstarter. 
   35:   This script should create a working directory in the directory you call parallelstarter for each process, change into
   36:   it if the process writes to the current dir, it should set up links to libraries, copy or link config files etc. See
   37:   the example for the matsim benchmark test by calling "setup_parallelstarter matsim".
   38: - edit parallelstarter.cleanup:
   39:   This file is a script for postprocessing, after all runs have finished.
   40: - The example of biogeme generates its argument list file automatically, see the code for more details.
   41:   
   42: Using parallelstarter:
   43: - Create a file with the differing command line arguments for your runs. biogeme and db_backup does this on its own, they 
   44:   need a different config file from which they create the argument list file automatically. For matsim benchmark, you can
   45:   test different JVM parameters for example by listing them in this file, each combination of switches on one line.
   46:   It's name is configured in parallelstarter.setup.
   47:   Each line in this file will trigger one run to start. The full command line for a process is put together as follows:
   48: 
   49: 	$RUN_BIN $COMMON_LEADING_ARGS <one line from $ARGLIST_FILE> $COMMON_POST_ARGS
   50: 	^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^                ^^^^^^^^^^^^^  ^^^^^^^^^^^^^^^^^
   51: 	(the underlined options are set up in parallelstarter.setup)
   52: 
   53: - start your runs: parallelstarter  [--quiet --quiet-run --rundir-prefix --runid-offset]
   54:   where the options have the following effect:
   55: 	--quiet			suppress all output but errors
   56: 	--quiet-run		suppress output from the started process
   57: 	--rundir-prefix		prefix each run's working directory with this string
   58: 	--runid-offset		start numbering each run's working directory with this number
   59: 	--help			all switches shortly explained