Return to README CVS log

Up to [Local Repository] / badi / public_scripts / parallelstarter

Hint to db_backup download.

parallelstarter quick how-to
----------------------------

parallelstarter is a script to start a process multiple times in parallel. It controls
the number of processes that run simultaniousely and it can call custom scripts to set
up an environment for each process. There are three examples included, one for the matsim
benchmark test, one for biogeme and one for a database backup script called db_backup.
The processes you start get their command line arguments from a file. Each line in this
file contains the arguments for one process to start; thus the number of lines give the 
number of processes to start in total.

System requirements: GNU/Linux


Installation/ Setup:
- check out the latest parallelstarter from IVT CVS
- cd into the checked out directory
- run ./install_parallelstarter
- cd to your working direcotory
- run setup_parallelstarter in the working directory you want to collect the output of
  your called processes. The script creates some softlinks to the predefined processes
  matsim, biogme or db_backup, depending the argument you pass to it.
- edit parallelstarter.setup to fit your needs:
  In this file you control the number of simultaniousely running processes,
  the command to start one and the command line arguments that are identical for all processes.
- you are now ready to call parallelstarter in this directory
- You may get db_backup from http://ente.limmat.ch/cvs/badi/public_scripts/db_backup/.

For your own process setup, do the following:
- edit parallelstarter.prepare_process_start:
  This file is a script to copy or link the files or directories needed to run one process. Predefined matsim 
  for example expects a template directory, where the matsim Benchmark.jar lies (get ist from 
  http://matsim.org/benchmark). This is the working directory you would use normally to start the process 
  without parallelstarter. 
  This script should create a working directory in the directory you call parallelstarter for each process, change into
  it if the process writes to the current dir, it should set up links to libraries, copy or link config files etc. See
  the example for the matsim benchmark test by calling "setup_parallelstarter matsim".
- edit parallelstarter.cleanup:
  This file is a script for postprocessing, after all runs have finished.
- The example of biogeme generates its argument list file automatically, see the code for more details.
  
Using parallelstarter:
- Create a file with the differing command line arguments for your runs. biogeme and db_backup does this on its own, they 
  need a different config file from which they create the argument list file automatically. For matsim benchmark, you can
  test different JVM parameters for example by listing them in this file, each combination of switches on one line.
  It's name is configured in parallelstarter.setup.
  Each line in this file will trigger one run to start. The full command line for a process is put together as follows:

	$RUN_BIN $COMMON_LEADING_ARGS <one line from $ARGLIST_FILE> $COMMON_POST_ARGS
	^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^                ^^^^^^^^^^^^^  ^^^^^^^^^^^^^^^^^
	(the underlined options are set up in parallelstarter.setup)

- start your runs: parallelstarter  [--quiet --quiet-run --rundir-prefix --runid-offset]
  where the options have the following effect:
	--quiet			suppress all output but errors
	--quiet-run		suppress output from the started process
	--rundir-prefix		prefix each run's working directory with this string
	--runid-offset		start numbering each run's working directory with this number
	--help			all switches shortly explained