Overview

automan is an open source, Python-based automation framework for numerical computing.

It is designed to automate the drudge work of managing many numerical simulations. As an automation framework it does the following:

  • helps you organize your simulations.

  • helps you orchestrate running simulations and then post-processing the results from these.

  • helps you reuse code for the post processing of your simulation data.

  • execute all your simulations and post-processing with one command.

  • optionally distribute your simulations among other computers on your network.

This greatly facilitates reproducibility. Automan is written in pure Python and is easy to install.

This document should help you use automan to improve your productivity. If you are interested in a more detailed article about automan see the automan paper a draft of which is available here: https://arxiv.org/abs/1712.04786

Installation

The easiest way to install automan is with pip:

$ pip install automan

If you wish to run the latest version that has not been relesed you may clone the git repository:

$ git clone https://github.com/pypr/automan

$ cd automan

And then run:

$ python setup.py develop

If you just want to run the latest version and do not have git you can do this:

$ pip install https://github.com/pypr/automan/zipball/master

Once this is done, move on to the next section that provides a gentle tutorial introduction to using automan.

Citing automan

If you find automan useful and wish to cite it you may use the following article:

  • Prabhu Ramachandran, “automan: A Python-Based Automation Framework for Numerical Computing,” in Computing in Science & Engineering, vol. 20, no. 5, pp. 81-97, 2018. doi:10.1109/MCSE.2018.05329818

You can find a draft of the article here: https://arxiv.org/abs/1712.04786

Changelog

0.6

  • Release date: still in development.

0.5

  • Release date: 4th November, 2021

  • Provide some handy functions to generate many simulations from parameters.

  • Add ability to add arbitrary tasks/problems to the automator using the add_task method.

  • Allow specification of negative n_core and n_thread for the job information. This is documented in the section on additional computational resources.

  • Improve ability to customize the styles used with compare_runs.

  • Add a --rm-remote-output argument to the command line arguments.

  • Add a convenient FileCommandTask.

  • Improve ability to customize command line arguments.

  • Fix issue with too many processes and open files.

  • Fix an issue with command tasks executing on a remote host and waiting.

  • Use github actions for tests.

  • 12 PRs were merged.

0.4

  • Release date: 26th November, 2018.

  • Support for inter Problem/Simulation/Task dependencies.

  • Print more useful messages when running tasks.

  • Fix bug with computing the available cores.

  • Improve handling of cases when tasks fail with errors.

  • Fix a few subtle bugs in the task runner.

  • Minor bug fixes.

0.3

  • Release date: 5th September, 2018.

  • Complete online documentation and examples.

  • Much improved and generalized cluster management with support for conda and edm in addition to virtualenvs.

  • Support multiple projects that use different bootstrap scripts.

  • Better testing for the cluster management.

  • Do not rewrite the path to python executables and run them as requested by the user.

  • Removed any lingering references or use of pysph.

  • Change the default root to automan instead of pysph_auto.

  • Support filtering cases with a callable.

  • Fix bug where a simulation with an error would always be re-run.

  • Fix bug caused due to missing --nfs option to automator CLI.

0.2

  • Release date: 28th August, 2017.

  • First public release of complete working package with features described in the paper.