BehaviorSpace Guide

This guide has three parts:

• What is BehaviorSpace?: A general description of the tool, including the ideas and principles behind it.
• How It Works: Walks you through how to use the tool and highlights its most commonly used features.
• Advanced Usage: How to use BehaviorSpace from the command line, or from your own Java code.

A number of new features were introduced in NetLogo 6.4:

• Subexperiment syntax: A syntax for allowing parameter combinations to be run separately, rather than being expanded combinatorically.
• Run metrics when: A reporter can be used to conditionally record measurements.
• Pre experiment commands: Commands can be run before the experiment begins.
• Post experiment commands: Commands can be run after the experiment ends.
• Statistics output: The mean and standard deviation of data from repetitions can be saved in an output file.
• Lists output: List data can be output in a file with one list element per cell.
• Importing and exporting: Experiments can now be exported to an XML file that can be used when running headlessly. Experiments can also be imported into a model.
• Paused experiments: Experiments can now be paused and resumed.
• Run options: update plots and monitors: Reduction in memory usage when box is unchecked.

More about changes to BehaviorSpace in NetLogo 6.4:

• Experiments created in versions prior to 6.4 can still be opened, but experiments created using new features are not backwards compatible.
• For information on format changes to output files see Output File Changes.
• The Experiment dialog now uses tooltips rather than text below each input element see Creating an experiment setup.
• Experiments using the new features can be found in the model Sample Models=>Biology=>Wolf Sheep Predation, with with additional notes in the Info tab.
• Hovering over the edit dialog entry titles will show a helpful ‘tooltip’.
• Additional minor changes can be found by searching this page for (Since 6.4).

What is BehaviorSpace?

BehaviorSpace is a software tool integrated with NetLogo that allows you to perform experiments with models.

BehaviorSpace runs a model many times, systematically varying the model’s settings and recording the results of each model run. This process is sometimes called “parameter sweeping”. It lets you explore the model’s “space” of possible behaviors and determine which combinations of settings cause the behaviors of interest.

If your computer has multiple processor cores, you can specify how many model runs will happen in parallel. Using multiple runs in parallel will increase the memory used by BehaviorSpace.

Why BehaviorSpace?

The need for this type of experiment is revealed by the following observations. Models often have many settings, each of which can take a range of values. Together they form what in mathematics is called a parameter space for the model, whose dimensions are the number of settings, and in which every point is a particular combination of values. Running a model with different settings (and sometimes even the same ones) can lead to drastically different behavior in the system being modeled. So, how are you to know which particular configuration of values, or types of configurations, will yield the kind of behavior you are interested in? This amounts to the question of where in its huge, multi-dimension parameter space does your model perform best?

For example, suppose you want speedy synchronization from the agents in the Fireflies model. The model has four sliders – number, cycle-length, flash-length and flashes-to-reset – that have approximately 2000, 100, 10 and 3 possible values, respectively. That means there are 2000 * 100 * 10 * 3 = 6,000,000 possible combinations of slider values! Trying combinations one at a time is hardly an efficient way to learn which one will evoke the speediest synchronization.

BehaviorSpace offers you a much better way to solve this problem by sampling the model’s parameter space – not exhaustively, but enough so that you will be able to see relationships form between different slider values and the behavior of the system. One way to do this is to specify a subset of values from the ranges of each slider. See Combinatorial syntaxes. BehaviorSpace will run the model with each possible combination of those values and, during each model run, record the results. Since NetLogo 6.4 it has been possible to specify non-combinatorial sets of slider values. See Subexperiment syntax. After all the runs are over, a dataset is generated which you can open in a different tool, such as a spreadsheet, database, or scientific visualization application, and explore.

By enabling you to explore the entire “space” of behaviors a model can exhibit, BehaviorSpace can be a powerful assistant to the modeler.

How It Works

To begin using BehaviorSpace, open your model, then choose the BehaviorSpace item on NetLogo’s Tools menu.

Managing experiment setups

The dialog that opens lets you create, edit, duplicate, delete, import, export, and run experiment setups. Experiments are listed by name and total number of model runs.

Experiment setups are considered part of a NetLogo model and are saved as part of the model, but can be also be exported as individual files. See Importing and exporting (Since 6.4).

To create a new experiment setup, press the “New” button. To edit an existing experiment setup, press the “Edit” button. The same dialog is used in both cases, what differs is the information that is already filled in. The dialog is non-blocking, which is useful if you want to copy something from the Code Tab, or view sliders in the Interface Tab (Since 6.4).

Creating an experiment setup

The information that can be included in the Experiment dialog is detailed below. Note that it is not necessary to specify everything; some parts can be left blank or with their default values. Hovering over the edit dialog entry titles will show a helpful ‘tooltip’, so you don’t have to revisit this page as often (Since 6.4).

Experiment name: Experiments in the same model must have different names. If you open a model that contains experiments with duplicate names, the conflicting names will be altered to ensure that all experiment names remain unique (Since 6.4).

Vary variables as follows: This is where you specify which settings you want varied, and what values you want them to take. Variables can include sliders, switches, choosers, and any global variables in your model. You may notice that the view and plots do not correspond to the values in the widgets once the experiment is completed. This is because the globals that are controlled by widgets are reset to their initial state at the end of the experiment. To synchronize your view and plots with the widgets, run your procedure that initializes the view.

Variables can also include max-pxcor, min-pxcor, max-pycor and min-pycor, world-width, world-height and random-seed. These are not variables, strictly speaking, but BehaviorSpace lets you vary them as if they were. Varying the world dimensions lets you explore the effect of world size upon your model. Setting world-width and world-height is valid under two circumstances. 1) If the origin is centered in width or height BehaviorSpace will keep it centered in that dimension. This requires that the corresponding value(s) world-width and/or world-height must be odd. 2) If one of the bounds is zero it will remain zero and the other bound will move. For example, if you start with a world with min-pxcor = 0, max-pxcor = 10 and you vary world-width like this:

["world-width" [11 1 14]]

min-pxcor will remain zero and max-pxcor will be set to 11, 12, and 13 for successive runs. If neither of these conditions is true you cannot you cannot vary world-height or world-width directly but must vary max-pxcor, max-pycor, min-pxcor and min-pycor instead.

Varying random-seed lets you repeat runs by using a known seed for the NetLogo random number generator. Note that you’re also free to use the random-seed command in your experiment’s setup commands. For more information on random seeds, see the Random Numbers section of the Programming Guide.

Combinatorial syntaxes

You may specify values either by listing the values you want used, or by specifying that you want to try every value within a given range. For example, to give a slider named number every value from 100 to 1000 in increments of 50, you would enter:

["number" [100 50 1000]]

Or, to give it only the values of 100, 200, 400, and 800, you would enter:

["number" 100 200 400 800]

Be careful with the brackets here. Note that there are fewer square brackets in the second example. Including or not including this extra set of brackets is how you tell BehaviorSpace whether you are listing individual values, or specifying a range.

Also note that the double quotes around the variable names are required.

All combinations of the specified values will be run. For example, if you have two values for a variable a and three values of a variable b six runs will result.

["a" 1 2]

["b" 2 4 10]

This would create six runs, organized as follows:

Run combinations in sequential order checkbox This box is checked by default, and causes variables specified later to vary more quickly than those specified earlier. When the box is unchecked, non-sequential order results, with variables specified earlier varying more quickly than those specified later. The non-sequential order for the variable specification above is:

Subexperiment syntax

(Since 6.4)

For more advanced users, there is a third available syntax for varying parameters, the subexperiment syntax. For example, in the Wolf Sheep Predation model, if you wanted to try two values for the grass-regrowth-time variable, each with its own values for the initial-number-sheep and initial-number-wolves variables, you could write it as follows:

[["grass-regrowth-time" 30]["initial-number-sheep" 116]["initial-number-wolves" 107]]

[["grass-regrowth-time" 40]["initial-number-sheep" 100]["initial-number-wolves" 77]]

Note the use of doubly nested square brackets to separate each variable within a subexperiment. This would create two runs, organized as follows:

The subexperiment syntax also allows you to define constants using the standard syntax, which will be applied to each subexperiment where they are not overwritten. To add to the example above, if you wanted to try those combinations of initial-number-sheep and initial-number-wolves but with the same value for grass-regrowth-time, you could write it as follows:

["grass-regrowth-time" 30]

[["initial-number-sheep" 116]["initial-number-wolves" 107]]

[["initial-number-sheep" 100]["initial-number-wolves" 77]]

This would set grass-regrowth-time to 30 for both subexperiments, resulting in the following runs:

You can also override a constant in a subexperiment, as in the following example:

["grass-regrowth-time" 30]

[["initial-number-sheep" 116]["initial-number-wolves" 107]]

[["grass-regrowth-time" 40]["initial-number-sheep" 100]["initial-number-wolves" 77]]

[["initial-number-sheep" 80]["initial-number-wolves" 153]]

This would produce three runs with the following combinations:

Note that all constants must be defined before any subexperiments.

You can vary as many settings as you want, including just one, or none at all. Any settings that you do not vary will retain their current values. Not varying any settings is useful if you just want to do many runs with the current settings.

The order in which you list the variables determines the run order. All values for a later variable will be tried before moving to the next value for an earlier variable. So for example if you vary both x and y from 1 to 3, and x is listed first, then the order of model runs will be: x=1 y=1, x=1 y=2, x=1 y=3, x=2 y=1, and so on.

Repetitions: Sometimes the behavior of a model can vary a lot from run to run even if the settings don’t change, if the model uses random numbers. If you want to run the model more than once at each combination of settings, enter a higher number. With sequential ordering repetitions occur in sequential runs:

With non-sequential ordering repetitions occur as a second group of runs:

Measure runs using these reporters: This is where you specify what data you want to collect from each run. For example, if you wanted to record how the population of turtles rose and fell during each run, you would enter:

count turtles

You can enter one reporter, or several, or none at all. If you enter several, each reporter must be on a line by itself, for example:

count frogs
count mice
count birds

If you don’t enter any reporters, the runs will still take place. This is useful if you want to record the results yourself your own way, such as with the export-world command. You can use reporters you have defined in the Code tab. Reporters appear as column headers. If you prefer compact headers you could replace count patches with [ pcolor = red ] with a reporter red-patches defined in the Code tab.

Run metrics every step: Normally NetLogo will measure model runs at every step, using the reporters you entered in the previous box. If you’re doing very long model runs, you might not want all that data. Uncheck this box if you want to either only measure model runs at the end of the run or if you want to specify certain conditions when measurements should be taken.

Run metrics when

(Since 6.4)

This reporter will be used to determine when measurements should be recorded if they are not being recorded at every step. Measurements will be always be taken at the end of each model run, even if this text box is empty as was previously the case when the Measure runs at every step (now Run metrics every step) was unchecked. For example ticks mod 10 = 0 would record every tenth tick, as well as the last tick. Multiple reporters can be combined using and and or.

Pre experiment commands

These commands will be run once, before the experiment begins.

Setup commands: These commands will be used to begin each model run. Typically, you will enter the name of a procedure that sets up the model, typically setup. But it is also possible to include other commands as well. If you want the same results each time you run an experiment, you could use something like random-seed 473 setup or to have different results for repetitions random-seed (474 + behaviorspace-run-number) setup

Go commands: These commands will be run over and over again to advance to the model to the next “step”. Typically, this will be the name of a procedure, such as go, but you may include any commands you like.

Stop condition: This lets you do model runs of varying length, ending each run when a certain condition becomes true. For example, suppose you wanted each run to last until there were no more turtles. Then you would enter:

not any? turtles

If you want the length of runs to all be of a fixed length, just leave this blank.

The run may also stop because the go commands use the stop command, in the same way that stop can be used to stop a forever button. The stop command may be used directly in the go commands, or in a procedure called directly by the go commands. (The intent is that the same go procedure should work both in a button and in a BehaviorSpace experiment.) Note that the step in which stop is used is considered to have been aborted, so no results will be recorded for that step. Therefore, the stopping test should be at the beginning of the go commands or procedure, not at the end.

Post run commands: These are any extra commands that you want run when each run ends. Usually this is left blank, but you might use it to call the export-world command or record the results of the run in some other way.

Post experiment commands

These are any commands that you want to run at the end of the experiment, after all runs have completed.

Time limit: This lets you set a fixed maximum length for each run. If you don’t want to set any maximum, but want the length of the runs to be controlled by the stop condition instead, enter 0.

Note on pre and post experiment commands: These commands execute outside of the actual runs, so they cannot affect the global variables or agents in the model (use the Setup commands and Post run commands for that). If you use primitives like file-open or the CSV extension in the pre or post experiment commands, their internal state will not carry over into the runs and errors will likely occur if used there.

Importing and exporting

(Since 6.4)

Although experiments are tied to a model and are usually saved along with a model, they can also be imported and exported individually to xml files. This allows you to easily transfer experiments between models, and also prepares experiments to be run headlessly. After an experiment is exported to an xml file, it can be edited by hand or by another script, not just within NetLogo.

The Import button allows you to import experiments from an xml file. The selected files may contain any number of experiments, but any experiments that are formatted incorrectly will not be loaded. If you load an experiment that has the same name as an existing experiment, the name of the loaded experiment will be slightly altered to ensure that experiment names remain unique.

The Export button allows you to export experiments to an xml file. Any number of experiments may be selected for export at once, but they will all be combined into a single output file.

Special primitives for BehaviorSpace experiments

Currently there are only two, behaviorspace-run-number and behaviorspace-experiment-name. The run number reported by the former primitive matches the run number used in the results files generated by BehaviorSpace. The experiment name reported by the latter matches the name with which the experiment was set up.

Running an experiment

When you’re done setting up your experiment, press the “OK” button, followed by the “Run” button. A dialog titled “Run Options” will appear.

Run options: formats

The “Run Options” dialog lets you choose to create data output files in two primary formats, Table output and Spreadsheet output. If one or both of these formats is selected, you can also select the supplementary Lists output and Statistics output. Each file path can be entered in its corresponding text box, or using the *Browse… button to select a file path using the system file dialog. If you don’t wish to use a particular format, you can clear the file text or click the Disable button and it’ll be cleared for you. The frequency of data collection is determined by the settings of the Run metrics every step and Run metrics when options. For Table output and Spreadsheet output formats, the initial state of the system is recorded, after the setup commands run but before the go commands run for the first time.

After selecting your output formats, BehaviorSpace will prompt you for the name of a file to save the results to. The default name ends in “.csv”. You can change it to any name you want, but don’t leave off the “.csv” part; that indicates the file is a Comma Separated Values (CSV) file. This is a plain-text data format that is readable by any text editor as well as by most popular spreadsheet and database programs.

All four output formats will include a header section that has rows for 1) the NetLogo app version used, 2) the name of the NetLogo model file used, 3) the name of the BehaviorSpace experiment used, 4) the date and time at the start, and 5) the dimensions of the world used at the start.

Table output

This format lists each measurement step from each run in its own row, with each metric in a separate column. The measurement rows will appear in the order they happen in real time. With the parallel runs option the measurements may appear in a mixed order as multiple runs can happen simultaneously. To help identify which run a row belongs to, there is a column titled [run number] along with a column titled [step] that indicates the measurement number for that run. Each row also includes the values used for the variables during initial setup of the run; these values are the same for each of that run’s measurement rows. Table output data is written to the file as each run completes.

The Table output format is good to use when you want to further process the data using another tool, such as importing into a database, a statistics package, or a spreadsheet application for analysis.

In the Table output sample image below, the header section is in red, the run number and step columns are in blue, the initial values of the variables for the run are in green, and the measurement metric data is in purple.

Spreadsheet output

This format lists the step numbers as well as each metric for each run in a separate column, with each row corresponding to a measurement step that applies to all runs. If one run finishes before another due to a stop condition, then its step numbers after that point will be blank. At the top of the file there is a [run number] row that will have the run number repeated for each metric at the top of the file. After that are rows for the initial values given to each variable for the run. The spreadsheet output also calculates the min, mean, max, and final values for the step numbers as well as each metric and lists those after the initial values before the individual run measurements. Then it lists the actual number of steps a run went through in a [total steps] row.

The Spreadsheet output data makes it easier to quickly compare runs against each other, as they will be aligned vertically when imported into a spreadsheet application. It can also make generating comparative graphs of results with initial variable changes across runs easy for the same reason.

It is important to note that Spreadsheet output data is not written to the results file until the experiment finishes. Since the data is stored in memory until the experiment is done, very large experiments could run out of memory. You should disable Spreadsheet output unless you really want it. Also, if anything interrupts the experiment no spreadsheet results will be written. Possible sources of interruptions would be runtime errors in the model, running out of memory, system crashes, or power outages. For long experiments you may want to also enable Table output format as a precaution so that if something happens and you get no Spreadsheet output you’ll at least get partial data output.

In the Spreadsheet output sample image below, the header section is in red, the run number row is in blue, the initial values of the variables for the run are in green, the extra calculated metrics are in orange, and the measurement metric data is in purple.

Statistics output

(Since 6.4)

You can create a file with the mean and standard deviation of each numeric metric by enabling the Statistics output. Metrics are the reporters specified in the “Measure runs using these reporters as metrics:” box. You must also use the Table output and/or the Spreadsheet output because one of them will used to calculate the mean and standard deviation of each numeric metric across repetitions for each step. These calculations are done at the end of the experiment. The statistics are then saved in the specified file. It is important to note that metrics that produce non-numeric values such as strings are not included in the statistics. If the metrics produce lists, the statistics are calculated across elements with the same index for each list. Statistics are not calculated for metrics that produce lists that contain non-numeric elements. Also, the type of the measurement must remain constant. For example, statistics are not calculated if a metric produces measurements that can be both lists and numbers. Furthermore, the standard deviation may be “N/A”, which means that there were two or fewer collected measurements for that step and parameter combination, producing a result that is not well-defined. The experiment must use at least 3 repetitions to produce standard deviation values.

In the Statistics output sample image below (creating using the “Red Queen” library model), the header section is in red, the parameter combinations are in green, the steps are in blue, and the statistics are in purple. Starting with column F, the statistics are the mean of “count frogs”, the standard deviation of “count frogs”, followed by the same statistics for “count snakes”, average-poison, and average-resistance. Column E shows the step, and columns A-D show the parameter combination that was used. The statistics are calculated across all the repetitions.

Lists output

(Since 6.4)

This format is a supplement to the other two primary formats, as opposed to a complete data collection format. If you have any reporters that return a list, you can use the Lists output format to get properly formatted output for those reporters. Data is produced for all list reporters in the “metrics” box. In both Spreadsheet output and Table output formats, lists returned by reporters will be condensed into a single cell, rather than their elements being spread out with one value per cell. If you need the list values in individual cells, select the List output format in addition to one or both of the other formats.

Note that the Lists output format, like the Spreadsheet output format, will not contain any data until the experiment is complete. Also note that the Lists output must be used in tandem with another format, it cannot be used on its own.

In the Lists output sample image below, the header section is in red, the reporter name is in orange, the run number and step columns are in blue, the initial values of the variables for the run are in green, and the expanded list data is in purple. The list data starts in row 7, with the headers (the index into the list, starting from 0) starting in column E.

Output File Changes

(Since 6.4)

• Output files have version numbers.
• Spreadsheet output always includes step information.
• Spreadsheet output column header [initial & final values] was changed to the more accurate [final value].
• Statistics output was added.
• Lists output was added.

Run options: update plots and monitors

The “Run Options” dialog lets you choose whether to update plots and monitors or not. Performance is better when the box is unchecked. Note that (Since 6.4) if you begin the experiment with the box unchecked, you will not be able to toggle between enabling and disabling the update plots checkbox in the “Running Experiments” dialog. This gives you an even greater performance improvement than was obtained before NetLogo 6.4, when such toggling was possible. Check the box if you you need to export plot data using primitives such as export-interface, export-plot, export-all-plots, and export-world.

Run options: parallel runs

The “Run Options” dialog also lets you select whether you want multiple model runs to happen in parallel, and if so, how many are allowed to be simultaneously active. The default and recommended maximum number of parallel runs are shown below the text box (Since 6.4). Your choice of number of parallel runs is remembered from experiment to experiment (Since 6.4). Because each additional run increases memory usage we recommend starting with the default number of runs, and then trying more parallel runs if you want. The recommended maximum number of parallel runs is just an estimate, and some users have found that performance is significantly decreased when using the maximum.

There are a few cautions associated with parallel runs.

First, if multiple runs are active, only one of them will be in the “foreground” and cause the view and plots to update. The other runs will happen invisibly in the background.

Second, invisible background runs can’t use primitives that only work in the GUI. For example, a background run can’t make a movie.

Third, since parallel runs progress independently of each other, table format output may contain interleaved, out-of-order results. When you analyze your table data, you may wish to sort it by run number first. (Spreadsheet format output is not affected by this issue, since it is not written until the experiment completes or is aborted.)

Fourth, using all available processor cores may make your computer slow to use for other tasks while the experiment is running or slow to complete runs as contention will build for memory between the runs themselves. If your model uses a large amount of memory, you may find that reducing the number of runs will enable the runs to complete in less time overall since work will be done by the system keeping the memory for each run available. A good rule of thumb might be to start with the default value shown in the “Run Options”, and bump up or down from there to see where your “sweet spot” is for least time to complete all runs.

Fifth, doing runs in parallel will multiply the experiment’s memory requirements accordingly. You may need to increase NetLogo’s memory ceiling (see this FAQ entry). By default NetLogo will not exceed 50% of your system’s memory.

Sixth, each parallel run will get its own world for the model to run in. This world is not cleared automatically by BehaviorSpace if a parallel run gets re-used for another repetition, which happens quite frequently. This means, for example, if you do ask patches [ set pcolor red ] in one run and do not use clear-all or clear-patches in the setup commands of the next run, then the patches will all still be red. In general using clear-all before each run would be a best practice, but there are times when you might not want to, such as loading data from a file that doesn’t change run-to-run. Just be careful with whatever data is not cleared out.

Seventh, there is a very, very small chance that at startup multiple parallel runs could wind up with the same random number generator state if they startup at the exact same moment in time. This means the runs would produce identical output for all random operations and likely the same results. This would have a chance to happen when running on very fast processors and with lots of parallel runs at once. If you need to make sure this doesn’t impact your results, you can add random-seed new-seed to your setup commands to re-generate a new unique random seed for each run. In fact, storing the new-seed as a global variable so you can output it with the rest of your results would let you re-run a run later on by manually using that value to set the random-seed.

Observing runs

After you complete the “Run Options” dialog, another dialog will appear, titled “Running Experiment”. In this dialog, you’ll see a progress report of how many runs have been completed so far and how much time has passed. If you entered any reporters for measuring the runs, and if you left the “Run metrics every step” box checked, then you’ll see a plot of how they vary over the course of each run.

You can also watch the runs in the main NetLogo window. (If the “Running Experiment” dialog is in the way, just move it to a different place on the screen.) If you don’t need to see the plots update, then use the checkboxes in the “Running Experiment” dialog to turn the updating off. This will make the experiment go faster. However, if you already disabled updating plots and monitors in the “Run Options” dialog, this checkbox will be disabled (Since 6.4).

If you want to stop your experiment before it’s finished, you have two options. To stop the experiment after the current runs have completed and save your progress for later, press the “Pause” button (Since 6.4). To stop the experiment immediately without waiting for the current runs to complete, press the “Abort” button. Any output generated so far will still be saved, but pressing “Abort” can lead to fragmented data, so aborted experiments cannot be resumed.

When all the runs have finished, the experiment is complete. Spreadsheet, Lists and Stats output are created at this point.

Paused experiments

(Since 6.4)

Paused experiments will appear in the BehaviorSpace window marked with “In Progress”. To resume an experiment where you paused it, select it and press the “Run” button. To reset a paused experiment to its initial state, select it and press the “Abort” button.

If you are using Spreadsheet output a file containing the data up until the experiment is paused will be written. This data will be used as part of the creation of a complete Spreadsheet file. Note that if your experiment is writing to its own external file you may need to make some changes in order for pausing to work correctly. For example you should use file-flush or file-close at the end of each run to ensure all the data is written to the file, and should do file-open before doing any writing during a run. When opening a file in writing mode, all new data will be appended to the end of the original file, which is probably the behavior you want.

Note that moving or deleting output files before resuming a paused experiment will cause an error. Outputting new experiment data to a file associated with an existing paused experiment may also ause an error when that experiment is resumed.

Advanced Usage

Running from the command line

It is possible to run BehaviorSpace experiments “headless”, that is, from the command line, without any graphical user interface (GUI). This is useful for automating runs on a single machine or a cluster of machines.

No Java programming is required. Experiment setups can be created in the GUI and then run later from the command line, or, if you prefer, you can create or edit experiment setups directly using XML.

How to use it

Run NetLogo using the NetLogo_Console app with the --headless command line argument. This is found in the root directory of your NetLogo installation. The NetLogo_Console script supports the following arguments:

• --headless: Enable headless mode to run a BehaviorSpace experiment (required, will open the graphical interface otherwise).
• --model <path>: pathname of model to open (required)
• --setup-file <path>: read experiment setups from this file instead of the model file
• --experiment <name>: name of experiment to run
• --table <path>: pathname to send table output to (or - for standard output)
• --spreadsheet <path>: pathname to send table output to (or - for standard output)
• --lists <path>: pathname to send lists output to (or - for standard output), cannot be used without --table or --spreadsheet
• –stats : pathname to send statistics output to (or - for standard output) cannot be used without --table or --spreadsheet
• --threads <number>: use this many threads to do model runs in parallel, or 1 to disable parallel runs. defaults to floor(0.75 * <number of processors>).
• --update-plots: enable plot updates. Include this if you want to export plot data, or exclude it for better performance.
• --min-pxcor <number>: override world size setting in model file
• --max-pxcor <number>: override world size setting in model file
• --min-pycor <number>: override world size setting in model file
• --max-pycor <number>: override world size setting in model file

--model is required. If you don’t specify --experiment, you must specify --setup-file. By default no results are generated, so you’ll usually want to specify either --table or --spreadsheet, or both. If you specify any of the world dimensions, you must specify all four.

Note that prior to NetLogo 6.3.0 the directions were to use netlogo-headless.sh (or netlogo-headless.bat on Windows) along with a separate installation of Java of the system to run BehaviorSpace experiments. The netlogo-headless.sh script is still included with NetLogo and can still be used as before, which might be preferrable in server environments where the installed Java version is strictly controlled. But the recommended method for on a personal computer is to use the NetLogo_Console --headless app. Because NetLogo_Console uses the Java that comes bundled with NetLogo it requires no extra software installation or configuration.

Examples

It is easiest if you create your experiment setup ahead of time in the GUI, so it is saved as part of the model. To run an experiment setup saved in a model, here is an example command line, run from the NetLogo 7.0.0-beta2 installation folder so the paths to the NetLogo_Console app and Wolf Sheep Simple 5.nlogo model are correct.

The below commands should work as-is in a terminal on macOS and Linux. On Windows in the Command Prompt you can use ^ instead of \ to break the command across multiple lines, or just put the command on a single line.

./NetLogo_Console --headless \
  --model "models/IABM Textbook/chapter 4/Wolf Sheep Simple 5.nlogo" \
  --experiment "Wolf Sheep Simple model analysis" \
  --table -

After the named experiment has run, the results are sent to standard output in table format, as CSV. - is how you specify standard output instead of output to a file.

When running NetLogo headless, it forces the system property java.awt.headless to be true. This tells Java to run in headless mode, allowing NetLogo to run on machines when a graphical display is not available.

The required --model argument is used to specify the model file you want to open.

The --experiment argument is used to specify the name of the experiment you want to run. (At the time you create an experiment setup in the GUI, you assign it a name.)

Here’s another example that shows some additional, optional arguments:

./NetLogo_Console --headless \
  --model "models/IABM Textbook/chapter 4/Wolf Sheep Simple 5.nlogo" \
  --experiment "Wolf Sheep Simple model analysis" \
  --max-pxcor 5 \
  --min-pxcor -5 \
  --max-pycor 5 \
  --min-pycor -5

Note the use of the optional --max-pxcor, --max-pycor, etc. arguments to specify a different world size than that saved in the model. (It’s also possible for the experiment setup to specify values for the world dimensions; if they are specified by the experiment setup, then there is no need to specify them on the command line.)

Since neither --table nor --spreadsheet is specified, no results will be generated. This is useful if the experiment setup generates all the output you need by some other means, such as exporting world files or writing to a text file.

Yet another example:

./NetLogo_Console --headless \
  --model "models/IABM Textbook/chapter 4/Wolf Sheep Simple 5.nlogo" \
  --experiment "Wolf Sheep Simple model analysis" \
  --table wsp5-table-output.csv \
  --spreadsheet wsp5-spreadsheet-output.csv \
  --lists wsp5-lists-output.csv \
  --stats wsp5-stats-output.csv

The optional --table <filename> argument specifies that output should be generated in a table format and written to the given file as CSV data. If - is specified as the filename, than the output is sent to the standard system output stream. Table data is written as it is generated, with each complete run.

The optional --spreadsheet <filename> argument specifies that spreadsheet output should be generated and written to the given file as CSV data. If - is specified as the filename, than the output is sent to the standard system output stream. Spreadsheet data is not written out until all runs in the experiment are finished.

The optional --lists <filename> argument specifies that lists output should be generated and written to the given file as CSV data. If - is specified as the filename, than the output is sent to the standard system output stream. Lists data is not written out until all runs in the experiment are finished.

The optional --stats <filename> argument specifies that stats output should be generated and written to the given file as CSV data. If - is specified as the filename, than the output is sent to the standard system output stream. Stats data is not written out until all runs in the experiment are finished.

Note that it is legal to specify both --table and --spreadsheet, and if you do, both kinds of output file will be generated. If you use --lists or --stats at least one of the --table or --spreadsheet options must be used.

Here is an example that shows how to run an experiment setup which is stored in a separate XML file, instead of in the model file (see below for more information on the XML file format). This assumes you’ve created a my-wsp-setups.xml file with a `My WSP Exploration" experiment and placed it in your home directory. The most straight-forward way to create a setup file is to create an experiment using BehaviorSpace in the NetLogo GUI and use the **Export ** option.

./NetLogo_Console --headless \
  --model "models/IABM Textbook/chapter 4/Wolf Sheep Simple 5.nlogo" \
  --setup-file ~/my-wsp-setups.xml \
  --experiment "My WSP Exploration"

If the XML file contains more than one experiment setup, it is necessary to use the --experiment argument to specify the name of the setup to use.

In order to run a NetLogo 3D experiment, run headless with the --3D argument, for example:

./NetLogo_Console --headless \
  --3D \
  --model "models/3D/Sample Models/GasLab/GasLab Free Gas 3D.nlogo3d" \
  --experiment "100 runs" \
  --table -

Note that you should supply a 3D model and there are also 3D arguments --max-pzcor <number> and --min-pzcor <number>.

The next section has information on how to create standalone experiment setup files using XML.

Setting up experiments in XML

We don’t yet have detailed documentation on authoring experiment setups in XML, but if you already have some familiarity with XML, then the following pointers may be enough to get you started.

The structure of BehaviorSpace experiment setups in XML is determined by a Document Type Definition (DTD) file. The DTD is stored in NetLogo.jar, as system/behaviorspace.dtd. (JAR files are also zip files, so you can extract the DTD from the JAR using Java’s “jar” utility or with any program that understands zip format.)

The easiest way to learn what setups look like in XML, though, is to author a few of them in BehaviorSpace’s GUI, save the model, export them (Since 6.4) and then examine the resulting .xml file(s) in a text editor. The experiment setups can also be found towards the end of the .nlogo file, in a section that begins and ends with a experiments tag. Example:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE experiments SYSTEM "behaviorspace.dtd">
<experiments>
  <experiment name="My WSP Exploration" repetitions="5" runMetricsEveryStep="false">
    <setup>setup</setup>
    <go>go</go>
    <timeLimit steps="2000"/>
    <metric>count wolves</metric>
    <metric>count sheep</metric>
    <metric>sum [grass-amount] of patches</metric>
    <enumeratedValueSet variable="energy-gain-from-grass">
      <value value="2"/>
    </enumeratedValueSet>
    <steppedValueSet variable="number-of-wolves" first="5" step="1" last="15"/>
    <enumeratedValueSet variable="movement-cost">
      <value value="0.5"/>
    </enumeratedValueSet>
    <enumeratedValueSet variable="energy-gain-from-sheep">
      <value value="5"/>
    </enumeratedValueSet>
    <enumeratedValueSet variable="number-of-sheep">
      <value value="500"/>
    </enumeratedValueSet>
    <enumeratedValueSet variable="grass-regrowth-rate">
      <value value="0.3"/>
    </enumeratedValueSet>
  </experiment>
</experiments>

In this example, only one experiment setup is given, but you can put as many as you want between the beginning and ending experiments tags.

Between looking at the DTD, and looking at examples you create in the GUI, it will hopefully be apparent how to use the tags to specify different kind of experiments. The DTD specifies which tags are required and which are optional, which may be repeated and which may not, and so forth.

If you want to create a setup file for NetLogo 6.3.0 and earlier versions for which Export is not available you need to know that in a model file the XML for experiment setups does not begin with any XML headers, because the not whole file is XML, only part of it. Therefore if you manually create a separate file for experiment setups, the extension on the file should be .xml not .nlogo, and you’ll need to begin the file with proper XML headers, as follows:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE experiments SYSTEM "behaviorspace.dtd">

The second line must be included exactly as shown. In the first line, you may specify a different encoding than UTF-8, such as ISO-8859-1.

Adjusting JVM Parameters

Opening the NetLogo Headless launcher script will show the options used to launch java when running NetLogo Headless. You can adjust various JVM parameters in this script. You may also pass in Java properties starting with -D to the launcher.

NetLogo allocates a maximum of half your total system memory for running your model as it is needed. If you want to set a maximum amount of memory for BehaviorSpace to use you can use the -Xmx setting to specify a particular heap size.

Note the use of -Dfile.encoding=UTF-8. This forces all file I/O to use UTF-8 encoding. Doing so ensures that NetLogo can load all models consistently, and that file-* primitives work consistently on all platforms, including models containing Unicode characters.

Controlling API

If BehaviorSpace is not sufficient for your needs, a possible alternative is to use our Controlling API, which lets you write Java code that controls NetLogo. The API lets you run BehaviorSpace experiments from Java code, or, you can write custom code that controls NetLogo more directly to do BehaviorSpace-like things. See the Controlling section of the User Manual for further details on both possibilities.