Skip to content

Running IRENA FlexTool from the terminal

IRENA FlexTool can be run directly from the command line without Spine Toolbox or a web browser. This is useful for scripting, automation, running on servers, or when you prefer a terminal-based workflow.

Prerequisites

  • Python 3.11+ with a virtual environment (e.g. ~/venv-spi/)
  • HiGHS solver binary available in the bin/ directory (included in the repository)
  • Dependencies installed: pip install -r requirements.txt
  • See the install page for full setup instructions

Activate the virtual environment before running any commands:

source ~/venv-spi/bin/activate

Quick start

Run the example scenario base from the bundled example database:

python execute_flextool_workflow.py \
    templates/examples.sqlite \
    output_info.sqlite \
    base \
    --skip-input-prep

This skips the input preparation phase (the database already exists) and runs the model followed by output generation. Results are written to output_plots/base/ and output_parquet/base/.

Scripts overview

Script Purpose
execute_flextool_workflow.py Unified entry point — runs the full workflow
run_flextool.py Runs the optimization model and writes outputs
write_outputs.py (Re-)generates plots, parquet, CSV, or Excel from results
scenario_results.py Compares results across multiple scenarios

execute_flextool_workflow.py — full workflow

This is the recommended entry point. It orchestrates three phases:

  1. Input preparation — convert tabular data (Excel/ODS/CSV) into a Spine database
  2. Model execution — run the FlexTool optimization model
  3. Output generation — process results into plots, parquet files, CSV, or Excel

Each phase can be skipped independently.

Usage

python execute_flextool_workflow.py INPUT_DB_URL OUTPUT_DB_URL SCENARIO_NAME [options]

Positional arguments

Argument Description
INPUT_DB_URL Input database URL or file path (e.g. sqlite:///input.sqlite or input.sqlite)
OUTPUT_DB_URL Output database URL for storing result metadata
SCENARIO_NAME Name of the scenario to execute

Options

Flag Description
--tabular-file-path PATH Path to Excel/ODS input file (mutually exclusive with --csv-directory-path)
--csv-directory-path PATH Path to directory containing CSV input files
--output-methods METHOD [...] Output formats: plot, parquet, excel, csv (default: plot parquet csv)
--output-subdir DIR Subdirectory for output files (default: scenario name)
--output-config PATH Path to output configuration YAML (default: templates/default_plots.yaml)
--skip-input-prep Skip input preparation (assumes database already exists)
--skip-model-run Skip model execution (assumes model has already been run)
--skip-output-write Skip output generation
--debug Enable debug output

Input sources

From an Excel or ODS file:

python execute_flextool_workflow.py \
    input.sqlite output_info.sqlite my_scenario \
    --tabular-file-path my_input.xlsx

From a directory of CSV files:

python execute_flextool_workflow.py \
    input.sqlite output_info.sqlite my_scenario \
    --csv-directory-path input_data/

From an existing Spine database (skip input prep):

python execute_flextool_workflow.py \
    sqlite:///input.sqlite output_info.sqlite my_scenario \
    --skip-input-prep

run_flextool.py — model execution

Runs the FlexTool optimization model directly. This is called internally by execute_flextool_workflow.py but can also be used standalone. After solving, it writes outputs and records scenario information in the output database.

Usage

python run_flextool.py INPUT_DB_URL OUTPUT_DB_URL [options]

Positional arguments

Argument Description
INPUT_DB_URL Input database URL
OUTPUT_DB_URL Output database URL for result metadata

Options

Flag Description
--scenario-name NAME Scenario name to execute (if omitted, uses database filter)
--write-methods METHOD [...] Output formats: plot, parquet, excel, csv (default: plot parquet)
--output-config PATH Path to output configuration YAML (default: templates/default_plots.yaml)
--debug Enable debug output

Example

python run_flextool.py \
    sqlite:///templates/examples.sqlite \
    sqlite:///output_info.sqlite \
    --scenario-name base

Return codes: 0 = success, 1 = infeasible or unbounded, -1 = failure.

write_outputs.py — output generation

Generates plots, parquet files, CSV, or Excel outputs from existing model results. Useful for re-plotting with different settings without re-running the model.

Usage

python write_outputs.py [options]

All arguments are optional. When run from the terminal, you typically provide --scenario-name.

Key options

Flag Description
--scenario-name NAME Scenario with raw outputs available
--read-parquet-dir DIR Read from existing parquet files instead of raw CSVs (reading from parquet is faster as long as they have been output)
--config-path PATH Output configuration YAML (default: templates/default_plots.yaml)
--active-configs NAME Which plot configuration set to use (default: default)
--write-methods METHOD [...] Output formats: plot, parquet, excel, db, csv (default: plot parquet csv)
--plot-rows START END First and last row to plot in time series (default: 0 167)
--subdir DIR Subdirectory for outputs (default: scenario name)
--output-location DIR Root directory for input/output locations (default: flextool root)
--debug Enable debug output

Examples

Re-plot from raw CSV outputs:

python write_outputs.py --scenario-name base

Re-plot from existing parquet files:

python write_outputs.py \
    --scenario-name base \
    --read-parquet-dir output_parquet/base

Generate only Excel output:

python write_outputs.py \
    --scenario-name base \
    --write-methods excel

Plot a different time range (rows 0 to 500):

python write_outputs.py \
    --scenario-name base \
    --plot-rows 0 500

scenario_results.py — cross-scenario comparison

Reads results from multiple scenarios and generates comparison plots and optional spreadsheets. The scenarios and their output locations are read from the output database (populated by run_flextool.py).

Usage

python scenario_results.py DB_URL [options]

Positional arguments

Argument Description
DB_URL Database URL containing scenario information (e.g. sqlite:///output_info.sqlite)

Options

Flag Description
--dispatch-plots Generate dispatch area plots for nodes and node groups
--summary-plots Generate summary bar chart plots
--all-plots Generate all plot types (dispatch and summary)
--show-plots Display plots interactively in addition to saving
--write-to-xlsx Write combined results to Excel file
--write-dispatch-xlsx Write dispatch data to Excel in the plot directory
--write-to-ods Write combined results to ODS file
--alternatives S [...] Specify alternative names manually
--parquet-subdir DIR Subdirectory with parquet files (default: output_parquet)
--output-config-path PATH Comparison plot configuration YAML (default: templates/default_comparison_plots.yaml)
--active_configs NAME Which plot configuration set to use (default: default)
--plot-rows START END First and last row to plot in time series (default: 0 167)
--plot-dir DIR Directory for comparison plots (default: output_plot_comparisons)

Examples

Generate all comparison plots:

python scenario_results.py sqlite:///output_info.sqlite --all-plots

Generate only summary bar charts and write to Excel:

python scenario_results.py sqlite:///output_info.sqlite \
    --summary-plots --write-to-xlsx

Generate dispatch plots for specific alternatives:

python scenario_results.py sqlite:///output_info.sqlite \
    --dispatch-plots --alternatives base high_RE

Worked examples

These examples use the bundled templates/examples.sqlite database and the base scenario.

1. Run a full workflow from an existing database

# Run model and generate outputs (plots, parquet, CSV)
python execute_flextool_workflow.py \
    templates/examples.sqlite \
    output_info.sqlite \
    base \
    --skip-input-prep

# Output plots are saved to output_plots/base/
# Parquet files are saved to output_parquet/base/
# CSV files are saved to output_csv/base/

2. Re-plot outputs from parquet files

After running the model at least once, you can regenerate plots from the saved parquet files without re-running the model:

python write_outputs.py \
    --scenario-name base \
    --read-parquet-dir output_parquet/base \
    --write-methods plot

3. Compare multiple scenarios

First, run two scenarios to populate the output database:

# Run first scenario
python execute_flextool_workflow.py \
    templates/examples.sqlite output_info.sqlite base \
    --skip-input-prep

# Run second scenario
python execute_flextool_workflow.py \
    templates/examples.sqlite output_info.sqlite network_all_tech \
    --skip-input-prep

Then compare them:

python scenario_results.py sqlite:///output_info.sqlite --all-plots

Comparison plots are saved to output_plot_comparisons/.

Output directory structure

After running the workflow, outputs are organized as follows:

flextool/
  output_plots/<scenario>/     # PNG plot files
  output_parquet/<scenario>/   # Parquet data files
  output_csv/<scenario>/       # CSV data files
  output_plot_comparisons/     # Cross-scenario comparison plots
  solve_data/                  # Solver progress and timing

Configuration

Output plots are controlled by YAML configuration files:

  • templates/default_plots.yaml — single-scenario plot configuration
  • templates/default_comparison_plots.yaml — cross-scenario comparison plot configuration

These files define which results to plot, plot types, layout, and styling. You can create custom configuration files and pass them via --output-config or --output-config-path.