Reading the repo
Opening the box…
Reading the repo
Opening the box…
How to install on macOS
Saved setup from 5 h ago
Will it work on macOS?
Then run bash setup.sh. It stops at the first error and is safe to run again. Servers and tests aren't started; it lists them at the end. Read it first.
Needs Git: xcode-select --install
git clone https://github.com/dauvillc/motif.git cd motif
Python 3.12.11
brew install uv
uv sync
The project requires Python 3.12 and uses uv for dependency management. To install the dependencies and the package in a virtual environment, run
uv sync
from the repository root. The package can then be used by activating the virtual environment with source .venv/bin/activate, or by prefixing commands with uv run.
The first step is to set the paths for your own environment by creating your own configuration in configs/paths/, using configs/paths/example.yaml as base. Raw dataset locations are defined there as derived paths from raw_datasets (tc_primed, sar_cyclobs, era5_weatherbench, etc.).
Raw data can be downloaded with Hydra, passing paths=<your_paths_config> (defaults to jz via configs/preproc.yaml):
python preproc/tc_primed/download_tc_primed.py paths=<your_paths_config>python preproc/sar/download_sar_cyclobs.py paths=<your_paths_config>python preproc/era5/dl_era5_64x32.py paths=<your_paths_config>Optional overrides include +year=2015, +basin=AL, and +workers=32 for TC-PRIMED (calendar year); sar_download.workers=4 and era5_download.workers=16 for the other downloads.
The dataset can be preprocessing using the following scripts:
python preproc/tc_primed/prepare_pmw_concat.py paths=<your_paths_config> +num_workers=<n_workers>python preproc/tc_primed/prepare_infrared.py paths=<your_paths_config> +num_workers=<n_workers>python preproc/train_val_test_split.py paths=<your_paths_config>python preproc/compute_normalization_constants.py paths=<your_paths_config> +num_workers=<n_workers>Similarly to the paths configuration, the first step is to create a configuration in configs/setup/ adapted to your computing environment, using configs/setup/exampe.yaml as base.
A training experiment can be run locally using
python scripts/train.py experiment=<experiment_cfg> model=motif_12b_d512 setup=<your_setup_cfg> dataloader.batch_size=2 wandb.name=<name_of_the_experiment> +run_local=true
On a SLURM cluster, the training can be directly submitted as a job using
python scripts/train.py experiment=<experiment_cfg> model=motif_12b_d512 setup=<your_setup_cfg> dataloader.batch_size=2 wandb.name=<name_of_the_experiment>
The following experiment configurations are available in configs/experiment/:
det_PI: Deterministic, self-supervised, microwave + infrared.fm_PI_gpm: Flow matching, supervised, GMI/GPM target on the PI setup.fm_pmw: Flow matching, self-supervised, microwave 37/89 GHz only.fm_PI: Flow matching, self-supervised, microwave + infrared.HPC command recipes: commands.md.
Every training run generates a run id that is printed out by the script. That run id is used as Weights and Biases id.
Predictions are generated using scripts/make_predictions.py. The script requires a run_id (the Weights and Biases id of the trained model), an inference configuration, and a data split:
python scripts/make_predictions.py run_id=<run_id> inference_cfg=<inference_cfg> split=<val|test> setup=<your_setup_cfg> +dataloader.batch_size=2
To run predictions for multiple models in a single command, provide a comma-separated list of run ids and add the --multirun flag:
python scripts/make_predictions.py run_id=<run_id_1>,<run_id_2> inference_cfg=<inference_cfg> split=<val|test> setup=<your_setup_cfg> +dataloader.batch_size=2 --multirun
The following inference configurations are available in configs/inference_cfg/:
fm_gpm_dt1: Flow matching model, GPM target, 1-hour time window.det_gpm_dt1: Deterministic model, GPM target, 1-hour time window.fm_gpm_dt6: Flow matching model, GPM target, 6-hour time window.fm_gpm_PI_dt6: Flow matching model, GPM target, 6-hour time window, requires at least one microwave and one infrared source.det_gpm_PI_dt6: Deterministic model, GPM target, 6-hour time window, requires at least one microwave and one infrared source.fm_gpm_PI_dt6_past: Same as fm_gpm_PI_dt6, restricting auxiliary sources to those observed before the target.det_gpm_PI_dt6_past: Same as det_gpm_PI_dt6, restricting auxiliary sources to those observed before the target.Evaluation is performed using scripts/eval.py. The models argument is a dictionary mapping display names to [run_id, inference_cfg] pairs, where inference_cfg must match the one used during prediction:
python scripts/eval.py models="{<model_name>: [<run_id>, <inference_cfg>]}" eval_class="[<eval_class_1>, <eval_class_2>]" +eval_name=<name> setup=<your_setup_cfg> num_workers=<n_workers> split=<val|test>
Multiple
…
No reports for macOS yet.
GitManual AI Agent reads the 3 key files and writes a step-by-step guide for macOS. Once written, it's saved for everyone.