Skip to content
Snippets Groups Projects
Mikhail Khalilov's avatar
Update README.md with experiment setup
Mikhail Khalilov authored
5d9385d3
Select Git revision
  • main default
  • pspin-master
  • dma_congestion
3 results
Compare History Find file
Name Last commit Last update
pspin @ 230583d8
.gitmodules
Dockerfile
README.md
docker-compose.yml
README.md

PsPIN/OSMOSIS

A. Verilator environment setup

1. Set up Docker container (15 minutes)

  1. host$ git clone git@spclgitlab.ethz.ch:mkhalilov/pspin-osmosis.git
  2. host$ cd ./pspin-osmosis/ && git submodule update --init --recursive
  3. host$ docker-compose -p osmosis-ae up -d

As a result container osmosis-ae will be built and run.

2. Build cycle-accurate PsPIN/OSMOSIS simulation core inside of container (30 minutes)

Last step needs around half of an hour and 128-256 GB of RAM.

  1. host $ docker ps # get the containerID
  2. host $ docker exec -it <containerID> bash
  3. osmosis-ae # cd /opt/pspin/
  4. osmosis-ae # source ./sourceme.sh
  5. osmosis-ae # cd ./hw/verilator_model/
  6. osmosis-ae # VERILATOR_COMPILER_WORKERS=$(nproc) make release

B. Experiments

0. Setup simulation environment (30 seconds)

  1. In case you stopped the container, re-run docker-compose as shown in Stage A1;
  2. (Re)-attach to the running osmosis-ae container by re-running commands 1 - 4 from the stage A2;
  3. osmosis-ae # cd /opt/pspin/examples/mt_apps/ - change to the root OSMOSIS experiments working directory;
  4. osmosis-ae # make SPIN_KERNEL_NAME=kernels deploy - compile SmartNIC application kernels.

1. Run experiment (2 min -- 6 hours)

The experiment pipeline contains the following steps:

  1. osmosis-ae # make SPIN_APP_NAME=<experiment_name> osmosis - compile experiment scenario from ./driver directory, i.e., experiment binary sim_<experiment_name> will be generated in the current directory;
  2. osmosis-ae # nohup bash ./scripts/run_<experiment_name>.sh $(pwd)/traces/ $(pwd)/logs/ & - run batched simulation to perform all measurements (we control sim_<experiment_name> binary config through environment variables) and input traffic trace. Raw simulation logs will be stored in ./logs/ directory. Pre-generated traces are stored in ./traces/ directory;
  3. osmosis-ae # python3 ./scripts/postprocess_<experiment_name>.py $(pwd)/logs - post-process experiment logs to data *.csv in ./logs/ directory;
  4. osmosis-ae # python3 ./scripts/plot_<experiment_name>.py - run plotting script on *.csv data to visualize it in *.pdf format in the ./figures/ directory.

To check out the figure from the host file-system (outside of container, e.g., in PDF reader) at $host <AE-repo-root-dir>/pspin/examples/mt_apps/figures/.

Experiment 1: HPU contention aka "Hello Fair World" (2 minutes)

The experiment demonstrates that OSMOSIS WLBVT scheduler achieves fair share of SmartNIC compute engines when compared to Round Robin.

We recommend to use this experiment to validate basic functionality of experimental pipeline (e.g., AE kick-the-tires stage).

  1. osmosis-ae # nohup bash ./scripts/run_hpu_contention.sh $(pwd)/traces/ $(pwd)/logs/ &
  2. osmosis-ae # python3 ./scripts/postprocess_hpu_contention.py $(pwd)/logs
  3. osmosis-ae # python3 ./scripts/plot_hpu_contention.py
  4. Checkout figures/hpu_occupation.pdf