Skip to content
Titanium Network

WispMark

WispMark

WispMark is a benchmarking tool for Wisp protocol implementations.

Installation:

To run this repository, install the Xonsh shell, and all the dependencies for the Wisp implementations.

You need:

  • Git
  • CPython and PyPy
  • NodeJS
  • Rust Nightly
  • GCC
  • iftop

Then run ./wispmark.xsh to start the tests.

Methodology:

This program pairs each Wisp server with each Wisp client, with a TCP echo server running on port 6002. The amount of traffic passing through that port is used to calculate the bandwidth that was achieved with each configuration.

Implementations Tested:

Server:

Client:

Usage:

usage: wispmark.xsh [-h] [--duration DURATION]


A benchmarking tool for Wisp protocol implementations.


options:
  -h, --help           show this help message and exit
  --duration DURATION  The duration of each test, in seconds. The default is 10s.

Current Results:

Note that test results can vary wildly across CPUs.

CPU: AMD EPYC 7763 64-Core Processor (x4)
Test duration: 60s
                             | wisp-client-js (1) | wisp-client-js (10) | wisp-mux (1) | wisp-mux (10)
-----------------------------+--------------------+---------------------+--------------+---------------
wisp-server-node             | 564.61 MiB/s       | 655.85 MiB/s        | DNF          | 516.09 MiB/s
-----------------------------+--------------------+---------------------+--------------+---------------
wisp-js                      | 455.18 MiB/s       | 524.83 MiB/s        | 483.85 MiB/s | 516.07 MiB/s
-----------------------------+--------------------+---------------------+--------------+---------------
wisp-server-python (python3) | 137.09 MiB/s       | 143.83 MiB/s        | DNF          | 569.98 MiB/s
-----------------------------+--------------------+---------------------+--------------+---------------
wisp-server-python (pypy3)   | 122.22 MiB/s       | 133.14 MiB/s        | 156.08 MiB/s | 144.29 MiB/s
-----------------------------+--------------------+---------------------+--------------+---------------
epoxy-server                 | 384.46 MiB/s       | 431.21 MiB/s        | 359.93 MiB/s | 434.56 MiB/s
-----------------------------+--------------------+---------------------+--------------+---------------
WispServerCpp                | DNF                | 448.17 MiB/s        | DNF          | 526.3 MiB/s


CPU: 13th Gen Intel(R) Core(TM) i7-1360P (x16)
Test duration: 10s
                             | wisp-client-js (1) | wisp-client-js (10) | wisp-mux (1)  | wisp-mux (10)
-----------------------------+--------------------+---------------------+---------------+---------------
wisp-server-node             | 848.73 MiB/s       | 1419.97 MiB/s       | 1277.58 MiB/s | 1264.52 MiB/s
-----------------------------+--------------------+---------------------+---------------+---------------
wisp-js                      | 812.4 MiB/s        | 1181.24 MiB/s       | 1192.8 MiB/s  | 1215.22 MiB/s
-----------------------------+--------------------+---------------------+---------------+---------------
wisp-server-python (python3) | 206.11 MiB/s       | 213.21 MiB/s        | 900.24 MiB/s  | 991.37 MiB/s
-----------------------------+--------------------+---------------------+---------------+---------------
wisp-server-python (pypy3)   | 238.14 MiB/s       | 240.52 MiB/s        | 379.67 MiB/s  | DNF
-----------------------------+--------------------+---------------------+---------------+---------------
epoxy-server                 | 803.06 MiB/s       | 1392.41 MiB/s       | 1302.48 MiB/s | 1716.66 MiB/s
-----------------------------+--------------------+---------------------+---------------+---------------
WispServerCpp                | 405.76 MiB/s       | 673.73 MiB/s        | 591.34 MiB/s  | 578.8 MiB/s


CPU: 13th Gen Intel(R) Core(TM) i7-1360P (x16)
Test duration: 60s
                             | wisp-client-js (1) | wisp-client-js (10) | wisp-mux (1)  | wisp-mux (10)
-----------------------------+--------------------+---------------------+---------------+---------------
wisp-server-node             | 893.27 MiB/s       | 1253.8 MiB/s        | 1382.92 MiB/s | 1402.81 MiB/s
-----------------------------+--------------------+---------------------+---------------+---------------
wisp-js                      | 900.04 MiB/s       | 1312.59 MiB/s       | 1296.05 MiB/s | 1282.25 MiB/s
-----------------------------+--------------------+---------------------+---------------+---------------
wisp-server-python (python3) | 212.61 MiB/s       | 186.74 MiB/s        | 941.82 MiB/s  | 1005.01 MiB/s
-----------------------------+--------------------+---------------------+---------------+---------------
wisp-server-python (pypy3)   | 217.36 MiB/s       | 221.13 MiB/s        | 397.41 MiB/s  | 419.53 MiB/s
-----------------------------+--------------------+---------------------+---------------+---------------
epoxy-server                 | 866.1 MiB/s        | 1154.24 MiB/s       | 1297.09 MiB/s | 1485.32 MiB/s
-----------------------------+--------------------+---------------------+---------------+---------------
WispServerCpp                | 385.36 MiB/s       | 728.39 MiB/s        | 540.07 MiB/s  | 567.05 MiB/s

This program is licensed under the GNU GPL v3.

WispMark: A benchmarking tool for Wisp protocol implementations.
Copyright (C) 2024 ading2210


This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.


This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.


You should have received a copy of the GNU General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.

Written largely by ading2210