Skip to content

Fix benchmark names #398

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
chentex wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

Fix benchmark names #398

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a69dc1d
Updates benchmark names
chentex Nov 29, 2021
f3bdbab
Fixing fs-drift repo link
chentex Nov 29, 2021
b411745
Update ycsb image name
chentex Nov 29, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ python3.7 ./snafu/run_snafu.py --tool archive --archive-file /tmp/my_sysbench_da
| YCSB | Database Performance | Working |
| Pgbench | Postgres Performance | Working |
| smallfile | metadata-intensive ops | Working |
| fs-drift | metadata-intensive mix | Working |
| fs_drift | metadata-intensive mix | Working |
| cyclictest | Real-Time Latency | Working |
| oslat | Real-Time Latency | Working |
| OpenShift Upgrade | Time to upgrade | Working |
Expand Down
4 changes: 2 additions & 2 deletions ci/common.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@ function update_operator_image() {
sed -i "s#${default_ripsaw_image_prefix}/${workload}:latest#${SNAFU_WRAPPER_IMAGE_PREFIX}/${workload}:${SNAFU_IMAGE_TAG}#g" roles/${workload}/templates/*
done
sed -i "s#${default_ripsaw_image_prefix}/fio:latest#${SNAFU_WRAPPER_IMAGE_PREFIX}/fio:${SNAFU_IMAGE_TAG}#g" roles/fio_distributed/templates/*
sed -i "s#${default_ripsaw_image_prefix}/ycsb-server:latest#${SNAFU_WRAPPER_IMAGE_PREFIX}/ycsb-server:${SNAFU_IMAGE_TAG}#g" roles/ycsb/templates/*
sed -i "s#${default_ripsaw_image_prefix}/ycsb:latest#${SNAFU_WRAPPER_IMAGE_PREFIX}/ycsb:${SNAFU_IMAGE_TAG}#g" roles/ycsb/templates/*
sed -i "s#${default_ripsaw_image_prefix}/pgbench:latest#${SNAFU_WRAPPER_IMAGE_PREFIX}/pgbench:${SNAFU_IMAGE_TAG}#g" roles/pgbench/defaults/main.yml
sed -i "s#${default_ripsaw_image_prefix}/fs-drift:latest#${SNAFU_WRAPPER_IMAGE_PREFIX}/fs-drift:${SNAFU_IMAGE_TAG}#g" roles/fs-drift/templates/* roles/fs-drift/tasks/*
sed -i "s#${default_ripsaw_image_prefix}/fs_drift:latest#${SNAFU_WRAPPER_IMAGE_PREFIX}/fs_drift:${SNAFU_IMAGE_TAG}#g" roles/fs_drift/templates/* roles/fs_drift/tasks/*
sed -i "s#${default_ripsaw_image_prefix}/smallfile:latest#${SNAFU_WRAPPER_IMAGE_PREFIX}/smallfile:${SNAFU_IMAGE_TAG}#g" roles/smallfile/templates/* roles/smallfile/tasks/*
image_spec=$image_location/$image_account/benchmark-operator:$SNAFU_IMAGE_TAG
make image-build image-push deploy IMG=$image_spec
Expand Down
3 changes: 2 additions & 1 deletion snafu/benchmarks/systemd_analyze/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
FROM registry.access.redhat.com/ubi8:latest

RUN dnf install -y --nodocs git python3-pip && dnf clean all
Copy link
Contributor

@learnitall learnitall Nov 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this change intentional here?

RUN dnf install -y --nodocs git python3-pip python3-devel gcc && dnf clean all
COPY snafu/image_resources/centos8-appstream.repo /etc/yum.repos.d/centos8-appstream.repo
RUN dnf install -y --nodocs redis --enablerepo=centos8-appstream && dnf clean all
RUN dnf install -y --nodocs hostname procps-ng iproute net-tools ethtool nmap iputils https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm && dnf clean all
RUN ln -s /usr/bin/python3 /usr/bin/python
RUN mkdir -p /opt/snafu/
COPY . /opt/snafu/
RUN pip3 install distro
RUN pip3 install --upgrade --no-cache-dir pip
RUN pip3 install -e /opt/snafu/
6 changes: 3 additions & 3 deletions snafu/fs_drift_wrapper/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,6 @@ COPY . /opt/snafu/
RUN pip3 install --upgrade pip
RUN pip3 install -e /opt/snafu/
ADD https://api.github.com/repos/parallel-fs-utils/fs-drift/git/refs/heads/master /tmp/bustcache
RUN git clone https://github.com/parallel-fs-utils/fs-drift /opt/fs-drift --depth 1
RUN ln -sv /opt/fs-drift/fs-drift.py /usr/local/bin/
RUN ln -sv /opt/fs-drift/rsptime_stats.py /usr/local/bin/
RUN git clone https://github.com/parallel-fs-utils/fs-drift /opt/fs_drift --depth 1
RUN ln -sv /opt/fs_drift/fs_drift.py /usr/local/bin/
RUN ln -sv /opt/fs_drift/rsptime_stats.py /usr/local/bin/
6 changes: 3 additions & 3 deletions snafu/fs_drift_wrapper/Dockerfile.ppc64le
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,6 @@ RUN python3 -m pip install --upgrade cython numpy importlib_metadata 'urllib3!=1
COPY . /opt/snafu/
RUN pip3 install -e /opt/snafu/
ADD https://api.github.com/repos/parallel-fs-utils/fs-drift/git/refs/heads/master /tmp/bustcache
RUN git clone https://github.com/parallel-fs-utils/fs-drift /opt/fs-drift --depth 1
RUN ln -sv /opt/fs-drift/fs-drift.py /usr/local/bin/
RUN ln -sv /opt/fs-drift/rsptime_stats.py /usr/local/bin/
RUN git clone https://github.com/parallel-fs-utils/fs-drift /opt/fs_drift --depth 1
RUN ln -sv /opt/fs_drift/fs_drift.py /usr/local/bin/
RUN ln -sv /opt/fs_drift/rsptime_stats.py /usr/local/bin/
2 changes: 1 addition & 1 deletion snafu/fs_drift_wrapper/ci_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ set -x
source ci/common.sh

# Build image for ci
image_spec=$SNAFU_WRAPPER_IMAGE_PREFIX/fs-drift:$SNAFU_IMAGE_TAG
image_spec=$SNAFU_WRAPPER_IMAGE_PREFIX/fs_drift:$SNAFU_IMAGE_TAG
build_and_push snafu/fs_drift_wrapper/Dockerfile $image_spec
pushd ripsaw
source tests/test_fs_drift.sh
2 changes: 1 addition & 1 deletion snafu/fs_drift_wrapper/fs_drift_wrapper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ def __init__(self, parser):
parser.add_argument(
"-d", "--dir", help="output parent directory", default=os.path.dirname(os.getcwd())
)
parser.add_argument("-y", "--yaml-input-file", help="fs-drift parameters passed via YAML input file")
parser.add_argument("-y", "--yaml-input-file", help="fs_drift parameters passed via YAML input file")
self.args = parser.parse_args()

if not self.args.top:
Expand Down
8 changes: 4 additions & 4 deletions snafu/fs_drift_wrapper/trigger_fs_drift.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,11 @@ def emit_actions(self):
if c.endswith(".csv"):
os.unlink(os.path.join(rsptime_dir, c))

json_output_file = os.path.join(self.result_dir, "fs-drift.json")
json_output_file = os.path.join(self.result_dir, "fs_drift.json")
network_shared_dir = os.path.join(self.working_dir, "network-shared")
rsptime_file = os.path.join(network_shared_dir, "stats-rsptimes.csv")
cmd = [
"fs-drift.py",
"fs_drift.py",
"--top",
self.working_dir,
"--output-json",
Expand All @@ -71,7 +71,7 @@ def emit_actions(self):
process = subprocess.check_call(cmd, stderr=subprocess.STDOUT)
except subprocess.CalledProcessError as e:
self.logger.exception(e)
raise FsDriftWrapperException("fs-drift.py non-zero process return code %d" % e.returncode)
raise FsDriftWrapperException("fs_drift.py non-zero process return code %d" % e.returncode)
self.logger.info("completed sample {} , results in {}".format(self.sample, json_output_file))
fsdict = get_vfs_stat_dict(self.working_dir)
with open(json_output_file) as f:
Expand Down Expand Up @@ -118,7 +118,7 @@ def emit_actions(self):
rsptime_date = start_time + int(flds[0])
rsptime_date_str = time.strftime("%Y-%m-%dT%H:%M:%S.000Z", time.gmtime(rsptime_date))
interval = {}
# number of fs-drift file operations in this interval
# number of fs_drift file operations in this interval
interval["op-count"] = int(flds[2])
if interval["op-count"] == 0:
self.logger.info(
Expand Down
6 changes: 3 additions & 3 deletions snafu/utils/wrapper_factory.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,14 @@
wrapper_dict = {
"fio": fio_wrapper,
"smallfile": smallfile_wrapper,
"fs-drift": fs_drift_wrapper,
"fs_drift": fs_drift_wrapper,
"hammerdb": hammerdb_wrapper,
"ycsb": ycsb_wrapper,
"pgbench": pgbench_wrapper,
"vegeta": vegeta_wrapper,
"scale": scale_openshift_wrapper,
"scale_openshift": scale_openshift_wrapper,
"stressng": stressng_wrapper,
"upgrade": upgrade_openshift_wrapper,
"upgrade_openshift": upgrade_openshift_wrapper,
"cyclictest": cyclictest_wrapper,
"oslat": oslat_wrapper,
"trex": trex_wrapper,
Expand Down
2 changes: 1 addition & 1 deletion snafu/ycsb_wrapper/ci_test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ set -x
source ci/common.sh

# Build image for ci
image_spec=$SNAFU_WRAPPER_IMAGE_PREFIX/ycsb-server:$SNAFU_IMAGE_TAG
image_spec=$SNAFU_WRAPPER_IMAGE_PREFIX/ycsb:$SNAFU_IMAGE_TAG
build_and_push snafu/ycsb_wrapper/Dockerfile $image_spec
pushd ripsaw
source tests/test_ycsb.sh