-
Notifications
You must be signed in to change notification settings - Fork 3
Benchmarks
Artem Trofimov edited this page Aug 1, 2022
·
13 revisions
- Kubeflow
- Airflow
- Metaflow
Metric: the time interval between user clicks run and the task is started.
The following setups should be covered:
- With custom conda env
- With custom docker env
ʎzy-specific cases/parameters:
- With and w/o cached environment
- With and w/o cached servants
Metric: the time taken on the data transfer of 1GB between 2 tasks.
Workload: data preparation on CPU + learning on GPU
Metric 1: the time interval between user clicks run and the task is started depending on the cluster size and the number of concurrent users.
Metric 2: the ratio of allocated VMs time to really consumed by tasks time.
ʎzy-specific cases/parameters:
- The number of VMs in pool
- Servants caching timeout
Workload: data preparation on CPU + learning on GPU