Skip to content

Commit ff7d9c7

Browse files
claynerobisonclaynerobison
authored
Adding README files for Intel® Data Center Flex Series GPUs (#125)
1 parent 362bd03 commit ff7d9c7

File tree

9 files changed

+604
-42
lines changed

9 files changed

+604
-42
lines changed

‎README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Model Zoo for Intel® Architecture
22

3-
This repository contains **links to pre-trained models, sample scripts, best practices, and step-by-step tutorials** for many popular open-source machine learning models optimized by Intel to run on Intel® Xeon® Scalable processors.
3+
This repository contains **links to pre-trained models, sample scripts, best practices, and step-by-step tutorials** for many popular open-source machine learning models optimized by Intel to run on Intel® Xeon® Scalable processors and Intel® Data Center GPUs.
44

55
Model packages and containers for running the Model Zoo's workloads can be found at the [Intel® Developer Catalog](https://software.intel.com/containers).
66

‎docs/general/FLEX_DEVCATALOG.md

+23
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Model Zoo for Intel® Architecture Workloads Optimized for the Intel® Data Center GPU Flex Series
2+
3+
This document provides links to step-by-step instructions on how to leverage Model Zoo docker containers to run optimized open-source Deep Learning inference workloads using Intel® Extension for PyTorch* and Intel® Extension for TensorFlow* on the [Intel® Data Center GPU Flex Series](https://www.intel.com/content/www/us/en/products/docs/discrete-gpus/data-center-gpu/flex-series/overview.html).
4+
5+
## Base Containers
6+
7+
| AI Framework | Extension | Documentation |
8+
| -----------------------------| ------------- | ----------------- |
9+
| PyTorch | Intel® Extension for PyTorch* | [Intel® Extension for PyTorch Container](https://github.com/IntelAI/models/blob/master/quickstart/ipex-tool-container/gpu/devcatalog.md) |
10+
| TensorFlow | Intel® Extension for TensorFlow* | [Intel® Extension for TensorFlow Container](https://github.com/IntelAI/models/blob/master/quickstart/tf-tool-container/gpu/devcatalog.md)|
11+
12+
## Optimized Workloads
13+
14+
The table below provides links to run each workload in a docker container. The containers are optimized for Linux*.
15+
16+
17+
| Model | Framework | Mode | Documentation | Dataset |
18+
| ----------------------------| ---------- | ----------| ------------------- | ------------ |
19+
| [ResNet 50 v1.5](https://github.com/tensorflow/models/tree/v2.11.0/official/legacy/image_classification/resnet) | TensorFlow | Inference| [INT8](https://github.com/IntelAI/models/blob/master/quickstart/image_recognition/tensorflow/resnet50v1_5/inference/gpu/devcatalog.md) | [ImageNet 2012](https://github.com/IntelAI/models/tree/master/datasets/imagenet/README.md) |
20+
| [ResNet 50 v1.5](https://arxiv.org/pdf/1512.03385.pdf) | PyTorch | Inference | [INT8 ](https://github.com/IntelAI/models/blob/master/quickstart/image_recognition/pytorch/resnet50v1_5/inference/gpu/devcatalog.md) | [ImageNet 2012](https://github.com/IntelAI/models/tree/master/datasets/imagenet/README.md) |
21+
| [SSD-MobileNet v1](https://arxiv.org/pdf/1704.04861.pdf) | PyTorch | Inference | [INT8](https://github.com/IntelAI/models/blob/master/quickstart/quickstart/object_detection/pytorch/ssd-mobilenet/inference/gpu/devcatalog.md) | [COCO 2017](https://github.com/IntelAI/models/blob/master/quickstart/object_detection/pytorch/ssd-mobilenet/inference/gpu/README.md#datasets) |
22+
| [YOLO v4](https://arxiv.org/pdf/1704.04861.pdf) | PyTorch | Inference |[INT8](https://github.com/IntelAI/models/blob/master/quickstart/object_detection/pytorch/yolov4/inference/gpu/devcatalog.md) | [COCO 2017](https://github.com/IntelAI/models/blob/master/quickstart/object_detection/pytorch/ssd-mobilenet/inference/gpu/README.md#datasets) |
23+
| [SSD-MobileNet](https://arxiv.org/pdf/1704.04861.pdf) | TensorFlow | Inference | [INT8](https://github.com/IntelAI/models/blob/master/quickstart/object_detection/tensorflow/ssd-mobilenet/inference/gpu/devcatalog.md)| [COCO 2017 validation dataset](https://github.com/IntelAI/models/tree/master/datasets/coco#download-and-preprocess-the-coco-validation-images) |

‎quickstart/image_recognition/pytorch/resnet50v1_5/inference/gpu/DEVCATALOG_FLEX.md

+102
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
# Running ResNet50 v1.5 Inference with Int8 on Intel® Data Center GPU Flex Series using Intel® Extension for PyTorch*
2+
3+
4+
## Overview
5+
6+
This document has instructions for running ResNet50v1.5 inference using Intel(R) Extension for PyTorch with GPU.
7+
8+
## Requirements
9+
| Item | Detail |
10+
| ------ | ------- |
11+
| Host machine | Intel® Data Center GPU Flex Series |
12+
| Drivers | GPU-compatible drivers need to be installed: [Download Driver 476.14](https://dgpu-docs.intel.com/releases/stable_476_14_20221021.html)
13+
| Software | Docker* Installed |
14+
15+
## Get Started
16+
17+
## Download Datasets
18+
19+
The [ImageNet](http://www.image-net.org/) validation dataset is used.
20+
21+
Download and extract the ImageNet2012 dataset from http://www.image-net.org/,
22+
then move validation images to labeled subfolders, using
23+
[the valprep.sh shell script](https://raw.githubusercontent.com/soumith/imagenetloader.torch/master/valprep.sh)
24+
25+
A after running the data prep script, your folder structure should look something like this:
26+
27+
```
28+
imagenet
29+
└── val
30+
├── ILSVRC2012_img_val.tar
31+
├── n01440764
32+
│ ├── ILSVRC2012_val_00000293.JPEG
33+
│ ├── ILSVRC2012_val_00002138.JPEG
34+
│ ├── ILSVRC2012_val_00003014.JPEG
35+
│ ├── ILSVRC2012_val_00006697.JPEG
36+
│ └── ...
37+
└── ...
38+
```
39+
The folder that contains the `val` directory should be set as the
40+
`DATASET_DIR`
41+
(for example: `export DATASET_DIR=/home/<user>/imagenet`).
42+
43+
## Quick Start Scripts
44+
45+
| Script name | Description |
46+
|-------------|-------------|
47+
| `inference_block_format.sh` | Runs ResNet50 inference (block format) for the specified precision (int8) |
48+
49+
## Run Using Docker
50+
51+
### Set up Docker Image
52+
53+
```
54+
docker pull intel/image-recognition:pytorch-flex-gpu-resnet50v1-5-inference
55+
```
56+
### Run Docker Image
57+
The ResNet50 v1-5 inference container includes scripts,model and libraries need to run int8 inference. To run the `inference_block_format.sh` quickstart script using this container, you'll need to provide volume mounts for the ImageNet dataset. You will need to provide an output directory where log files will be written.
58+
59+
```
60+
export PRECISION=int8
61+
export OUTPUT_DIR=<path to output directory>
62+
export DATASET_DIR=<path to the preprocessed imagenet dataset>
63+
export SCRIPT=quickstart/inference_block_format.sh
64+
65+
DOCKER_ARGS=${DOCKER_ARGS:---rm -it}
66+
IMAGE_NAME=intel/image-recognition:pytorch-flex-gpu-resnet50v1-5-inference
67+
68+
69+
VIDEO=$(getent group video | sed -E 's,^video:[^:]*:([^:]*):.*$,\1,')
70+
RENDER=$(getent group render | sed -E 's,^render:[^:]*:([^:]*):.*$,\1,')
71+
72+
test -z "$RENDER" || RENDER_GROUP="--group-add ${RENDER}"
73+
74+
docker run \
75+
-v <your-local-dir>:/workspace \
76+
--group-add ${VIDEO} \
77+
${RENDER_GROUP} \
78+
--device=/dev/dri \
79+
--ipc=host \
80+
--env PRECISION=${PRECISION} \
81+
--env OUTPUT_DIR=${OUTPUT_DIR} \
82+
--env DATASET_DIR=${DATASET_DIR} \
83+
--env http_proxy=${http_proxy} \
84+
--env https_proxy=${https_proxy} \
85+
--env no_proxy=${no_proxy} \
86+
--volume ${OUTPUT_DIR}:${OUTPUT_DIR} \
87+
--volume ${DATASET_DIR}:${DATASET_DIR} \
88+
${DOCKER_ARGS} \
89+
${IMAGE_NAME} \
90+
/bin/bash $SCRIPT
91+
```
92+
93+
## Documentation and Sources
94+
95+
[GitHub* Repository](https://github.com/IntelAI/models/tree/master/dockerfiles/model_containers)
96+
97+
## Support
98+
Support for Intel® Extension for PyTorch* is found via the [Intel® AI Analytics Toolkit.](https://www.intel.com/content/www/us/en/developer/tools/oneapi/ai-analytics-toolkit.html#gs.qbretz) Additionally, the Intel® Extension for PyTorch* team tracks both bugs and enhancement requests using [GitHub issues](https://github.com/intel/intel-extension-for-pytorch/issues). Before submitting a suggestion or bug report, please search the GitHub issues to see if your issue has already been reported.
99+
100+
## License Agreement
101+
102+
LEGAL NOTICE: By accessing, downloading or using this software and any required dependent software (the “Software Package”), you agree to the terms and conditions of the software license agreements for the Software Package, which may also include notices, disclaimers, or license terms for third party software included with the Software Package. Please refer to the [license file](https://github.com/IntelAI/models/tree/master/third_party) for additional details.

‎quickstart/image_recognition/tensorflow/resnet50v1_5/inference/gpu/devcatalog.md

+36-21
Original file line numberDiff line numberDiff line change
@@ -1,56 +1,65 @@
1-
# ResNet50 v1.5 Inference
1+
# Running ResNet50 v1.5 Inference with Int8 on Intel® Data Center GPU Flex Series using Intel® Extension for TensorFlow*
22

3-
## Description
3+
## Overview
44

5-
This document has instructions for running ResNet50 v1.5 inference using
6-
Intel(R) Extension for TensorFlow* with Intel(R) Data Center GPU Flex Series.
5+
This document has instructions for running ResNet50 v1.5 inference using Intel(R) Extension for TensorFlow* with Intel(R) Data Center GPU Flex Series.
76

8-
## Datasets
7+
8+
## Requirements
9+
| Item | Detail |
10+
| ------ | ------- |
11+
| Host machine | Intel® Data Center GPU Flex Series |
12+
| Drivers | GPU-compatible drivers need to be installed: [Download Driver 476.14](https://dgpu-docs.intel.com/releases/stable_476_14_20221021.html)
13+
| Software | Docker* Installed |
14+
15+
## Get Started
16+
17+
### Download Datasets
918

1019
Download and preprocess the ImageNet dataset using the [instructions here](https://github.com/IntelAI/models/blob/master/datasets/imagenet/README.md).
1120
After running the conversion script you should have a directory with the
1221
ImageNet dataset in the TF records format.
1322

1423
Set the `DATASET_DIR` to point to the TF records directory when running ResNet50 v1.5.
1524

16-
## Quick Start Scripts
25+
### Quick Start Scripts
1726

1827
| Script name | Description |
1928
|:-------------:|:-------------:|
20-
| `online_inference` | Runs online inference for int8 precision |
29+
| `online_inference` | Runs online inference for int8 precision |
2130
| `batch_inference` | Runs batch inference for int8 precision |
2231
| `accuracy` | Measures the model accuracy for int8 precision |
2332

24-
## Docker
2533

26-
Requirements:
27-
* Host machine has Intel(R) Data Center GPU Flex Series
28-
* Follow instructions to install GPU-compatible driver [419.40](https://dgpu-docs.intel.com/releases/stable_419_40_20220914.html)
29-
* Docker
34+
## Run Using Docker
3035

31-
### Docker pull command:
36+
### Set up Docker Image
3237

3338
```
34-
docker pull intel/image-recognition:tf-atsm-gpu-resnet50v1-5-inference
39+
docker pull intel/image-recognition:tf-flex-gpu-resnet50v1-5-inference
3540
```
3641

37-
The ResNet50 v1-5 inference container includes scripts,model and libraries need to run int8 inference. To run one of the inference quickstart scripts using this container, you'll need to provide volume mounts for the ImageNet dataset for running `accuracy.sh` script. For `online_inference.sh` and `batch_inference.sh` dummy dataset will be used. You will need to provide an output directory where log files will be written.
42+
### Run Docker Image
43+
The ResNet50 v1-5 inference container includes scripts,model and libraries need to run int8 inference. To run one of the inference quickstart scripts using this container, you'll need to provide volume mounts for the ImageNet dataset for running `accuracy.sh` script. For `online_inference.sh` and `batch_inference.sh` dummy dataset will be used. You will need to provide an output directory where log files will be written.
3844

3945
```
4046
export PRECISION=int8
4147
export OUTPUT_DIR=<path to output directory>
4248
export DATASET_DIR=<path to the preprocessed imagenet dataset>
43-
IMAGE_NAME=intel/image-recognition:tf-atsm-gpu-resnet50v1-5-inference
49+
DOCKER_ARGS=${DOCKER_ARGS:---rm -it}
50+
IMAGE_NAME=intel/image-recognition:tf-flex-gpu-resnet50v1-5-inference
4451
4552
VIDEO=$(getent group video | sed -E 's,^video:[^:]*:([^:]*):.*$,\1,')
4653
RENDER=$(getent group render | sed -E 's,^render:[^:]*:([^:]*):.*$,\1,')
4754
55+
test -z "$RENDER" || RENDER_GROUP="--group-add ${RENDER}"
56+
4857
docker run \
58+
-v <your-local-dir>:/workspace \
4959
--group-add ${VIDEO} \
5060
${RENDER_GROUP} \
5161
--device=/dev/dri \
5262
--ipc=host \
53-
--privileged \
5463
--env PRECISION=${PRECISION} \
5564
--env OUTPUT_DIR=${OUTPUT_DIR} \
5665
--env DATASET_DIR=${DATASET_DIR} \
@@ -59,16 +68,22 @@ docker run \
5968
--env no_proxy=${no_proxy} \
6069
--volume ${OUTPUT_DIR}:${OUTPUT_DIR} \
6170
--volume ${DATASET_DIR}:${DATASET_DIR} \
62-
--rm -it \
63-
$IMAGE_NAME \
71+
${DOCKER_ARGS} \
72+
${IMAGE_NAME} \
6473
/bin/bash quickstart/<script name>.sh
6574
```
6675

6776
## Documentation and Sources
6877

69-
**Get Started**
78+
[GitHub* Repository](https://github.com/IntelAI/models/tree/master/dockerfiles/model_containers)
79+
80+
## Summary and Next Steps
81+
82+
Now you are inside container with Python 3.9 and Tensorflow 2.10.0 preinstalled. You can run your own script
83+
to run on intel GPU.
7084

71-
[Docker* Repository](https://hub.docker.com/r/intel/image-recognition)
85+
## Support
86+
Support for Intel® Extension for TensorFlow* is found via the [Intel® AI Analytics Toolkit.](https://www.intel.com/content/www/us/en/developer/tools/oneapi/ai-analytics-toolkit.html#gs.qbretz) Additionally, the Intel® Extension for TensorFlow* team tracks both bugs and enhancement requests using [GitHub issues](https://github.com/intel/intel-extension-for-tensorflow/issues). Before submitting a suggestion or bug report, please search the GitHub issues to see if your issue has already been reported.
7287

7388
## License Agreement
7489

‎quickstart/ipex-tool-container/gpu/devcatalog.md

+87
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
# Optimizations for Intel® Data Center GPU Flex Series using Intel® Extension for PyTorch*
2+
3+
## Overview
4+
5+
This document has instruction for running Intel® Extension for PyTorch* (IPEX) for
6+
GPU in container.
7+
8+
## Requirements
9+
| Item | Detail |
10+
| ------ | ------- |
11+
| Host machine | Intel® Data Center GPU Flex Series |
12+
| Drivers | GPU-compatible drivers need to be installed: [Download Driver 476.14](https://dgpu-docs.intel.com/releases/stable_476_14_20221021.html)
13+
| Software | Docker* Installed |
14+
15+
## Get Started
16+
17+
### Installing the Intel Extensions for PyTorch
18+
#### Docker pull command:
19+
20+
`docker pull intel/intel-extension-for-pytorch:xpu-flex`
21+
22+
### Running container:
23+
24+
Run following commands to start IPEX GPU tools container. You can use `-v` option to mount your
25+
local directory into container. The `-v` argument can be omitted if you do not need
26+
access to a local directory in the container. Pass the video and render groups to your
27+
docker container so that the GPU is accessible.
28+
```
29+
IMAGE_NAME=intel/intel-extension-for-pytorch:xpu-flex
30+
DOCKER_ARGS=${DOCKER_ARGS:---rm -it}
31+
32+
VIDEO=$(getent group video | sed -E 's,^video:[^:]*:([^:]*):.*$,\1,')
33+
RENDER=$(getent group render | sed -E 's,^render:[^:]*:([^:]*):.*$,\1,')
34+
35+
test -z "$RENDER" || RENDER_GROUP="--group-add ${RENDER}"
36+
37+
docker run --rm \
38+
-v <your-local-dir>:/workspace \
39+
--group-add ${VIDEO} \
40+
${RENDER_GROUP} \
41+
--device=/dev/dri \
42+
--ipc=host \
43+
-e http_proxy=$http_proxy \
44+
-e https_proxy=$https_proxy \
45+
-e no_proxy=$no_proxy \
46+
${DOCKER_ARGS} \
47+
${IMAGE_NAME} \
48+
bash
49+
```
50+
51+
#### Verify if XPU is accessible from PyTorch:
52+
You are inside container now. Run following command to verify XPU is visible to PyTorch:
53+
```
54+
python -c "import torch;print(torch.device('xpu'))"
55+
```
56+
Sample output looks like below:
57+
```
58+
xpu
59+
```
60+
Then, verify that the XPU device is available to IPEX:
61+
```
62+
python -c "import intel_extension_for_pytorch as ipex;print(ipex.xpu.is_available())"
63+
```
64+
Sample output looks like below:
65+
```
66+
True
67+
```
68+
Finally, use the following command to check whether MKL is enabled as default:
69+
```
70+
python -c "import intel_extension_for_pytorch as ipex;print(ipex.xpu.has_onemkl())"
71+
```
72+
Sample output looks like below:
73+
```
74+
True
75+
```
76+
77+
## Summary and Next Steps
78+
Now you are inside container with Python 3.9, PyTorch and IPEX preinstalled. You can run your own script
79+
to run on Intel GPU.
80+
81+
## Documentation and Sources
82+
83+
[GitHub* Repository](https://github.com/intel/intel-extension-for-pytorch/tree/master/docker)
84+
85+
86+
## Support
87+
Support for Intel® Extension for PyTorch* is found via the [Intel® AI Analytics Toolkit.](https://www.intel.com/content/www/us/en/developer/tools/oneapi/ai-analytics-toolkit.html#gs.qbretz) Additionally, the Intel® Extension for PyTorch* team tracks both bugs and enhancement requests using [GitHub issues](https://github.com/intel/intel-extension-for-pytorch/issues). Before submitting a suggestion or bug report, please search the GitHub issues to see if your issue has already been reported.

0 commit comments

Comments
 (0)