|
3 | 3 | ## Terminology
|
4 | 4 |
|
5 | 5 | * **NGraph operation**: Building block of neural networks, such as convolution or pooling.
|
6 |
| -* **(clDNN) Primitive**: Basic NN operation that was defined in clDNN. One primitive is usually mapped to one ngraph operation, but graph compilation may cause the mapping not to be 1-to-1. |
| 6 | +* **(GPU) Primitive**: Basic NN operation that was defined in GPU. One primitive is usually mapped to one ngraph operation, but graph compilation may cause the mapping not to be 1-to-1. |
7 | 7 | * **Kernel**: Actual body of execution in GPU. It also refers to specific implementations of **Primitive** for GPU, such as `convolution_gpu_winograd_2x3_s1.cl`. Usually, single kernel fulfills the operation of a single primitive, but several kernels may be used to support one primitive.
|
8 |
| -* **Unittest**: Single-layer test within clDNN. |
| 8 | +* **Unittest**: Single-layer test within GPU plugin. |
9 | 9 | * **Functional test**: Single-layer test in IE.
|
10 | 10 |
|
11 | 11 | ## Adding new primitive
|
|
14 | 14 | * Review the [ngraph operation spec](https://github.com/openvinotoolkit/openvino/tree/master/docs/ops)
|
15 | 15 | * IE operations(a.k.a primitive or NN-layer) are defined by ngraph.
|
16 | 16 | * You can check ngraph reference implementation of the primitive as well
|
17 |
| - * For example, [Scatter Elements Update in nGraph](https://github.com/openvinotoolkit/openvino/blob/master/src/core/reference/include/ngraph/runtime/reference/scatter_elements_update.hpp) |
| 17 | + * For example, [Scatter Elements Update in nGraph](https://github.com/openvinotoolkit/openvino/blob/master/src/core/reference/include/openvino/reference/scatter_elements_update.hpp) |
18 | 18 |
|
19 | 19 | 1. Try to find existing primitive that fully or partially covers this operation.
|
20 | 20 | * It is also possible to transform the network so that the missing primitive is covered from existing primitive.
|
21 | 21 | * For example, [replace reduce with pooling](https://github.com/openvinotoolkit/openvino/blob/23808f46f7b5d464fd649ad278f253eec12721b3/inference-engine/src/cldnn_engine/cldnn_engine.cpp#L205).
|
22 | 22 |
|
23 |
| -1. Add new / extend existing clDNN primitive according to the operation spec. |
24 |
| - 1. This phase is to enable primitive within clDNN library, without exposing it to IE. |
| 23 | +1. Add new / extend existing GPU primitive according to the operation spec. |
| 24 | + 1. This phase is to enable primitive within GPU plugin, without exposing it to IE. |
25 | 25 | 1. Implement **reference parallel kernel** that supports all parameters of the operation and all input/output data types and layouts.
|
26 | 26 |
|
27 | 27 | | File | Description |
|
28 | 28 | |------|-------------|
|
29 | 29 | | [scatter_elements_update_ref.cl](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/src/kernel_selector/cl_kernels/scatter_elements_update_ref.cl) | OpenCL Kernel body. For more detail, please see [How to write OCL kernel](#writing-ocl-kernel) section |
|
30 | 30 | | [scatter_elements_update_kernel_ref.(cpp,h)](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/src/kernel_selector/kernels/scatter_update/scatter_elements_update_kernel_ref.cpp) | Counterpart of kernel body for host |
|
31 | 31 | | [scatter_elements_update_kernel_selector.(cpp,h)](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/src/kernel_selector/kernels/scatter_update/scatter_elements_update_kernel_selector.cpp) | Kernel selector for a primitive |
|
32 |
| - | [register_gpu.(cpp,hpp)](https://github.com/openvinotoolkit/openvino/blob/master/inference-engine/thirdparty/clDNN/src/gpu/register_gpu.cpp) | Primitive registration | |
33 |
| - | [scatter_elements_update_gpu.cpp](https://github.com/openvinotoolkit/openvino/blob/master/inference-engine/thirdparty/clDNN/src/gpu/scatter_elements_update_gpu.cpp) | Primitive registration, input spec | |
34 |
| - | [scatter_elements_update_inst.h](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/src/graph/include/scatter_elements_update_inst.h) | Node type declaration for clDNN program | |
35 |
| - | [clDNN/src/scatter_elements_update.cpp](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/src/graph/scatter_elements_update.cpp) | Code for scatter_elements_update_inst.h | |
36 |
| - | [clDNN/api/cldnn/primitives/scatter_elements_update.hpp](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/include/intel_gpu/primitives/scatter_elements_update.hpp) | clDNN primitive definition | |
| 32 | + | [register.(cpp,hpp)](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/src/graph/impls/ocl/register.cpp) | Primitive registration | |
| 33 | + | [scatter_elements_update.cpp](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/src/graph/impls/ocl/scatter_elements_update.cpp) | Primitive registration, input spec | |
| 34 | + | [scatter_elements_update_inst.h](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/src/graph/include/scatter_elements_update_inst.h) | Node type declaration for GPU program | |
| 35 | + | [src/graph/scatter_elements_update.cpp](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/src/graph/scatter_elements_update.cpp) | Code for scatter_elements_update_inst.h | |
| 36 | + | [primitives/scatter_elements_update.hpp](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/include/intel_gpu/primitives/scatter_elements_update.hpp) | GPU primitive definition | |
37 | 37 | | [common_types.h](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/src/kernel_selector/common_types.h) | Enum declaration for KernelType and arguments |
|
38 | 38 |
|
39 | 39 | 1. Add unit tests for the new operation.
|
40 | 40 |
|
41 | 41 | | File | Description |
|
42 | 42 | |------|-------------|
|
43 |
| - | [scatter_elements_update_gpu_test.cpp](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/tests/test_cases/scatter_elements_update_gpu_test.cpp) | Unittest for layer | |
| 43 | + | [scatter_elements_update_gpu_test.cpp](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/tests/unit/test_cases/scatter_elements_update_gpu_test.cpp) | Unittest for layer | |
44 | 44 |
|
45 | 45 | * You need to add reference code or expected result for checking the result.
|
46 | 46 |
|
|
67 | 67 | 1. Support layer fusion, if applicable
|
68 | 68 | * It is usually easy to fuse some layers, such as *scale*, *activation*, *quantize*, and *eltwise*, into the previous layer. This fusing rule can be added to `prepare_primitive_fusing::fuse_simple_primitives`.
|
69 | 69 | * `fuse_simple_primitives` is called during [graph compilation phase](https://github.com/openvinotoolkit/openvino/blob/71c50c224964bf8c24378d16f015d74e2c1e1ce8/inference-engine/thirdparty/clDNN/src/program.cpp#L430)
|
70 |
| - * See general description of layer fusion [here](https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_supported_plugins_CL_DNN.html#optimizations) |
71 |
| - * Unit tests for layer fusion are placed in a single file: [fusings_gpu_test.cpp](https://github.com/openvinotoolkit/openvino/blob/master/inference-engine/thirdparty/clDNN/tests/test_cases/fusings_gpu_test.cpp). It is also compiled into `ov_gpu_unit_tests`. |
| 70 | + * Unit tests for layer fusion: [link](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/tests/unit/fusions/scatter_elements_update_fusion_test.cpp). It is also compiled into `ov_gpu_unit_tests`. |
72 | 71 | * Code for fused layers are generated with `jitter`. It is created as `FUSED_OPS..` macro in OCL code. This generation logic is in `KernelBase::MakeFusedOpsJitConstants`.
|
73 | 72 |
|
74 | 73 | 1. Add / update factory for this operation in the GPU plugin to use new primitive in inference-engine.
|
75 | 74 |
|
76 | 75 | | File | Description |
|
77 | 76 | |------|-------------|
|
78 |
| - | [cldnn_engine/ops/scatter_elements_update.cpp](https://github.com/openvinotoolkit/openvino/blob/master/inference-engine/src/cldnn_engine/ops/scatter_elements_update.cpp) | Instantiation from clDNN plugin for IE | |
79 |
| - | [cldnn_primitives_list.hpp](https://github.com/openvinotoolkit/openvino/blob/master/inference-engine/src/cldnn_engine/cldnn_primitives_list.hpp) | Registration for primitives | |
| 77 | + | [plugin/ops/scatter_elements_update.cpp](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/src/plugin/ops/scatter_elements_update.cpp) | Instantiation of gpu plugin primitive from IE | |
| 78 | + | [primitives_list.hpp](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/include/intel_gpu/plugin/primitives_list.hpp) | Registration for primitives | |
80 | 79 |
|
81 | 80 | 1. Add functional single-layer tests for the operation and try to cover most of the different use cases of this operation.
|
82 | 81 |
|
@@ -126,7 +125,7 @@ Jitter generates macros for index calculations. With these macros, you can progr
|
126 | 125 |
|
127 | 126 | If a kernel is not performance-critical, you can support `bfyx`, `bfzyx` and `bfwzyx` only for layout. Those are default layouts. As an optimized format, `b_fs_yx_fsv16`, `b_fs_yx_fsv4` or `byxf` can be used as well.
|
128 | 127 |
|
129 |
| -[General description of layout can be found here](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/docs/gpu_memory_formats.md) and [header file is here](https://github.com/openvinotoolkit/openvino/blob/master/inference-engine/thirdparty/clDNN/api/tensor.hpp). |
| 128 | +[General description of layout can be found here](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/docs/gpu_memory_formats.md) and [header file is here](https://github.com/openvinotoolkit/openvino/blob/master/src/plugins/intel_gpu/include/intel_gpu/runtime/format.hpp). |
130 | 129 |
|
131 | 130 | ### Layer fusion
|
132 | 131 |
|
|
0 commit comments