[v9,00/34] Implementation of revised ml/cnxk driver

  This patch series is an implementation of revised ml/cnxk driver
to support models compiled with TVM compiler framework. TVM models
use a hybrid mode for execution, with regions of the model executing
on the ML accelerator and the rest executing on CPU cores.

This series of commits reorganizes the ml/cnxk driver and adds support
to execute multiple regions with-in a TVM model.

v9:
  - Fixed incorrect IO layout for TVM Marvell models
  - Set byte offset to zero for I/O tensors
  - Updated max layers macro definition. Set to TVMDP max layers.
  - Fixed TVM model IO type to RTE IO type map

v8:
  - Updated CMake dependency resolution of external dependencies
  - Updated mldevs/cnxk documentation
  - Updated meson config files for cn9k and cn10k to include cmake

v7:
  - Updated steps to build dependencies in cnxk mldev documentation
  - Replace str functions with rte_str functions
  - Drop use of rte_exit in ml/cnxk driver

v6:
  - Added depends info for series. This series depends on patch-132887
  - Fix merge conflicts with dpdk-23.11-rc1
  - Fix issues with ml/cnxk driver release notes
  - Added build dependency information for dlpack headers

v5:
  - Fix build failures for individual patches in the series
  - Finished build testing with devtools/test-meson-builds.sh script

v4:
  - Squashed release notes
  - Updated external build dependency info in documentation

v3:
  - Reduced use of RTE_MLDEV_CNXK_ENABLE_MVTVM macro
  - Added stubs file with dummy functions to use when TVM is disabled
  - Dropped patch with internal function to read firmware
  - Updated ML CNXK PMD documentation
  - Added external library dependency info in documentation
  - Added release notes for 23.11

v2:
  - Fix xstats reporting
  - Fix issues reported by klocwork static analysis tool
  - Update external header inclusions

v1:
  - Initial changes

Anup Prabhu (2):
  ml/cnxk: enable OCM check for multilayer TVM model
  ml/cnxk: enable fast-path ops for TVM models

Prince Takkar (2):
  ml/cnxk: update internal TVM model info structure
  ml/cnxk: support quantize and dequantize callback

Srikanth Yalavarthi (30):
  ml/cnxk: drop support for register polling
  ml/cnxk: add generic cnxk device structure
  ml/cnxk: add generic model and layer structures
  ml/cnxk: add generic cnxk request structure
  ml/cnxk: add generic cnxk xstats structures
  ml/cnxk: rename cnxk ops function pointers struct
  ml/cnxk: update device handling functions
  ml/cnxk: update queue-pair handling functions
  ml/cnxk: update model load and unload functions
  ml/cnxk: update model start and stop functions
  ml/cnxk: update model utility functions
  ml/cnxk: update data quantization functions
  ml/cnxk: update device debug functions
  ml/cnxk: update device stats functions
  ml/cnxk: update device and model xstats functions
  ml/cnxk: update fast path functions
  ml/cnxk: move error handling to cnxk layer
  ml/cnxk: support config and close of tvmdp library
  ml/cnxk: add structures to support TVM model type
  ml/cnxk: add support for identify model type
  ml/cnxk: add support to parse TVM model objects
  ml/cnxk: fetch layer info and load TVM model
  ml/cnxk: update internal info for TVM model
  ml/cnxk: enable model unload in tvmdp library
  ml/cnxk: support start and stop for TVM models
  ml/cnxk: support device dump for TVM models
  ml/cnxk: enable reporting model runtime as xstats
  ml/cnxk: implement I/O alloc and free callbacks
  ml/cnxk: add generic ML malloc and free callback
  ml/cnxk: enable creation of mvtvm virtual device

 config/arm/arm64_cn10k_linux_gcc       |    1 +
 config/arm/arm64_cn9k_linux_gcc        |    1 +
 doc/guides/mldevs/cnxk.rst             |  223 +-
 doc/guides/rel_notes/release_23_11.rst |    3 +
 drivers/ml/cnxk/cn10k_ml_dev.c         |  416 ++--
 drivers/ml/cnxk/cn10k_ml_dev.h         |  457 +---
 drivers/ml/cnxk/cn10k_ml_model.c       |  403 ++--
 drivers/ml/cnxk/cn10k_ml_model.h       |  151 +-
 drivers/ml/cnxk/cn10k_ml_ocm.c         |  111 +-
 drivers/ml/cnxk/cn10k_ml_ocm.h         |   15 +-
 drivers/ml/cnxk/cn10k_ml_ops.c         | 2828 ++++++++----------------
 drivers/ml/cnxk/cn10k_ml_ops.h         |  358 ++-
 drivers/ml/cnxk/cnxk_ml_dev.c          |   22 +
 drivers/ml/cnxk/cnxk_ml_dev.h          |  120 +
 drivers/ml/cnxk/cnxk_ml_io.c           |   95 +
 drivers/ml/cnxk/cnxk_ml_io.h           |   90 +
 drivers/ml/cnxk/cnxk_ml_model.c        |   94 +
 drivers/ml/cnxk/cnxk_ml_model.h        |  192 ++
 drivers/ml/cnxk/cnxk_ml_ops.c          | 1690 ++++++++++++++
 drivers/ml/cnxk/cnxk_ml_ops.h          |   87 +
 drivers/ml/cnxk/cnxk_ml_utils.c        |   15 +
 drivers/ml/cnxk/cnxk_ml_utils.h        |   17 +
 drivers/ml/cnxk/cnxk_ml_xstats.h       |  152 ++
 drivers/ml/cnxk/meson.build            |   70 +
 drivers/ml/cnxk/mvtvm_ml_dev.c         |  196 ++
 drivers/ml/cnxk/mvtvm_ml_dev.h         |   40 +
 drivers/ml/cnxk/mvtvm_ml_model.c       |  409 ++++
 drivers/ml/cnxk/mvtvm_ml_model.h       |   90 +
 drivers/ml/cnxk/mvtvm_ml_ops.c         |  652 ++++++
 drivers/ml/cnxk/mvtvm_ml_ops.h         |   82 +
 drivers/ml/cnxk/mvtvm_ml_stubs.c       |  141 ++
 drivers/ml/cnxk/mvtvm_ml_stubs.h       |   36 +
 32 files changed, 6298 insertions(+), 2959 deletions(-)
 create mode 100644 drivers/ml/cnxk/cnxk_ml_dev.c
 create mode 100644 drivers/ml/cnxk/cnxk_ml_dev.h
 create mode 100644 drivers/ml/cnxk/cnxk_ml_io.c
 create mode 100644 drivers/ml/cnxk/cnxk_ml_io.h
 create mode 100644 drivers/ml/cnxk/cnxk_ml_model.c
 create mode 100644 drivers/ml/cnxk/cnxk_ml_model.h
 create mode 100644 drivers/ml/cnxk/cnxk_ml_ops.c
 create mode 100644 drivers/ml/cnxk/cnxk_ml_ops.h
 create mode 100644 drivers/ml/cnxk/cnxk_ml_utils.c
 create mode 100644 drivers/ml/cnxk/cnxk_ml_utils.h
 create mode 100644 drivers/ml/cnxk/cnxk_ml_xstats.h
 create mode 100644 drivers/ml/cnxk/mvtvm_ml_dev.c
 create mode 100644 drivers/ml/cnxk/mvtvm_ml_dev.h
 create mode 100644 drivers/ml/cnxk/mvtvm_ml_model.c
 create mode 100644 drivers/ml/cnxk/mvtvm_ml_model.h
 create mode 100644 drivers/ml/cnxk/mvtvm_ml_ops.c
 create mode 100644 drivers/ml/cnxk/mvtvm_ml_ops.h
 create mode 100644 drivers/ml/cnxk/mvtvm_ml_stubs.c
 create mode 100644 drivers/ml/cnxk/mvtvm_ml_stubs.h
  

On Fri, Oct 27, 2023 at 12:12 AM Srikanth Yalavarthi
<syalavarthi@marvell.com> wrote:
>
> This patch series is an implementation of revised ml/cnxk driver
> to support models compiled with TVM compiler framework. TVM models
> use a hybrid mode for execution, with regions of the model executing
> on the ML accelerator and the rest executing on CPU cores.
>
> This series of commits reorganizes the ml/cnxk driver and adds support
> to execute multiple regions with-in a TVM model.
>
> v9:
>   - Fixed incorrect IO layout for TVM Marvell models
>   - Set byte offset to zero for I/O tensors
>   - Updated max layers macro definition. Set to TVMDP max layers.
>   - Fixed TVM model IO type to RTE IO type map

Series applied to dpdk-next-net-mrvl/for-next-net with following chages. Thanks.

1) cnxl ml driver update in doc/guides/rel_notes/release_23_11.rst
moved to close to mldev subsystem changes
2)
[for-next-net]dell[dpdk-next-net-mrvl] $ git diff
diff --git a/doc/guides/mldevs/cnxk.rst b/doc/guides/mldevs/cnxk.rst
index 28e5b5b87f..25e8ff783a 100644
--- a/doc/guides/mldevs/cnxk.rst
+++ b/doc/guides/mldevs/cnxk.rst
@@ -212,9 +212,9 @@ not part of DPDK and must be installed separately:
 .. note::

     In order for meson to find the dependencies during the configure stage,
-    it is required to add the cmake paths <install_prefix>/lib/cmake/dlpack,
-    <install_prefix>/lib/cmake/dmlc and <install_prefix>/lib/cmake/tvm to
-    CMAKE_PREFIX_PATH and <install_prefix>/lib/pkgconfig to PKG_CONFIG_PATH.
+    it is required to update CMAKE_PREFIX_PATH and PKG_CONFIG_PATH as below.
+    CMAKE_PREFIX_PATH='<install_prefix>/lib/cmake/tvm:<install_prefix>/lib/cmake/dlpack:<install_prefix>/lib/cmake/dmlc'
+    PKG_CONFIG_PATH='<install_prefix>/lib/pkgconfig'


For the record, I have used following build command to test code with
external build dep.

Assumung all depended libraries installed at /export/cross_ml/install/

CMAKE_PREFIX_PATH='/export/cross_ml/install/lib/cmake/tvm:/export/cross_ml/install/lib/cmake/dlpack:/export/cross_ml/install/lib/cmake/dmlc'
PKG_CONFIG_PATH='/export/cross_ml/install/lib/pkgconfig/' meson setup
 --cross config/arm/arm64_cn10k_linux_gcc  -Denable_docs=true
-Dexamples=all -Dc_args='-I/export/cross_ml/install/include'
-Dc_link_args='-L/export/cross_ml/install/lib' build

Also, -Dc_args='-I/export/cross_ml/install/include'
-Dc_link_args='-L/export/cross_ml/install/lib' could be removed when
following patch merged through main tree.
https://patches.dpdk.org/project/dpdk/patch/20231029082004.5576-1-syalavarthi@marvell.com/