[v2,3/3] compress/uadk: support burst enqueue/dequeue

Message ID 20240525140753.122-4-zhangfei.gao@linaro.org (mailing list archive)
State Superseded
Delegated to: akhil goyal
Headers
Series Introduce UADK compression driver |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/loongarch-compilation success Compilation OK
ci/loongarch-unit-testing success Unit Testing PASS
ci/iol-mellanox-Performance success Performance Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/github-robot: build success github build: passed
ci/iol-intel-Functional success Functional Testing PASS
ci/iol-abi-testing success Testing PASS
ci/iol-compile-amd64-testing success Testing PASS
ci/iol-unit-amd64-testing success Testing PASS
ci/iol-sample-apps-testing success Testing PASS
ci/iol-unit-arm64-testing success Testing PASS
ci/iol-compile-arm64-testing success Testing PASS
ci/Intel-compilation success Compilation OK
ci/intel-Testing success Testing PASS
ci/intel-Functional success Functional PASS
ci/iol-broadcom-Performance success Performance Testing PASS
ci/iol-broadcom-Functional success Functional Testing PASS

Commit Message

Zhangfei Gao May 25, 2024, 2:07 p.m. UTC
  This commit adds the burst enqueue and dequeue operations,
and adds compressdev uadk info to doc

Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
---
 MAINTAINERS                               |  6 ++
 doc/guides/compressdevs/index.rst         |  1 +
 doc/guides/compressdevs/uadk.rst          | 98 +++++++++++++++++++++++
 doc/guides/rel_notes/release_24_07.rst    |  5 ++
 drivers/compress/uadk/uadk_compress_pmd.c | 88 +++++++++++++++++++-
 5 files changed, 196 insertions(+), 2 deletions(-)
 create mode 100644 doc/guides/compressdevs/uadk.rst
  

Comments

Akhil Goyal June 7, 2024, 11:49 a.m. UTC | #1
> This commit adds the burst enqueue and dequeue operations,
> and adds compressdev uadk info to doc
> 
> Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
> ---
>  MAINTAINERS                               |  6 ++
>  doc/guides/compressdevs/index.rst         |  1 +
>  doc/guides/compressdevs/uadk.rst          | 98 +++++++++++++++++++++++
>  doc/guides/rel_notes/release_24_07.rst    |  5 ++
>  drivers/compress/uadk/uadk_compress_pmd.c | 88 +++++++++++++++++++-
>  5 files changed, 196 insertions(+), 2 deletions(-)
>  create mode 100644 doc/guides/compressdevs/uadk.rst
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index c9adff9846..8e541d23e5 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1213,6 +1213,12 @@ F: drivers/compress/octeontx/
>  F: doc/guides/compressdevs/octeontx.rst
>  F: doc/guides/compressdevs/features/octeontx.ini
> 
> +HiSilicon UADK compress
> +M: Zhangfei Gao <zhangfei.gao@linaro.org>
> +F: drivers/compress/uadk/

Above 3 lines should be part of patch 1 as compress/uadk is added in that.


> +F: doc/guides/compressdevs/uadk.rst
> +F: doc/guides/compressdevs/features/uadk.ini

uadk.ini file is missing in the patch.
  
Zhangfei Gao June 7, 2024, 1:54 p.m. UTC | #2
Hi, Akhil

On Fri, 7 Jun 2024 at 19:49, Akhil Goyal <gakhil@marvell.com> wrote:
>
> > This commit adds the burst enqueue and dequeue operations,
> > and adds compressdev uadk info to doc
> >
> > Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
> > ---
> >  MAINTAINERS                               |  6 ++
> >  doc/guides/compressdevs/index.rst         |  1 +
> >  doc/guides/compressdevs/uadk.rst          | 98 +++++++++++++++++++++++
> >  doc/guides/rel_notes/release_24_07.rst    |  5 ++
> >  drivers/compress/uadk/uadk_compress_pmd.c | 88 +++++++++++++++++++-
> >  5 files changed, 196 insertions(+), 2 deletions(-)
> >  create mode 100644 doc/guides/compressdevs/uadk.rst
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index c9adff9846..8e541d23e5 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -1213,6 +1213,12 @@ F: drivers/compress/octeontx/
> >  F: doc/guides/compressdevs/octeontx.rst
> >  F: doc/guides/compressdevs/features/octeontx.ini
> >
> > +HiSilicon UADK compress
> > +M: Zhangfei Gao <zhangfei.gao@linaro.org>
> > +F: drivers/compress/uadk/
>
> Above 3 lines should be part of patch 1 as compress/uadk is added in that.

How about

patch 1:
 MAINTAINERS
 doc/guides/compressdevs/index.rst
 doc/guides/compressdevs/uadk.rst
 doc/guides/compressdevs/features/uadk.ini

Last patch3
doc/guides/rel_notes/release_24_07.rst


>
>
> > +F: doc/guides/compressdevs/uadk.rst
> > +F: doc/guides/compressdevs/features/uadk.ini
>
> uadk.ini file is missing in the patch.
Oops, sorry.

Thanks
>
  
Akhil Goyal June 10, 2024, 6:27 a.m. UTC | #3
> -----Original Message-----
> From: Zhangfei Gao <zhangfei.gao@linaro.org>
> Sent: Friday, June 7, 2024 7:24 PM
> To: Akhil Goyal <gakhil@marvell.com>
> Cc: Fan Zhang <fanzhang.oss@gmail.com>; Ashish Gupta
> <ashishg@marvell.com>; dev@dpdk.org
> Subject: Re: [EXTERNAL] [PATCH v2 3/3] compress/uadk: support burst
> enqueue/dequeue
> 
> Hi, Akhil
> 
> On Fri, 7 Jun 2024 at 19:49, Akhil Goyal <gakhil@marvell.com> wrote:
> >
> > > This commit adds the burst enqueue and dequeue operations,
> > > and adds compressdev uadk info to doc
> > >
> > > Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
> > > ---
> > >  MAINTAINERS                               |  6 ++
> > >  doc/guides/compressdevs/index.rst         |  1 +
> > >  doc/guides/compressdevs/uadk.rst          | 98 +++++++++++++++++++++++
> > >  doc/guides/rel_notes/release_24_07.rst    |  5 ++
> > >  drivers/compress/uadk/uadk_compress_pmd.c | 88 +++++++++++++++++++-
> > >  5 files changed, 196 insertions(+), 2 deletions(-)
> > >  create mode 100644 doc/guides/compressdevs/uadk.rst
> > >
> > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > index c9adff9846..8e541d23e5 100644
> > > --- a/MAINTAINERS
> > > +++ b/MAINTAINERS
> > > @@ -1213,6 +1213,12 @@ F: drivers/compress/octeontx/
> > >  F: doc/guides/compressdevs/octeontx.rst
> > >  F: doc/guides/compressdevs/features/octeontx.ini
> > >
> > > +HiSilicon UADK compress
> > > +M: Zhangfei Gao <zhangfei.gao@linaro.org>
> > > +F: drivers/compress/uadk/
> >
> > Above 3 lines should be part of patch 1 as compress/uadk is added in that.
> 
> How about
> 
> patch 1:
>  MAINTAINERS
>  doc/guides/compressdevs/index.rst
>  doc/guides/compressdevs/uadk.rst
>  doc/guides/compressdevs/features/uadk.ini
> 
> Last patch3
> doc/guides/rel_notes/release_24_07.rst

You can do that. But your implementation is completed in patch 3.
Hence details about test etc will need to be in patch 3.
  

Patch

diff --git a/MAINTAINERS b/MAINTAINERS
index c9adff9846..8e541d23e5 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1213,6 +1213,12 @@  F: drivers/compress/octeontx/
 F: doc/guides/compressdevs/octeontx.rst
 F: doc/guides/compressdevs/features/octeontx.ini
 
+HiSilicon UADK compress
+M: Zhangfei Gao <zhangfei.gao@linaro.org>
+F: drivers/compress/uadk/
+F: doc/guides/compressdevs/uadk.rst
+F: doc/guides/compressdevs/features/uadk.ini
+
 Intel QuickAssist
 M: Kai Ji <kai.ji@intel.com>
 F: drivers/compress/qat/
diff --git a/doc/guides/compressdevs/index.rst b/doc/guides/compressdevs/index.rst
index 849f211688..87ed4f72a4 100644
--- a/doc/guides/compressdevs/index.rst
+++ b/doc/guides/compressdevs/index.rst
@@ -15,4 +15,5 @@  Compression Device Drivers
     nitrox
     octeontx
     qat_comp
+    uadk
     zlib
diff --git a/doc/guides/compressdevs/uadk.rst b/doc/guides/compressdevs/uadk.rst
new file mode 100644
index 0000000000..7e7f9f2548
--- /dev/null
+++ b/doc/guides/compressdevs/uadk.rst
@@ -0,0 +1,98 @@ 
+.. SPDX-License-Identifier: BSD-3-Clause
+   Copyright 2024-2025 Huawei Technologies Co.,Ltd. All rights reserved.
+   Copyright 2024-2025 Linaro ltd.
+
+UADK Compression Poll Mode Driver
+=================================
+
+UADK compression PMD provides poll mode compression & decompression driver
+All compression operations are using UADK library compression API, which is
+algorithm-level API, abstracting accelerators' low-level implementations.
+
+UADK compression PMD relies on `UADK library <https://github.com/Linaro/uadk>`_.
+
+UADK is a framework for user applications to access hardware accelerators.
+UADK relies on IOMMU SVA (Shared Virtual Address) feature,
+which shares the same page table between IOMMU and MMU.
+As a result, the user application can directly use the virtual address for
+device DMA, which enhances performance as well as easy usability.
+
+
+Features
+--------
+
+UADK compression PMD has support for:
+
+Compression/Decompression algorithm:
+
+    * DEFLATE - using Fixed and Dynamic Huffman encoding
+
+Window size support:
+
+    * 32K
+
+
+Test steps
+----------
+
+#. Build UADK
+
+   .. code-block:: console
+
+      git clone https://github.com/Linaro/uadk.git
+      cd uadk
+      mkdir build
+      ./autogen.sh
+      ./configure --prefix=$PWD/build
+      make
+      make install
+
+   .. note::
+
+      Without ``--prefix``, UADK will be installed to ``/usr/local/lib`` by default.
+
+   .. note::
+
+      If get error: "cannot find -lnuma", please install the libnuma-dev.
+
+#. Run pkg-config libwd to ensure env is setup correctly
+
+   .. code-block:: console
+
+      export PKG_CONFIG_PATH=$PWD/build/lib/pkgconfig
+      pkg-config libwd --cflags --libs -I/usr/local/include -L/usr/local/lib -lwd
+
+   .. note::
+
+      export ``PKG_CONFIG_PATH`` is required on demand,
+      not needed if UADK is installed to ``/usr/local/lib``.
+
+#. Build DPDK
+
+   .. code-block:: console
+
+      cd dpdk
+      mkdir build
+      meson setup build (--reconfigure)
+      cd build
+      ninja
+      sudo ninja install
+
+#. Prepare hugepages for DPDK (see also :doc:`../tools/hugepages`)
+
+   .. code-block:: console
+
+      echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
+      echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
+      echo 1024 > /sys/devices/system/node/node2/hugepages/hugepages-2048kB/nr_hugepages
+      echo 1024 > /sys/devices/system/node/node3/hugepages/hugepages-2048kB/nr_hugepages
+      mkdir -p /mnt/huge_2mb
+      mount -t hugetlbfs none /mnt/huge_2mb -o pagesize=2MB
+
+#. Run test app
+
+   .. code-block:: console
+
+	sudo dpdk-test --vdev=compress_uadk
+	RTE>>compressdev_autotest
+	RTE>>quit
diff --git a/doc/guides/rel_notes/release_24_07.rst b/doc/guides/rel_notes/release_24_07.rst
index a69f24cf99..a7574045f0 100644
--- a/doc/guides/rel_notes/release_24_07.rst
+++ b/doc/guides/rel_notes/release_24_07.rst
@@ -24,6 +24,11 @@  DPDK Release 24.07
 New Features
 ------------
 
+* **Added UADK compress driver.**
+
+  Added a new compress driver for the UADK library. See the
+  :doc:`../compressdevs/uadk` guide for more details on this new driver.
+
 .. This section should contain new features added in this release.
    Sample format:
 
diff --git a/drivers/compress/uadk/uadk_compress_pmd.c b/drivers/compress/uadk/uadk_compress_pmd.c
index 9e2dc3e308..1f4c4cfd00 100644
--- a/drivers/compress/uadk/uadk_compress_pmd.c
+++ b/drivers/compress/uadk/uadk_compress_pmd.c
@@ -291,6 +291,90 @@  static struct rte_compressdev_ops uadk_compress_pmd_ops = {
 		.private_xform_free	= uadk_compress_pmd_xform_free,
 };
 
+static uint16_t
+uadk_compress_pmd_enqueue_burst_sync(void *queue_pair,
+				     struct rte_comp_op **ops, uint16_t nb_ops)
+{
+	struct uadk_compress_qp *qp = queue_pair;
+	struct uadk_compress_xform *xform;
+	struct rte_comp_op *op;
+	uint16_t enqd = 0;
+	int i, ret = 0;
+
+	for (i = 0; i < nb_ops; i++) {
+		op = ops[i];
+
+		if (op->op_type == RTE_COMP_OP_STATEFUL) {
+			op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+		} else {
+			/* process stateless ops */
+			xform = op->private_xform;
+			if (xform) {
+				struct wd_comp_req req = {0};
+				uint16_t dst_len = rte_pktmbuf_data_len(op->m_dst);
+
+				req.src = rte_pktmbuf_mtod(op->m_src, uint8_t *);
+				req.src_len = op->src.length;
+				req.dst = rte_pktmbuf_mtod(op->m_dst, uint8_t *);
+				req.dst_len = dst_len;
+				req.op_type = (enum wd_comp_op_type)xform->type;
+				req.cb = NULL;
+				req.data_fmt = WD_FLAT_BUF;
+				do {
+					ret = wd_do_comp_sync(xform->handle, &req);
+				} while (ret == -WD_EBUSY);
+
+				op->consumed += req.src_len;
+
+				if (req.dst_len <= dst_len) {
+					op->produced += req.dst_len;
+					op->status = RTE_COMP_OP_STATUS_SUCCESS;
+				} else  {
+					op->status = RTE_COMP_OP_STATUS_OUT_OF_SPACE_TERMINATED;
+				}
+
+				if (ret) {
+					op->status = RTE_COMP_OP_STATUS_ERROR;
+					break;
+				}
+			} else {
+				op->status = RTE_COMP_OP_STATUS_INVALID_ARGS;
+			}
+		}
+
+		/* Whatever is out of op, put it into completion queue with
+		 * its status
+		 */
+		if (!ret)
+			ret = rte_ring_enqueue(qp->processed_pkts, (void *)op);
+
+		if (unlikely(ret)) {
+			/* increment count if failed to enqueue op */
+			qp->qp_stats.enqueue_err_count++;
+		} else {
+			qp->qp_stats.enqueued_count++;
+			enqd++;
+		}
+	}
+
+	return enqd;
+}
+
+static uint16_t
+uadk_compress_pmd_dequeue_burst_sync(void *queue_pair,
+				     struct rte_comp_op **ops,
+				     uint16_t nb_ops)
+{
+	struct uadk_compress_qp *qp = queue_pair;
+	unsigned int nb_dequeued = 0;
+
+	nb_dequeued = rte_ring_dequeue_burst(qp->processed_pkts,
+			(void **)ops, nb_ops, NULL);
+	qp->qp_stats.dequeued_count += nb_dequeued;
+
+	return nb_dequeued;
+}
+
 static int
 uadk_compress_probe(struct rte_vdev_device *vdev)
 {
@@ -318,8 +402,8 @@  uadk_compress_probe(struct rte_vdev_device *vdev)
 	}
 
 	compressdev->dev_ops = &uadk_compress_pmd_ops;
-	compressdev->dequeue_burst = NULL;
-	compressdev->enqueue_burst = NULL;
+	compressdev->dequeue_burst = uadk_compress_pmd_dequeue_burst_sync;
+	compressdev->enqueue_burst = uadk_compress_pmd_enqueue_burst_sync;
 	compressdev->feature_flags = RTE_COMPDEV_FF_HW_ACCELERATED;
 
 	return 0;