[v3,2/5] dma/cnxk: create and initialize dmadev on pci probe

Message ID 20211103180150.10416-2-radhac@marvell.com (mailing list archive)
State Accepted, archived
Delegated to: Thomas Monjalon
Headers
Series [v3,1/5] common/cnxk: add DPI DMA support |

Checks

Context Check Description
ci/checkpatch success coding style OK

Commit Message

Radha Chintakuntla Nov. 3, 2021, 6:01 p.m. UTC
  This patch creates and initializes a dmadev device on pci probe.

Signed-off-by: Radha Mohan Chintakuntla <radhac@marvell.com>
---
 MAINTAINERS                    |   6 ++
 doc/guides/dmadevs/cnxk.rst    |  58 +++++++++++++++++++
 doc/guides/dmadevs/index.rst   |   1 +
 doc/guides/platform/cnxk.rst   |   5 +-
 drivers/dma/cnxk/cnxk_dmadev.c | 100 +++++++++++++++++++++++++++++++++
 drivers/dma/cnxk/cnxk_dmadev.h |  11 ++++
 drivers/dma/cnxk/meson.build   |   7 +++
 drivers/dma/meson.build        |   1 +
 8 files changed, 188 insertions(+), 1 deletion(-)
 create mode 100644 doc/guides/dmadevs/cnxk.rst
 create mode 100644 drivers/dma/cnxk/cnxk_dmadev.c
 create mode 100644 drivers/dma/cnxk/cnxk_dmadev.h
 create mode 100644 drivers/dma/cnxk/meson.build
  

Comments

Thomas Monjalon Nov. 7, 2021, 8:55 p.m. UTC | #1
03/11/2021 19:01, Radha Mohan Chintakuntla:
> This patch creates and initializes a dmadev device on pci probe.
> 
> Signed-off-by: Radha Mohan Chintakuntla <radhac@marvell.com>
> ---
>  MAINTAINERS                    |   6 ++
>  doc/guides/dmadevs/cnxk.rst    |  58 +++++++++++++++++++
>  doc/guides/dmadevs/index.rst   |   1 +
>  doc/guides/platform/cnxk.rst   |   5 +-
>  drivers/dma/cnxk/cnxk_dmadev.c | 100 +++++++++++++++++++++++++++++++++
>  drivers/dma/cnxk/cnxk_dmadev.h |  11 ++++
>  drivers/dma/cnxk/meson.build   |   7 +++
>  drivers/dma/meson.build        |   1 +
>  8 files changed, 188 insertions(+), 1 deletion(-)

This patch does not compile because drivers/dma/cnxk/version.map is missing.
  
Thomas Monjalon Nov. 7, 2021, 11:04 p.m. UTC | #2
03/11/2021 19:01, Radha Mohan Chintakuntla:
> This patch creates and initializes a dmadev device on pci probe.
> 
> Signed-off-by: Radha Mohan Chintakuntla <radhac@marvell.com>

Series applied with few small improvements in the doc,
compilation and other more or less important details
like the release notes.

I think you forgot to update dpdk-devbind script.
See also this patch:
https://patches.dpdk.org/project/dpdk/patch/20211107225708.3087968-1-thomas@monjalon.net/

Overall I have doubts about the quality but I want to move forward
with the new DMA drivers.
I will be probably less flexible with the next patches.
  
Radha Mohan Nov. 9, 2021, 3:52 a.m. UTC | #3
On Sun, Nov 7, 2021 at 6:04 PM Thomas Monjalon <thomas@monjalon.net> wrote:
>
> 03/11/2021 19:01, Radha Mohan Chintakuntla:
> > This patch creates and initializes a dmadev device on pci probe.
> >
> > Signed-off-by: Radha Mohan Chintakuntla <radhac@marvell.com>
>
> Series applied with few small improvements in the doc,
> compilation and other more or less important details
> like the release notes.

Thank you.
>
> I think you forgot to update dpdk-devbind script.
> See also this patch:
> https://patches.dpdk.org/project/dpdk/patch/20211107225708.3087968-1-thomas@monjalon.net/

Yes need to remove it from "misc" but also rename octeontx2_dma to
cnxk_dma and add under "dma_devices". I missed this.

I can push a patch which does both or if are ok to update yours then
that would be great.

diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index bb00f43702..643860b3ae 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -41,7 +41,7 @@
                  'SVendor': None, 'SDevice': None}
 octeontx2_npa = {'Class': '08', 'Vendor': '177d', 'Device': 'a0fb,a0fc',
                  'SVendor': None, 'SDevice': None}
-octeontx2_dma = {'Class': '08', 'Vendor': '177d', 'Device': 'a081',
+cnxk_dma = {'Class': '08', 'Vendor': '177d', 'Device': 'a081',
                  'SVendor': None, 'SDevice': None}
 octeontx2_ree = {'Class': '08', 'Vendor': '177d', 'Device': 'a0f4',
                  'SVendor': None, 'SDevice': None}
@@ -71,13 +71,14 @@
 network_devices = [network_class, cavium_pkx, avp_vnic, ifpga_class]
 baseband_devices = [acceleration_class]
 crypto_devices = [encryption_class, intel_processor_class]
-dma_devices = [intel_idxd_spr, intel_ioat_bdw, intel_ioat_icx, intel_ioat_skx]
+dma_devices = [intel_idxd_spr, intel_ioat_bdw, intel_ioat_icx, intel_ioat_skx
+               cnxk_dma]
 eventdev_devices = [cavium_sso, cavium_tim, intel_dlb, octeontx2_sso]
 mempool_devices = [cavium_fpa, octeontx2_npa]
 compress_devices = [cavium_zip]
 regex_devices = [octeontx2_ree]
-misc_devices = [cnxk_bphy, cnxk_bphy_cgx, cnxk_inl_dev, intel_ntb_skx,
-                intel_ntb_icx, octeontx2_dma]
+misc_devices = [cnxk_bphy, cnxk_bphy_cgx, cnxk_inl_dev,
+                intel_ntb_skx, intel_ntb_icx]


>
> Overall I have doubts about the quality but I want to move forward
> with the new DMA drivers.
> I will be probably less flexible with the next patches.

Ok. Please let know if you find any specific ones that needs improvements.
>
>
  
Thomas Monjalon Nov. 9, 2021, 8:11 a.m. UTC | #4
09/11/2021 04:52, Radha Mohan:
> On Sun, Nov 7, 2021 at 6:04 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> >
> > 03/11/2021 19:01, Radha Mohan Chintakuntla:
> > > This patch creates and initializes a dmadev device on pci probe.
> > >
> > > Signed-off-by: Radha Mohan Chintakuntla <radhac@marvell.com>
> >
> > Series applied with few small improvements in the doc,
> > compilation and other more or less important details
> > like the release notes.
> 
> Thank you.
> >
> > I think you forgot to update dpdk-devbind script.
> > See also this patch:
> > https://patches.dpdk.org/project/dpdk/patch/20211107225708.3087968-1-thomas@monjalon.net/
> 
> Yes need to remove it from "misc" but also rename octeontx2_dma to
> cnxk_dma and add under "dma_devices". I missed this.
> 
> I can push a patch which does both or if are ok to update yours then
> that would be great.

I'll wait for your patch.
Please make sure to rebase on the latest branch.
  

Patch

diff --git a/MAINTAINERS b/MAINTAINERS
index be2c9b6815..60560a6a3b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1202,6 +1202,12 @@  M: Conor Walsh <conor.walsh@intel.com>
 F: drivers/dma/ioat/
 F: doc/guides/dmadevs/ioat.rst
 
+Marvell CNXK DPI DMA
+M: Radha Mohan Chintakuntla <radhac@marvell.com>
+M: Veerasenareddy Burru <vburru@marvell.com>
+F: drivers/dma/cnxk/
+F: doc/guides/dmadevs/cnxk.rst
+
 
 RegEx Drivers
 -------------
diff --git a/doc/guides/dmadevs/cnxk.rst b/doc/guides/dmadevs/cnxk.rst
new file mode 100644
index 0000000000..b29bd59a01
--- /dev/null
+++ b/doc/guides/dmadevs/cnxk.rst
@@ -0,0 +1,58 @@ 
+..  SPDX-License-Identifier: BSD-3-Clause
+    Copyright(c) 2021 Marvell International Ltd.
+
+.. include:: <isonum.txt>
+
+CNXK DMA Device Driver
+======================
+
+The ``cnxk`` dmadev driver provides a poll-mode driver (PMD) for Marvell DPI DMA
+Hardware Accelerator block found in OCTEONTX2 and OCTEONTX3 family of SoCs. Each
+DMA queue is exposed as a VF function when SRIOV is enabled.
+
+The block supports following modes of DMA transfers
+
+#. Internal - DMA within SoC DRAM to DRAM
+
+#. Inbound  - Host DRAM to SoC DRAM when SoC is in PCIe Endpoint
+
+#. Outbound - SoC DRAM to Host DRAM when SoC is in PCIe Endpoint
+
+Prerequisites and Compilation procedure
+---------------------------------------
+
+   See :doc:`../platform/cnxk` for setup information.
+
+Device Setup
+-------------
+The ``dpdk-devbind.py`` script, included with DPDK, can be used to show the
+presence of supported hardware. Running ``dpdk-devbind.py --status-dev dma``
+will show all the CNXK DMA devices.
+
+Devices using VFIO drivers
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The HW devices to be used will need to be bound to a user-space IO driver for use.
+The ``dpdk-devbind.py`` script can be used to view the state of the devices
+and to bind them to a suitable DPDK-supported driver, such as ``vfio-pci``.
+For example::
+
+     $ dpdk-devbind.py -b vfio-pci 0000:05:00.1
+
+Device Probing and Initialization
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To use the devices from an application, the dmadev API can be used.
+CNXK DMA device configuration requirements:
+
+* Only one ``vchan`` is supported per device.
+* CNXK DMA devices do not support silent mode.
+
+Once configured, the device can then be made ready for use by calling the
+ ``rte_dma_start()`` API.
+
+Performing Data Copies
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Refer to the :ref:`Enqueue / Dequeue APIs <dmadev_enqueue_dequeue>` section of the dmadev library
+documentation for details on operation enqueue and submission API usage.
diff --git a/doc/guides/dmadevs/index.rst b/doc/guides/dmadevs/index.rst
index 20476039a5..227fa00c68 100644
--- a/doc/guides/dmadevs/index.rst
+++ b/doc/guides/dmadevs/index.rst
@@ -11,5 +11,6 @@  an application through DMA API.
    :maxdepth: 2
    :numbered:
 
+   cnxk
    idxd
    ioat
diff --git a/doc/guides/platform/cnxk.rst b/doc/guides/platform/cnxk.rst
index d7adf43165..770f5e846b 100644
--- a/doc/guides/platform/cnxk.rst
+++ b/doc/guides/platform/cnxk.rst
@@ -62,7 +62,7 @@  DPDK subsystem.
    +---+-----+--------------------------------------------------------------+
    | 7 | LBK | rte_ethdev                                                   |
    +---+-----+--------------------------------------------------------------+
-   | 8 | DPI | rte_rawdev                                                   |
+   | 8 | DPI | rte_dmadev                                                   |
    +---+-----+--------------------------------------------------------------+
    | 9 | SDP | rte_ethdev                                                   |
    +---+-----+--------------------------------------------------------------+
@@ -151,6 +151,9 @@  This section lists dataplane H/W block(s) available in cnxk SoC.
 #. **Baseband PHY Driver**
    See :doc:`../rawdevs/cnxk_bphy` for Baseband PHY driver information.
 
+#. **Dmadev Driver**
+   See :doc:`../dmadevs/cnxk` for DPI Dmadev driver information.
+
 Procedure to Setup Platform
 ---------------------------
 
diff --git a/drivers/dma/cnxk/cnxk_dmadev.c b/drivers/dma/cnxk/cnxk_dmadev.c
new file mode 100644
index 0000000000..268482677d
--- /dev/null
+++ b/drivers/dma/cnxk/cnxk_dmadev.c
@@ -0,0 +1,100 @@ 
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright (C) 2021 Marvell International Ltd.
+ */
+
+#include <string.h>
+#include <unistd.h>
+
+#include <rte_bus.h>
+#include <rte_bus_pci.h>
+#include <rte_common.h>
+#include <rte_eal.h>
+#include <rte_lcore.h>
+#include <rte_mempool.h>
+#include <rte_pci.h>
+#include <rte_dmadev.h>
+#include <rte_dmadev_pmd.h>
+
+#include <roc_api.h>
+#include <cnxk_dmadev.h>
+
+static int
+cnxk_dmadev_probe(struct rte_pci_driver *pci_drv __rte_unused,
+		  struct rte_pci_device *pci_dev)
+{
+	struct cnxk_dpi_vf_s *dpivf = NULL;
+	char name[RTE_DEV_NAME_MAX_LEN];
+	struct rte_dma_dev *dmadev;
+	struct roc_dpi *rdpi = NULL;
+	int rc;
+
+	if (!pci_dev->mem_resource[0].addr)
+		return -ENODEV;
+
+	rc = roc_plt_init();
+	if (rc) {
+		plt_err("Failed to initialize platform model, rc=%d", rc);
+		return rc;
+	}
+	memset(name, 0, sizeof(name));
+	rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+
+	dmadev = rte_dma_pmd_allocate(name, pci_dev->device.numa_node,
+				      sizeof(*dpivf));
+	if (dmadev == NULL) {
+		plt_err("dma device allocation failed for %s", name);
+		return -ENOMEM;
+	}
+
+	dpivf = dmadev->data->dev_private;
+
+	dmadev->device = &pci_dev->device;
+	dmadev->fp_obj->dev_private = dpivf;
+
+	rdpi = &dpivf->rdpi;
+
+	rdpi->pci_dev = pci_dev;
+	rc = roc_dpi_dev_init(rdpi);
+	if (rc < 0)
+		goto err_out_free;
+
+	return 0;
+
+err_out_free:
+	if (dmadev)
+		rte_dma_pmd_release(name);
+
+	return rc;
+}
+
+static int
+cnxk_dmadev_remove(struct rte_pci_device *pci_dev)
+{
+	char name[RTE_DEV_NAME_MAX_LEN];
+
+	memset(name, 0, sizeof(name));
+	rte_pci_device_name(&pci_dev->addr, name, sizeof(name));
+
+	return rte_dma_pmd_release(name);
+}
+
+static const struct rte_pci_id cnxk_dma_pci_map[] = {
+	{
+		RTE_PCI_DEVICE(PCI_VENDOR_ID_CAVIUM,
+			       PCI_DEVID_CNXK_DPI_VF)
+	},
+	{
+		.vendor_id = 0,
+	},
+};
+
+static struct rte_pci_driver cnxk_dmadev = {
+	.id_table  = cnxk_dma_pci_map,
+	.drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA,
+	.probe     = cnxk_dmadev_probe,
+	.remove    = cnxk_dmadev_remove,
+};
+
+RTE_PMD_REGISTER_PCI(cnxk_dmadev_pci_driver, cnxk_dmadev);
+RTE_PMD_REGISTER_PCI_TABLE(cnxk_dmadev_pci_driver, cnxk_dma_pci_map);
+RTE_PMD_REGISTER_KMOD_DEP(cnxk_dmadev_pci_driver, "vfio-pci");
diff --git a/drivers/dma/cnxk/cnxk_dmadev.h b/drivers/dma/cnxk/cnxk_dmadev.h
new file mode 100644
index 0000000000..f573e17bf6
--- /dev/null
+++ b/drivers/dma/cnxk/cnxk_dmadev.h
@@ -0,0 +1,11 @@ 
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2021 Marvell International Ltd.
+ */
+#ifndef CNXK_DMADEV_H
+#define CNXK_DMADEV_H
+
+struct cnxk_dpi_vf_s {
+	struct roc_dpi rdpi;
+};
+
+#endif
diff --git a/drivers/dma/cnxk/meson.build b/drivers/dma/cnxk/meson.build
new file mode 100644
index 0000000000..9489d6e6dc
--- /dev/null
+++ b/drivers/dma/cnxk/meson.build
@@ -0,0 +1,7 @@ 
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(C) 2021 Marvell International Ltd.
+#
+
+deps += ['bus_pci', 'common_cnxk', 'dmadev']
+sources = files('cnxk_dmadev.c')
+headers = files('cnxk_dmadev.h')
diff --git a/drivers/dma/meson.build b/drivers/dma/meson.build
index a69418ce9b..c562c8b429 100644
--- a/drivers/dma/meson.build
+++ b/drivers/dma/meson.build
@@ -2,6 +2,7 @@ 
 # Copyright 2021 HiSilicon Limited
 
 drivers = [
+        'cnxk',
         'idxd',
         'ioat',
         'skeleton',