From patchwork Thu Apr 8 09:50:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90861 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 35FECA0579; Thu, 8 Apr 2021 11:51:08 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2CC3D140FF2; Thu, 8 Apr 2021 11:51:07 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 0DEBF140FC2 for ; Thu, 8 Apr 2021 11:51:05 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1389oMYA026741 for ; Thu, 8 Apr 2021 02:51:02 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=Fgkbeikgom1Q8YbccuH17YcBTCPaK2Gji+VGdyfFnYg=; b=AFap5G0QyWwhZ6g0akch6jdbSqOd+qrMMMGCltgMUoF5OBUiiwgo8LfG2UcqeEtNyPvj fSsnJ7yLGmu/FnGhrPpmS6hKUUs03VYrinJXSmEviEtnXDO5GAZ9+YhDikifFJXu3LqF RawbjEWSSQbFksr2gFsEkvvJINTBGCl31oiVorgQAVPhWd9YYJ9IX1vNj9jf0RM+06Ir vNLikaNpEHeWkRtG3sJGgJW326F1lHbcDUAaXwCh/3pJdGo6NaSZDfLb5jK254+6mN7J suPtOd4WmrzvjL/5FEWt2PP6XCDk7/wMKiy6P7g2zLlzBDNwun3/KV6kWaRIw+FU818v OA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 37shqxjhy1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 08 Apr 2021 02:51:02 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 8 Apr 2021 02:51:01 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 8 Apr 2021 02:51:00 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 29CB03F7043; Thu, 8 Apr 2021 02:50:57 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , , Nithin Dabilpuram Date: Thu, 8 Apr 2021 15:20:39 +0530 Message-ID: <20210408095049.3100322-2-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210408095049.3100322-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210408095049.3100322-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: OL100QD_77xb4seepB3s-sB5VUCMw4u7 X-Proofpoint-GUID: OL100QD_77xb4seepB3s-sB5VUCMw4u7 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-04-08_02:2021-04-08, 2021-04-08 signatures=0 Subject: [dpdk-dev] [PATCH v4 01/11] mempool/cnxk: add build infra and doc X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the meson based build infrastructure for Marvell CNXK mempool driver along with stub implementations for mempool device probe. Also add Marvell CNXK mempool base documentation. Signed-off-by: Pavan Nikhilesh Signed-off-by: Jerin Jacob Signed-off-by: Nithin Dabilpuram Signed-off-by: Ashwin Sekhar T K --- MAINTAINERS | 6 ++ doc/guides/mempool/cnxk.rst | 55 ++++++++++++++++++ doc/guides/mempool/index.rst | 1 + doc/guides/platform/cnxk.rst | 3 + doc/guides/rel_notes/release_21_05.rst | 2 + drivers/mempool/cnxk/cnxk_mempool.c | 78 ++++++++++++++++++++++++++ drivers/mempool/cnxk/meson.build | 13 +++++ drivers/mempool/cnxk/version.map | 3 + drivers/mempool/meson.build | 3 +- 9 files changed, 163 insertions(+), 1 deletion(-) create mode 100644 doc/guides/mempool/cnxk.rst create mode 100644 drivers/mempool/cnxk/cnxk_mempool.c create mode 100644 drivers/mempool/cnxk/meson.build create mode 100644 drivers/mempool/cnxk/version.map diff --git a/MAINTAINERS b/MAINTAINERS index f72ce479e2..aa819bcd16 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -501,6 +501,12 @@ M: Artem V. Andreev M: Andrew Rybchenko F: drivers/mempool/bucket/ +Marvell cnxk +M: Ashwin Sekhar T K +M: Pavan Nikhilesh +F: drivers/mempool/cnxk/ +F: doc/guides/mempool/cnxk.rst + Marvell OCTEON TX2 M: Jerin Jacob M: Nithin Dabilpuram diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst new file mode 100644 index 0000000000..e72a77c361 --- /dev/null +++ b/doc/guides/mempool/cnxk.rst @@ -0,0 +1,55 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2021 Marvell. + +CNXK NPA Mempool Driver +============================ + +The CNXK NPA PMD (**librte_mempool_cnxk**) provides mempool driver support for +the integrated mempool device found in **Marvell OCTEON CN9K/CN10K** SoC family. + +More information about CNXK SoC can be found at `Marvell Official Website +`_. + +Features +-------- + +CNXK NPA PMD supports: + +- Up to 128 NPA LFs +- 1M Pools per LF +- HW mempool manager +- Ethdev Rx buffer allocation in HW to save CPU cycles in the Rx path. +- Ethdev Tx buffer recycling in HW to save CPU cycles in the Tx path. + +Prerequisites and Compilation procedure +--------------------------------------- + + See :doc:`../platform/cnxk` for setup information. + +Pre-Installation Configuration +------------------------------ + + +Debugging Options +~~~~~~~~~~~~~~~~~ + +.. _table_cnxk_mempool_debug_options: + +.. table:: CNXK mempool debug options + + +---+------------+-------------------------------------------------------+ + | # | Component | EAL log command | + +===+============+=======================================================+ + | 1 | NPA | --log-level='pmd\.mempool.cnxk,8' | + +---+------------+-------------------------------------------------------+ + +Standalone mempool device +~~~~~~~~~~~~~~~~~~~~~~~~~ + + The ``usertools/dpdk-devbind.py`` script shall enumerate all the mempool + devices available in the system. In order to avoid, the end user to bind the + mempool device prior to use ethdev and/or eventdev device, the respective + driver configures an NPA LF and attach to the first probed ethdev or eventdev + device. In case, if end user need to run mempool as a standalone device + (without ethdev or eventdev), end user needs to bind a mempool device using + ``usertools/dpdk-devbind.py`` diff --git a/doc/guides/mempool/index.rst b/doc/guides/mempool/index.rst index a0e55467e6..ce53bc1ac7 100644 --- a/doc/guides/mempool/index.rst +++ b/doc/guides/mempool/index.rst @@ -11,6 +11,7 @@ application through the mempool API. :maxdepth: 2 :numbered: + cnxk octeontx octeontx2 ring diff --git a/doc/guides/platform/cnxk.rst b/doc/guides/platform/cnxk.rst index d8fc00df40..489569ef4e 100644 --- a/doc/guides/platform/cnxk.rst +++ b/doc/guides/platform/cnxk.rst @@ -142,6 +142,9 @@ HW Offload Drivers This section lists dataplane H/W block(s) available in CNXK SoC. +#. **Mempool Driver** + See :doc:`../mempool/cnxk` for NPA mempool driver information. + Procedure to Setup Platform --------------------------- diff --git a/doc/guides/rel_notes/release_21_05.rst b/doc/guides/rel_notes/release_21_05.rst index 9ca1cb244c..4255ab3372 100644 --- a/doc/guides/rel_notes/release_21_05.rst +++ b/doc/guides/rel_notes/release_21_05.rst @@ -73,6 +73,8 @@ New Features * Added common/cnxk driver consisting of common API to be used by net, crypto and event PMD's. + * Added mempool/cnxk driver which provides the support for the integrated + mempool device. * **Updated Arkville PMD driver.** diff --git a/drivers/mempool/cnxk/cnxk_mempool.c b/drivers/mempool/cnxk/cnxk_mempool.c new file mode 100644 index 0000000000..947078c052 --- /dev/null +++ b/drivers/mempool/cnxk/cnxk_mempool.c @@ -0,0 +1,78 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "roc_api.h" + +static int +npa_remove(struct rte_pci_device *pci_dev) +{ + RTE_SET_USED(pci_dev); + + return 0; +} + +static int +npa_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) +{ + RTE_SET_USED(pci_drv); + RTE_SET_USED(pci_dev); + + return 0; +} + +static const struct rte_pci_id npa_pci_map[] = { + { + .class_id = RTE_CLASS_ANY_ID, + .vendor_id = PCI_VENDOR_ID_CAVIUM, + .device_id = PCI_DEVID_CNXK_RVU_NPA_PF, + .subsystem_vendor_id = PCI_VENDOR_ID_CAVIUM, + .subsystem_device_id = PCI_SUBSYSTEM_DEVID_CN10KA, + }, + { + .class_id = RTE_CLASS_ANY_ID, + .vendor_id = PCI_VENDOR_ID_CAVIUM, + .device_id = PCI_DEVID_CNXK_RVU_NPA_PF, + .subsystem_vendor_id = PCI_VENDOR_ID_CAVIUM, + .subsystem_device_id = PCI_SUBSYSTEM_DEVID_CN10KAS, + }, + { + .class_id = RTE_CLASS_ANY_ID, + .vendor_id = PCI_VENDOR_ID_CAVIUM, + .device_id = PCI_DEVID_CNXK_RVU_NPA_VF, + .subsystem_vendor_id = PCI_VENDOR_ID_CAVIUM, + .subsystem_device_id = PCI_SUBSYSTEM_DEVID_CN10KA, + }, + { + .class_id = RTE_CLASS_ANY_ID, + .vendor_id = PCI_VENDOR_ID_CAVIUM, + .device_id = PCI_DEVID_CNXK_RVU_NPA_VF, + .subsystem_vendor_id = PCI_VENDOR_ID_CAVIUM, + .subsystem_device_id = PCI_SUBSYSTEM_DEVID_CN10KAS, + }, + { + .vendor_id = 0, + }, +}; + +static struct rte_pci_driver npa_pci = { + .id_table = npa_pci_map, + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_NEED_IOVA_AS_VA, + .probe = npa_probe, + .remove = npa_remove, +}; + +RTE_PMD_REGISTER_PCI(mempool_cnxk, npa_pci); +RTE_PMD_REGISTER_PCI_TABLE(mempool_cnxk, npa_pci_map); +RTE_PMD_REGISTER_KMOD_DEP(mempool_cnxk, "vfio-pci"); diff --git a/drivers/mempool/cnxk/meson.build b/drivers/mempool/cnxk/meson.build new file mode 100644 index 0000000000..0be0802373 --- /dev/null +++ b/drivers/mempool/cnxk/meson.build @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2021 Marvell. +# + +if not is_linux or not dpdk_conf.get('RTE_ARCH_64') + build = false + reason = 'only supported on 64-bit Linux' + subdir_done() +endif + +sources = files('cnxk_mempool.c') + +deps += ['eal', 'mbuf', 'kvargs', 'bus_pci', 'common_cnxk', 'mempool'] diff --git a/drivers/mempool/cnxk/version.map b/drivers/mempool/cnxk/version.map new file mode 100644 index 0000000000..ee80c51721 --- /dev/null +++ b/drivers/mempool/cnxk/version.map @@ -0,0 +1,3 @@ +INTERNAL { + local: *; +}; diff --git a/drivers/mempool/meson.build b/drivers/mempool/meson.build index 4428813dae..a2814c1dfa 100644 --- a/drivers/mempool/meson.build +++ b/drivers/mempool/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2017 Intel Corporation -drivers = ['bucket', 'dpaa', 'dpaa2', 'octeontx', 'octeontx2', 'ring', 'stack'] +drivers = ['bucket', 'cnxk', 'dpaa', 'dpaa2', 'octeontx', 'octeontx2', 'ring', + 'stack'] std_deps = ['mempool'] From patchwork Thu Apr 8 09:50:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90862 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2A6C9A0579; Thu, 8 Apr 2021 11:51:17 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id C09E4141063; Thu, 8 Apr 2021 11:51:09 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id D472014103E for ; Thu, 8 Apr 2021 11:51:07 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1389jJ8h010928 for ; Thu, 8 Apr 2021 02:51:06 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=hFp3F8sTs/agzuR6lgFT2IrFAHA6g8ldQXi8Cl/qULU=; b=NdCQ/eVOxXT79vP/e+9xuZbNTpkObrvBqaHaAFYFlq0qu2OunrFyftO/WPjgXlOVawbu SVNwG0uUZJYr2X2ayygBJ3jMgvVa9C5lr13LsHF4KWl3Rvj49BR+sv9qVXnB74Q2ZKDr mkmqhyaBUDpClBwX0wCTOF6OCZg6e1IgS/pyxPvIL41qvJMlIfqdrBoCuxSjYCi0v35k vr9kxCuX8+5RFrCe9THVkGB+h4cU1GMz7uDkaZNEdmGrwAZqwXR+TYuFYyJ09ZcXjrEn yPYAUXeSiNbo1Ru86YjYeI9ST3ElxX7e1YkpIc1HbXeIfZfqyKTN9xyb36onyVqlJ//y Jg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37swewgf98-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 08 Apr 2021 02:51:06 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 8 Apr 2021 02:51:04 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 8 Apr 2021 02:51:04 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 5A9BC3F7051; Thu, 8 Apr 2021 02:51:01 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Thu, 8 Apr 2021 15:20:40 +0530 Message-ID: <20210408095049.3100322-3-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210408095049.3100322-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210408095049.3100322-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: 08JdinWgY256ktKud86D06SuA-eSAhn6 X-Proofpoint-GUID: 08JdinWgY256ktKud86D06SuA-eSAhn6 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-04-08_02:2021-04-08, 2021-04-08 signatures=0 Subject: [dpdk-dev] [PATCH v4 02/11] mempool/cnxk: add device probe/remove X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the implementation for CNXk mempool device probe and remove. Signed-off-by: Pavan Nikhilesh Signed-off-by: Ashwin Sekhar T K --- doc/guides/mempool/cnxk.rst | 23 +++++ drivers/mempool/cnxk/cnxk_mempool.c | 132 +++++++++++++++++++++++++++- 2 files changed, 151 insertions(+), 4 deletions(-) diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst index e72a77c361..907c19c841 100644 --- a/doc/guides/mempool/cnxk.rst +++ b/doc/guides/mempool/cnxk.rst @@ -30,6 +30,29 @@ Pre-Installation Configuration ------------------------------ +Runtime Config Options +~~~~~~~~~~~~~~~~~~~~~~ + +- ``Maximum number of mempools per application`` (default ``128``) + + The maximum number of mempools per application needs to be configured on + HW during mempool driver initialization. HW can support up to 1M mempools, + Since each mempool costs set of HW resources, the ``max_pools`` ``devargs`` + parameter is being introduced to configure the number of mempools required + for the application. + For example:: + + -a 0002:02:00.0,max_pools=512 + + With the above configuration, the driver will set up only 512 mempools for + the given application to save HW resources. + +.. note:: + + Since this configuration is per application, the end user needs to + provide ``max_pools`` parameter to the first PCIe device probed by the given + application. + Debugging Options ~~~~~~~~~~~~~~~~~ diff --git a/drivers/mempool/cnxk/cnxk_mempool.c b/drivers/mempool/cnxk/cnxk_mempool.c index 947078c052..dd4d74ca05 100644 --- a/drivers/mempool/cnxk/cnxk_mempool.c +++ b/drivers/mempool/cnxk/cnxk_mempool.c @@ -15,21 +15,143 @@ #include "roc_api.h" +#define CNXK_NPA_DEV_NAME RTE_STR(cnxk_npa_dev_) +#define CNXK_NPA_DEV_NAME_LEN (sizeof(CNXK_NPA_DEV_NAME) + PCI_PRI_STR_SIZE) +#define CNXK_NPA_MAX_POOLS_PARAM "max_pools" + +static inline uint32_t +npa_aura_size_to_u32(uint8_t val) +{ + if (val == NPA_AURA_SZ_0) + return 128; + if (val >= NPA_AURA_SZ_MAX) + return BIT_ULL(20); + + return 1 << (val + 6); +} + static int -npa_remove(struct rte_pci_device *pci_dev) +parse_max_pools(const char *key, const char *value, void *extra_args) +{ + RTE_SET_USED(key); + uint32_t val; + + val = atoi(value); + if (val < npa_aura_size_to_u32(NPA_AURA_SZ_128)) + val = 128; + if (val > npa_aura_size_to_u32(NPA_AURA_SZ_1M)) + val = BIT_ULL(20); + + *(uint8_t *)extra_args = rte_log2_u32(val) - 6; + return 0; +} + +static inline uint8_t +parse_aura_size(struct rte_devargs *devargs) +{ + uint8_t aura_sz = NPA_AURA_SZ_128; + struct rte_kvargs *kvlist; + + if (devargs == NULL) + goto exit; + kvlist = rte_kvargs_parse(devargs->args, NULL); + if (kvlist == NULL) + goto exit; + + rte_kvargs_process(kvlist, CNXK_NPA_MAX_POOLS_PARAM, &parse_max_pools, + &aura_sz); + rte_kvargs_free(kvlist); +exit: + return aura_sz; +} + +static inline char * +npa_dev_to_name(struct rte_pci_device *pci_dev, char *name) +{ + snprintf(name, CNXK_NPA_DEV_NAME_LEN, CNXK_NPA_DEV_NAME PCI_PRI_FMT, + pci_dev->addr.domain, pci_dev->addr.bus, pci_dev->addr.devid, + pci_dev->addr.function); + + return name; +} + +static int +npa_init(struct rte_pci_device *pci_dev) { - RTE_SET_USED(pci_dev); + char name[CNXK_NPA_DEV_NAME_LEN]; + const struct rte_memzone *mz; + struct roc_npa *dev; + int rc = -ENOMEM; + + mz = rte_memzone_reserve_aligned(npa_dev_to_name(pci_dev, name), + sizeof(*dev), SOCKET_ID_ANY, 0, + RTE_CACHE_LINE_SIZE); + if (mz == NULL) + goto error; + + dev = mz->addr; + dev->pci_dev = pci_dev; + + roc_idev_npa_maxpools_set(parse_aura_size(pci_dev->device.devargs)); + rc = roc_npa_dev_init(dev); + if (rc) + goto mz_free; + + return 0; + +mz_free: + rte_memzone_free(mz); +error: + plt_err("failed to initialize npa device rc=%d", rc); + return rc; +} + +static int +npa_fini(struct rte_pci_device *pci_dev) +{ + char name[CNXK_NPA_DEV_NAME_LEN]; + const struct rte_memzone *mz; + int rc; + + mz = rte_memzone_lookup(npa_dev_to_name(pci_dev, name)); + if (mz == NULL) + return -EINVAL; + + rc = roc_npa_dev_fini(mz->addr); + if (rc) { + if (rc != -EAGAIN) + plt_err("Failed to remove npa dev, rc=%d", rc); + return rc; + } + rte_memzone_free(mz); return 0; } +static int +npa_remove(struct rte_pci_device *pci_dev) +{ + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + return npa_fini(pci_dev); +} + static int npa_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev) { + int rc; + RTE_SET_USED(pci_drv); - RTE_SET_USED(pci_dev); - return 0; + rc = roc_plt_init(); + if (rc < 0) + return rc; + + if (rte_eal_process_type() != RTE_PROC_PRIMARY) + return 0; + + return npa_init(pci_dev); } static const struct rte_pci_id npa_pci_map[] = { @@ -76,3 +198,5 @@ static struct rte_pci_driver npa_pci = { RTE_PMD_REGISTER_PCI(mempool_cnxk, npa_pci); RTE_PMD_REGISTER_PCI_TABLE(mempool_cnxk, npa_pci_map); RTE_PMD_REGISTER_KMOD_DEP(mempool_cnxk, "vfio-pci"); +RTE_PMD_REGISTER_PARAM_STRING(mempool_cnxk, + CNXK_NPA_MAX_POOLS_PARAM "=<128-1048576>"); From patchwork Thu Apr 8 09:50:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90863 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 66C54A0579; Thu, 8 Apr 2021 11:51:23 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1D7D814107B; Thu, 8 Apr 2021 11:51:12 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 61A4F14107B for ; Thu, 8 Apr 2021 11:51:10 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1389jJ1b010931 for ; Thu, 8 Apr 2021 02:51:09 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=6m2bw8SgR16WsjRk1Rmjs3hOVJ5bVx/JJBtsyFJiX5Y=; b=Xx+gDd9pRgAGnPz1c5BvXfIJoWtBZqjY3ETfjeVTFHGHKhY+WCKHn3+1aJ1UxiLtEWaa FEeRUXpgRSJ2lOHAlL0NACK9oOXdUynLsu95KtI9NjV8tJNrBTiPBHd12yqYtKqxflU3 EQ5sL8SpsqrWjMg7FjbniwkenVP2McFJhiUze+QCbU3thDpSfUgIxG2xR6KG4WNnY2ZZ YKrX3ugG2yatgYzuG92n1rjgl0bTXkuCEV7rM5/PYGphR1E6qqKSgAfXDzhdlbQJekzY MMLrJpbu1w+aV7qzwyALNLxpFBQIpTRawtpCXIHZKbUveLQPqv/p1zfy+rQR+FHcQbNI iQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37swewgf9k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 08 Apr 2021 02:51:09 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 8 Apr 2021 02:51:07 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 8 Apr 2021 02:51:07 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id 87D063F703F; Thu, 8 Apr 2021 02:51:04 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Thu, 8 Apr 2021 15:20:41 +0530 Message-ID: <20210408095049.3100322-4-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210408095049.3100322-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210408095049.3100322-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: P_eHMGR_6mjo9Chj1NN4pFs0o2Ws9yMw X-Proofpoint-GUID: P_eHMGR_6mjo9Chj1NN4pFs0o2Ws9yMw X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-04-08_02:2021-04-08, 2021-04-08 signatures=0 Subject: [dpdk-dev] [PATCH v4 03/11] mempool/cnxk: add generic ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add generic CNXk mempool ops which will enqueue/dequeue from pool one element at a time. Signed-off-by: Pavan Nikhilesh Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cnxk_mempool.h | 26 ++++ drivers/mempool/cnxk/cnxk_mempool_ops.c | 171 ++++++++++++++++++++++++ drivers/mempool/cnxk/meson.build | 3 +- 3 files changed, 199 insertions(+), 1 deletion(-) create mode 100644 drivers/mempool/cnxk/cnxk_mempool.h create mode 100644 drivers/mempool/cnxk/cnxk_mempool_ops.c diff --git a/drivers/mempool/cnxk/cnxk_mempool.h b/drivers/mempool/cnxk/cnxk_mempool.h new file mode 100644 index 0000000000..099b7f6998 --- /dev/null +++ b/drivers/mempool/cnxk/cnxk_mempool.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#ifndef _CNXK_MEMPOOL_H_ +#define _CNXK_MEMPOOL_H_ + +#include + +unsigned int cnxk_mempool_get_count(const struct rte_mempool *mp); +ssize_t cnxk_mempool_calc_mem_size(const struct rte_mempool *mp, + uint32_t obj_num, uint32_t pg_shift, + size_t *min_chunk_size, size_t *align); +int cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, + void *vaddr, rte_iova_t iova, size_t len, + rte_mempool_populate_obj_cb_t *obj_cb, + void *obj_cb_arg); +int cnxk_mempool_alloc(struct rte_mempool *mp); +void cnxk_mempool_free(struct rte_mempool *mp); + +int __rte_hot cnxk_mempool_enq(struct rte_mempool *mp, void *const *obj_table, + unsigned int n); +int __rte_hot cnxk_mempool_deq(struct rte_mempool *mp, void **obj_table, + unsigned int n); + +#endif diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c new file mode 100644 index 0000000000..2ce1816c04 --- /dev/null +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -0,0 +1,171 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include + +#include "roc_api.h" +#include "cnxk_mempool.h" + +int __rte_hot +cnxk_mempool_enq(struct rte_mempool *mp, void *const *obj_table, unsigned int n) +{ + unsigned int index; + + /* Ensure mbuf init changes are written before the free pointers + * are enqueued to the stack. + */ + rte_io_wmb(); + for (index = 0; index < n; index++) + roc_npa_aura_op_free(mp->pool_id, 0, + (uint64_t)obj_table[index]); + + return 0; +} + +int __rte_hot +cnxk_mempool_deq(struct rte_mempool *mp, void **obj_table, unsigned int n) +{ + unsigned int index; + uint64_t obj; + + for (index = 0; index < n; index++, obj_table++) { + int retry = 4; + + /* Retry few times before failing */ + do { + obj = roc_npa_aura_op_alloc(mp->pool_id, 0); + } while (retry-- && (obj == 0)); + + if (obj == 0) { + cnxk_mempool_enq(mp, obj_table - index, index); + return -ENOENT; + } + *obj_table = (void *)obj; + } + + return 0; +} + +unsigned int +cnxk_mempool_get_count(const struct rte_mempool *mp) +{ + return (unsigned int)roc_npa_aura_op_available(mp->pool_id); +} + +ssize_t +cnxk_mempool_calc_mem_size(const struct rte_mempool *mp, uint32_t obj_num, + uint32_t pg_shift, size_t *min_chunk_size, + size_t *align) +{ + size_t total_elt_sz; + + /* Need space for one more obj on each chunk to fulfill + * alignment requirements. + */ + total_elt_sz = mp->header_size + mp->elt_size + mp->trailer_size; + return rte_mempool_op_calc_mem_size_helper( + mp, obj_num, pg_shift, total_elt_sz, min_chunk_size, align); +} + +int +cnxk_mempool_alloc(struct rte_mempool *mp) +{ + uint64_t aura_handle = 0; + struct npa_aura_s aura; + struct npa_pool_s pool; + uint32_t block_count; + size_t block_size; + int rc = -ERANGE; + + block_size = mp->elt_size + mp->header_size + mp->trailer_size; + block_count = mp->size; + if (mp->header_size % ROC_ALIGN != 0) { + plt_err("Header size should be multiple of %dB", ROC_ALIGN); + goto error; + } + + if (block_size % ROC_ALIGN != 0) { + plt_err("Block size should be multiple of %dB", ROC_ALIGN); + goto error; + } + + memset(&aura, 0, sizeof(struct npa_aura_s)); + memset(&pool, 0, sizeof(struct npa_pool_s)); + pool.nat_align = 1; + pool.buf_offset = mp->header_size / ROC_ALIGN; + + /* Use driver specific mp->pool_config to override aura config */ + if (mp->pool_config != NULL) + memcpy(&aura, mp->pool_config, sizeof(struct npa_aura_s)); + + rc = roc_npa_pool_create(&aura_handle, block_size, block_count, &aura, + &pool); + if (rc) { + plt_err("Failed to alloc pool or aura rc=%d", rc); + goto error; + } + + /* Store aura_handle for future queue operations */ + mp->pool_id = aura_handle; + plt_npa_dbg("block_sz=%lu block_count=%d aura_handle=0x%" PRIx64, + block_size, block_count, aura_handle); + + return 0; +error: + return rc; +} + +void +cnxk_mempool_free(struct rte_mempool *mp) +{ + int rc = 0; + + plt_npa_dbg("aura_handle=0x%" PRIx64, mp->pool_id); + rc = roc_npa_pool_destroy(mp->pool_id); + if (rc) + plt_err("Failed to free pool or aura rc=%d", rc); +} + +int +cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, + void *vaddr, rte_iova_t iova, size_t len, + rte_mempool_populate_obj_cb_t *obj_cb, void *obj_cb_arg) +{ + size_t total_elt_sz, off; + int num_elts; + + if (iova == RTE_BAD_IOVA) + return -EINVAL; + + total_elt_sz = mp->header_size + mp->elt_size + mp->trailer_size; + + /* Align object start address to a multiple of total_elt_sz */ + off = total_elt_sz - ((((uintptr_t)vaddr - 1) % total_elt_sz) + 1); + + if (len < off) + return -EINVAL; + + vaddr = (char *)vaddr + off; + iova += off; + len -= off; + num_elts = len / total_elt_sz; + + plt_npa_dbg("iova %" PRIx64 ", aligned iova %" PRIx64 "", iova - off, + iova); + plt_npa_dbg("length %" PRIu64 ", aligned length %" PRIu64 "", + (uint64_t)(len + off), (uint64_t)len); + plt_npa_dbg("element size %" PRIu64 "", (uint64_t)total_elt_sz); + plt_npa_dbg("requested objects %" PRIu64 ", possible objects %" PRIu64 + "", (uint64_t)max_objs, (uint64_t)num_elts); + + roc_npa_aura_op_range_set(mp->pool_id, iova, + iova + num_elts * total_elt_sz); + + if (roc_npa_pool_range_update_check(mp->pool_id) < 0) + return -EBUSY; + + return rte_mempool_op_populate_helper( + mp, RTE_MEMPOOL_POPULATE_F_ALIGN_OBJ, max_objs, vaddr, iova, + len, obj_cb, obj_cb_arg); +} diff --git a/drivers/mempool/cnxk/meson.build b/drivers/mempool/cnxk/meson.build index 0be0802373..52244e728b 100644 --- a/drivers/mempool/cnxk/meson.build +++ b/drivers/mempool/cnxk/meson.build @@ -8,6 +8,7 @@ if not is_linux or not dpdk_conf.get('RTE_ARCH_64') subdir_done() endif -sources = files('cnxk_mempool.c') +sources = files('cnxk_mempool.c', + 'cnxk_mempool_ops.c') deps += ['eal', 'mbuf', 'kvargs', 'bus_pci', 'common_cnxk', 'mempool'] From patchwork Thu Apr 8 09:50:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90864 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 656E4A0579; Thu, 8 Apr 2021 11:51:29 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 48658140FC2; Thu, 8 Apr 2021 11:51:14 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id BB53A141083 for ; Thu, 8 Apr 2021 11:51:13 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1389jbwk011022 for ; Thu, 8 Apr 2021 02:51:13 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=n6sxRHF0qyLReygfA6XpJiN5vRpsGIHobtURGk3feB4=; b=XNPxgTV0RDrs/F8yExomCoAFBKFu8/2d0sRMudysfX+u9MZKMhLwSQJcqpzN8ngjEqRZ /HumCHzq+dc/+R3yqKFjfxteXvGbgJ8sTlfWL9g+Zpa0eqlpZ5Zgg79H1nB7INAxkLvy Z9xyA2TtLRULK5aXiNClKJQYSGU8P2G6HSq1ILcDf8IH6iq2lHWROO4LHgWO46ijJ/I3 NT9HqnKuoXlg3XZZ/NbafiojtZPErp+267P2PH43Yd0lEEbKkj4hlEtoZGs2Y27hDpbG 9ruLIJbSkmTu6ut+p7B7UqzKiyVhM0oPN/hwhnmKnSGNmy3G7b1MKpdFFYyBCH1jugb8 dQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com with ESMTP id 37swewgf9n-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 08 Apr 2021 02:51:13 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 8 Apr 2021 02:51:10 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 8 Apr 2021 02:51:10 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id BACB13F7044; Thu, 8 Apr 2021 02:51:07 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Thu, 8 Apr 2021 15:20:42 +0530 Message-ID: <20210408095049.3100322-5-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210408095049.3100322-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210408095049.3100322-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: BIEFgWU8572sEtXJNwY4eRClMw_dSgEA X-Proofpoint-GUID: BIEFgWU8572sEtXJNwY4eRClMw_dSgEA X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-04-08_02:2021-04-08, 2021-04-08 signatures=0 Subject: [dpdk-dev] [PATCH v4 04/11] mempool/cnxk: register plt init callback X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Register the CNXk mempool plt init callback which will set the appropriate mempool ops to be used for the platform. Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cnxk_mempool_ops.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c index 2ce1816c04..e8f64be76b 100644 --- a/drivers/mempool/cnxk/cnxk_mempool_ops.c +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -2,6 +2,7 @@ * Copyright(C) 2021 Marvell. */ +#include #include #include "roc_api.h" @@ -169,3 +170,17 @@ cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, mp, RTE_MEMPOOL_POPULATE_F_ALIGN_OBJ, max_objs, vaddr, iova, len, obj_cb, obj_cb_arg); } + +static int +cnxk_mempool_plt_init(void) +{ + if (roc_model_is_cn10k() || roc_model_is_cn9k()) + rte_mbuf_set_platform_mempool_ops("cnxk_mempool_ops"); + + return 0; +} + +RTE_INIT(cnxk_mempool_ops_init) +{ + roc_plt_init_cb_register(cnxk_mempool_plt_init); +} From patchwork Thu Apr 8 09:50:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90865 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 22C94A0579; Thu, 8 Apr 2021 11:51:35 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 77D2F141099; Thu, 8 Apr 2021 11:51:16 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 46EB1141086 for ; Thu, 8 Apr 2021 11:51:15 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1389oMYF026741 for ; Thu, 8 Apr 2021 02:51:14 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=HpstXKB+6W62wiqjljt914QJoN0/ZmXdtWfYh1+iAfo=; b=kvSkjPO5fLVrzKAzxrCQIfct01Pp71wvVIMbYF2Hu2hWy/EALM3IduMm/KfnByo1vJ5j f+UvgddUvovcR7698+ae2/6IXtO7My4Q7XQBlUOO/I1NFq32ETFA+QCRqjRpic0K0qkt 67qhKCewGHALMT+zuKzYnuL+MJvVK06EPY3nr3PYaWtWERnffBN8S6pjfYG9xDN6na+p ykHK193TIekHX0uhaqo97htoouBjevk3YDc9wA9lrZC/JELi3Sm9g8hQ4cUeQEPUZBtm 9ojMF1OiKg0LxaT44SC0zBvJvWo3B/OLS5UIi9IhKiSkBuaE6OodbKZdWH4XbOj8e0Vc 0w== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 37shqxjhyv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 08 Apr 2021 02:51:14 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 8 Apr 2021 02:51:13 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 8 Apr 2021 02:51:13 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id B08493F7045; Thu, 8 Apr 2021 02:51:10 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Thu, 8 Apr 2021 15:20:43 +0530 Message-ID: <20210408095049.3100322-6-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210408095049.3100322-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210408095049.3100322-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: KSotBRwlrbNF7tZ_mieInsbdhzoqDyUG X-Proofpoint-GUID: KSotBRwlrbNF7tZ_mieInsbdhzoqDyUG X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-04-08_02:2021-04-08, 2021-04-08 signatures=0 Subject: [dpdk-dev] [PATCH v4 05/11] mempool/cnxk: add cn9k mempool ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Marvell CN9k mempool ops and implement CN9k mempool alloc which makes sure that the element size always occupy odd number of cachelines to ensure even distribution among of elements among L1D cache sets. Signed-off-by: Pavan Nikhilesh Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cn9k_mempool_ops.c | 54 +++++++++++++++++++++++++ drivers/mempool/cnxk/cnxk_mempool_ops.c | 4 +- drivers/mempool/cnxk/meson.build | 3 +- 3 files changed, 59 insertions(+), 2 deletions(-) create mode 100644 drivers/mempool/cnxk/cn9k_mempool_ops.c diff --git a/drivers/mempool/cnxk/cn9k_mempool_ops.c b/drivers/mempool/cnxk/cn9k_mempool_ops.c new file mode 100644 index 0000000000..f5ac163af9 --- /dev/null +++ b/drivers/mempool/cnxk/cn9k_mempool_ops.c @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include + +#include "roc_api.h" +#include "cnxk_mempool.h" + +static int +cn9k_mempool_alloc(struct rte_mempool *mp) +{ + size_t block_size, padding; + + block_size = mp->elt_size + mp->header_size + mp->trailer_size; + /* Align header size to ROC_ALIGN */ + if (mp->header_size % ROC_ALIGN != 0) { + padding = RTE_ALIGN_CEIL(mp->header_size, ROC_ALIGN) - + mp->header_size; + mp->header_size += padding; + block_size += padding; + } + + /* Align block size to ROC_ALIGN */ + if (block_size % ROC_ALIGN != 0) { + padding = RTE_ALIGN_CEIL(block_size, ROC_ALIGN) - block_size; + mp->trailer_size += padding; + block_size += padding; + } + + /* + * Marvell CN9k has 8 sets, 41 ways L1D cache, VA<9:7> bits dictate the + * set selection. Add additional padding to ensure that the element size + * always occupies odd number of cachelines to ensure even distribution + * of elements among L1D cache sets. + */ + padding = ((block_size / ROC_ALIGN) % 2) ? 0 : ROC_ALIGN; + mp->trailer_size += padding; + + return cnxk_mempool_alloc(mp); +} + +static struct rte_mempool_ops cn9k_mempool_ops = { + .name = "cn9k_mempool_ops", + .alloc = cn9k_mempool_alloc, + .free = cnxk_mempool_free, + .enqueue = cnxk_mempool_enq, + .dequeue = cnxk_mempool_deq, + .get_count = cnxk_mempool_get_count, + .calc_mem_size = cnxk_mempool_calc_mem_size, + .populate = cnxk_mempool_populate, +}; + +MEMPOOL_REGISTER_OPS(cn9k_mempool_ops); diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c index e8f64be76b..d8ed37ec1a 100644 --- a/drivers/mempool/cnxk/cnxk_mempool_ops.c +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -174,7 +174,9 @@ cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, static int cnxk_mempool_plt_init(void) { - if (roc_model_is_cn10k() || roc_model_is_cn9k()) + if (roc_model_is_cn9k()) + rte_mbuf_set_platform_mempool_ops("cn9k_mempool_ops"); + else if (roc_model_is_cn10k()) rte_mbuf_set_platform_mempool_ops("cnxk_mempool_ops"); return 0; diff --git a/drivers/mempool/cnxk/meson.build b/drivers/mempool/cnxk/meson.build index 52244e728b..ff31893ff4 100644 --- a/drivers/mempool/cnxk/meson.build +++ b/drivers/mempool/cnxk/meson.build @@ -9,6 +9,7 @@ if not is_linux or not dpdk_conf.get('RTE_ARCH_64') endif sources = files('cnxk_mempool.c', - 'cnxk_mempool_ops.c') + 'cnxk_mempool_ops.c', + 'cn9k_mempool_ops.c') deps += ['eal', 'mbuf', 'kvargs', 'bus_pci', 'common_cnxk', 'mempool'] From patchwork Thu Apr 8 09:50:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90866 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D4E28A0579; Thu, 8 Apr 2021 11:51:40 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A7BA5141044; Thu, 8 Apr 2021 11:51:20 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 82367141058 for ; Thu, 8 Apr 2021 11:51:19 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1389pJBl020773 for ; Thu, 8 Apr 2021 02:51:19 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=mj8J/pQ7kUa3fsAvqWMQnyHPacc5py+6X7c4uteKjLQ=; b=b5xtSVTvD8gXKPG54ETrap85UXqS/tvSwGxnv98i+x5xp3CkMuTU2JX1x1SJg2fSwFwW pUMvbnW9URt2cGy6Q/Z2AI8ebZmlwkgeBHK9WaknHUR1JVdYXFn0EcaAP6xOFPhePc7r 6pct0aVy30eV4tOy7IVM9GVkl7ll02HzT/pZiHYCdNoDTrcLY6c19BbPVZreMHMtdO85 Sl1wfeSael39t5RqLQxM1QyvhUvFaiHIuchQAZQ9MwC6QAOvG1iY2wce4X5aRn+NNYdf BVXVa5JzyFyXnO8CpdRWUHhp44eruUmufpj0keaHA7EhUCd+wjNX8VIjZTELVFVzyGwp OQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37swewgf9x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 08 Apr 2021 02:51:18 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 8 Apr 2021 02:51:16 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 8 Apr 2021 02:51:16 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id A44F83F703F; Thu, 8 Apr 2021 02:51:13 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Thu, 8 Apr 2021 15:20:44 +0530 Message-ID: <20210408095049.3100322-7-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210408095049.3100322-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210408095049.3100322-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: _fFYYNrxjQ2f5DzDBTWoTNMXOcdtTBP8 X-Proofpoint-GUID: _fFYYNrxjQ2f5DzDBTWoTNMXOcdtTBP8 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-04-08_02:2021-04-08, 2021-04-08 signatures=0 Subject: [dpdk-dev] [PATCH v4 06/11] mempool/cnxk: add cn9k optimized mempool enqueue/dequeue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Marvell CN9k mempool enqueue/dequeue. Marvell CN9k supports burst dequeue which allows to dequeue up to 32 pointers using pipelined casp instructions. Signed-off-by: Pavan Nikhilesh Signed-off-by: Ashwin Sekhar T K --- doc/guides/mempool/cnxk.rst | 4 +++ drivers/mempool/cnxk/cn9k_mempool_ops.c | 39 +++++++++++++++++++++++-- 2 files changed, 41 insertions(+), 2 deletions(-) diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst index 907c19c841..f51532b101 100644 --- a/doc/guides/mempool/cnxk.rst +++ b/doc/guides/mempool/cnxk.rst @@ -21,6 +21,10 @@ CNXK NPA PMD supports: - Ethdev Rx buffer allocation in HW to save CPU cycles in the Rx path. - Ethdev Tx buffer recycling in HW to save CPU cycles in the Tx path. +CN9k NPA supports: + +- Burst alloc of up to 32 pointers. + Prerequisites and Compilation procedure --------------------------------------- diff --git a/drivers/mempool/cnxk/cn9k_mempool_ops.c b/drivers/mempool/cnxk/cn9k_mempool_ops.c index f5ac163af9..c0cdba640b 100644 --- a/drivers/mempool/cnxk/cn9k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn9k_mempool_ops.c @@ -7,6 +7,41 @@ #include "roc_api.h" #include "cnxk_mempool.h" +static int __rte_hot +cn9k_mempool_enq(struct rte_mempool *mp, void *const *obj_table, unsigned int n) +{ + /* Ensure mbuf init changes are written before the free pointers + * are enqueued to the stack. + */ + rte_io_wmb(); + roc_npa_aura_op_bulk_free(mp->pool_id, (const uint64_t *)obj_table, n, + 0); + + return 0; +} + +static inline int __rte_hot +cn9k_mempool_deq(struct rte_mempool *mp, void **obj_table, unsigned int n) +{ + unsigned int count; + + count = roc_npa_aura_op_bulk_alloc(mp->pool_id, (uint64_t *)obj_table, + n, 0, 1); + + if (unlikely(count != n)) { + /* If bulk alloc failed to allocate all pointers, try + * allocating remaining pointers with the default alloc + * with retry scheme. + */ + if (cnxk_mempool_deq(mp, &obj_table[count], n - count)) { + cn9k_mempool_enq(mp, obj_table, count); + return -ENOENT; + } + } + + return 0; +} + static int cn9k_mempool_alloc(struct rte_mempool *mp) { @@ -44,8 +79,8 @@ static struct rte_mempool_ops cn9k_mempool_ops = { .name = "cn9k_mempool_ops", .alloc = cn9k_mempool_alloc, .free = cnxk_mempool_free, - .enqueue = cnxk_mempool_enq, - .dequeue = cnxk_mempool_deq, + .enqueue = cn9k_mempool_enq, + .dequeue = cn9k_mempool_deq, .get_count = cnxk_mempool_get_count, .calc_mem_size = cnxk_mempool_calc_mem_size, .populate = cnxk_mempool_populate, From patchwork Thu Apr 8 09:50:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90867 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 910DFA0579; Thu, 8 Apr 2021 11:51:47 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 37A131410A9; Thu, 8 Apr 2021 11:51:24 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 032321410B0 for ; Thu, 8 Apr 2021 11:51:21 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1389pJBm020773 for ; Thu, 8 Apr 2021 02:51:21 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=hbRDLeqKGzlBzSKM8ZPiGEMZqOVSNja5lpHI6Om/QxY=; b=EFmkJsdRySha7AyRnhSf7C7p5H2P/TESwdaRk6Iaws0vK3stJ1z/+GrGPqpOObtSoHDr W2wzRf7umk777yewfMmx7pLm3i1qLQ4Jx/ZGrP01mrXtmAAXpG3fn4tCibdTejoiaB4X VbKy0T6xKiBO28/8+wK0i9+1qlqu9TwIMZIgxzvyhlXxmVIJ9w44ll/IZySM8t1h0S04 jPdQoSIL9CIYWlX8++jL5X5Ls7BZI20VJqvRJH6wABPa8RjVthlXMxvorDgRFtIodAFO w1ClgV6PMZGgyLw35ln+Ug6m3c6/3H4e9zvyToyGSUh3zIrtenFR/vlbJlhwLHX8KWsx xQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37swewgfa4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 08 Apr 2021 02:51:21 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 8 Apr 2021 02:51:19 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 8 Apr 2021 02:51:19 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id CFBDF3F7040; Thu, 8 Apr 2021 02:51:16 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Thu, 8 Apr 2021 15:20:45 +0530 Message-ID: <20210408095049.3100322-8-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210408095049.3100322-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210408095049.3100322-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: AG6Vqab9sFZf8IUd15a-4tJXS27ZI_RV X-Proofpoint-GUID: AG6Vqab9sFZf8IUd15a-4tJXS27ZI_RV X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-04-08_02:2021-04-08, 2021-04-08 signatures=0 Subject: [dpdk-dev] [PATCH v4 07/11] mempool/cnxk: add cn10k mempool ops X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add Marvell CN10k mempool ops and implement CN10k mempool alloc. CN10k has 64 bytes L1D cache line size. Hence the CN10k mempool alloc does not make the element size an odd multiple L1D cache line size as NPA requires the element sizes to be multiples of 128 bytes. Signed-off-by: Ashwin Sekhar T K --- doc/guides/mempool/cnxk.rst | 4 ++ drivers/mempool/cnxk/cn10k_mempool_ops.c | 52 ++++++++++++++++++++++++ drivers/mempool/cnxk/cnxk_mempool_ops.c | 2 +- drivers/mempool/cnxk/meson.build | 3 +- 4 files changed, 59 insertions(+), 2 deletions(-) create mode 100644 drivers/mempool/cnxk/cn10k_mempool_ops.c diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst index f51532b101..783368e690 100644 --- a/doc/guides/mempool/cnxk.rst +++ b/doc/guides/mempool/cnxk.rst @@ -80,3 +80,7 @@ Standalone mempool device device. In case, if end user need to run mempool as a standalone device (without ethdev or eventdev), end user needs to bind a mempool device using ``usertools/dpdk-devbind.py`` + + Example command to run ``mempool_autotest`` test with standalone CN10K NPA device:: + + echo "mempool_autotest" | /app/test/dpdk-test -c 0xf0 --mbuf-pool-ops-name="cn10k_mempool_ops" diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c new file mode 100644 index 0000000000..9b63789006 --- /dev/null +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -0,0 +1,52 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2021 Marvell. + */ + +#include + +#include "roc_api.h" +#include "cnxk_mempool.h" + +static int +cn10k_mempool_alloc(struct rte_mempool *mp) +{ + uint32_t block_size; + size_t padding; + + block_size = mp->elt_size + mp->header_size + mp->trailer_size; + /* Align header size to ROC_ALIGN */ + if (mp->header_size % ROC_ALIGN != 0) { + padding = RTE_ALIGN_CEIL(mp->header_size, ROC_ALIGN) - + mp->header_size; + mp->header_size += padding; + block_size += padding; + } + + /* Align block size to ROC_ALIGN */ + if (block_size % ROC_ALIGN != 0) { + padding = RTE_ALIGN_CEIL(block_size, ROC_ALIGN) - block_size; + mp->trailer_size += padding; + block_size += padding; + } + + return cnxk_mempool_alloc(mp); +} + +static void +cn10k_mempool_free(struct rte_mempool *mp) +{ + cnxk_mempool_free(mp); +} + +static struct rte_mempool_ops cn10k_mempool_ops = { + .name = "cn10k_mempool_ops", + .alloc = cn10k_mempool_alloc, + .free = cn10k_mempool_free, + .enqueue = cnxk_mempool_enq, + .dequeue = cnxk_mempool_deq, + .get_count = cnxk_mempool_get_count, + .calc_mem_size = cnxk_mempool_calc_mem_size, + .populate = cnxk_mempool_populate, +}; + +MEMPOOL_REGISTER_OPS(cn10k_mempool_ops); diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c index d8ed37ec1a..42c02bf14e 100644 --- a/drivers/mempool/cnxk/cnxk_mempool_ops.c +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -177,7 +177,7 @@ cnxk_mempool_plt_init(void) if (roc_model_is_cn9k()) rte_mbuf_set_platform_mempool_ops("cn9k_mempool_ops"); else if (roc_model_is_cn10k()) - rte_mbuf_set_platform_mempool_ops("cnxk_mempool_ops"); + rte_mbuf_set_platform_mempool_ops("cn10k_mempool_ops"); return 0; } diff --git a/drivers/mempool/cnxk/meson.build b/drivers/mempool/cnxk/meson.build index ff31893ff4..3282b5e5a6 100644 --- a/drivers/mempool/cnxk/meson.build +++ b/drivers/mempool/cnxk/meson.build @@ -10,6 +10,7 @@ endif sources = files('cnxk_mempool.c', 'cnxk_mempool_ops.c', - 'cn9k_mempool_ops.c') + 'cn9k_mempool_ops.c', + 'cn10k_mempool_ops.c') deps += ['eal', 'mbuf', 'kvargs', 'bus_pci', 'common_cnxk', 'mempool'] From patchwork Thu Apr 8 09:50:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90868 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2454FA0579; Thu, 8 Apr 2021 11:51:53 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 638F0141080; Thu, 8 Apr 2021 11:51:27 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id E035814106D for ; Thu, 8 Apr 2021 11:51:25 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1389pLj5020807 for ; Thu, 8 Apr 2021 02:51:24 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=gjRglxpZZtZ3Wnhw0WdVxE7hcJr6gVU2JmmEwXsrAuE=; b=fhbUsbftpHxxvX99OcCvel9enUDahekPcCB2hX9OACifX2BdnujUepv9lkbEVQuRomTW Htuz6uR7Y18DoSPZhJfnsxF51OFodgG7wAzdSloLffmXjDBKBvePDxfiA49XtW/YJgvb lc+JMbxZTibD4Rs5AO8iZ+eOWcCXvQ+1bPPfsVLB8Bo7BKqQEYQLk4dsBrgj/RBfue9X yWFfMhtb/5JAWSuAcmjXb8iZ30keWDnEmICGAd5wpY0JBxhZzL6ba97McxZPeM26NEWP vC4XANdMg2sazF9dA2suDxr1cc1vBNRSlJfwaSG7VFtAU+ih71trawj6WtgNqw9/p++3 9Q== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37swewgfac-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 08 Apr 2021 02:51:24 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 8 Apr 2021 02:51:22 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 8 Apr 2021 02:51:22 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id C4B543F7043; Thu, 8 Apr 2021 02:51:19 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Thu, 8 Apr 2021 15:20:46 +0530 Message-ID: <20210408095049.3100322-9-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210408095049.3100322-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210408095049.3100322-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: Wc9wbCYqByt9ZNnnjX7eyxBiXkk9fza7 X-Proofpoint-GUID: Wc9wbCYqByt9ZNnnjX7eyxBiXkk9fza7 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-04-08_02:2021-04-08, 2021-04-08 signatures=0 Subject: [dpdk-dev] [PATCH v4 08/11] mempool/cnxk: add batch op init X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Marvell CN10k mempool supports batch enqueue/dequeue which can dequeue up to 512 pointers and enqueue up to 15 pointers using a single instruction. These batch operations require a DMA memory to enqueue/dequeue pointers. This patch adds the initialization of this DMA memory. Signed-off-by: Ashwin Sekhar T K --- doc/guides/mempool/cnxk.rst | 5 + drivers/mempool/cnxk/cn10k_mempool_ops.c | 147 ++++++++++++++++++++++- drivers/mempool/cnxk/cnxk_mempool.h | 2 + drivers/mempool/cnxk/cnxk_mempool_ops.c | 11 +- 4 files changed, 160 insertions(+), 5 deletions(-) diff --git a/doc/guides/mempool/cnxk.rst b/doc/guides/mempool/cnxk.rst index 783368e690..286ee29003 100644 --- a/doc/guides/mempool/cnxk.rst +++ b/doc/guides/mempool/cnxk.rst @@ -25,6 +25,11 @@ CN9k NPA supports: - Burst alloc of up to 32 pointers. +CN10k NPA supports: + +- Batch dequeue of up to 512 pointers with single instruction. +- Batch enqueue of up to 15 pointers with single instruction. + Prerequisites and Compilation procedure --------------------------------------- diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c index 9b63789006..a3aef0ddb2 100644 --- a/drivers/mempool/cnxk/cn10k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -7,11 +7,136 @@ #include "roc_api.h" #include "cnxk_mempool.h" +#define BATCH_ALLOC_SZ ROC_CN10K_NPA_BATCH_ALLOC_MAX_PTRS +#define BATCH_OP_DATA_TABLE_MZ_NAME "batch_op_data_table_mz" + +enum batch_op_status { + BATCH_ALLOC_OP_NOT_ISSUED = 0, + BATCH_ALLOC_OP_ISSUED = 1, + BATCH_ALLOC_OP_DONE +}; + +struct batch_op_mem { + unsigned int sz; + enum batch_op_status status; + uint64_t objs[BATCH_ALLOC_SZ] __rte_aligned(ROC_ALIGN); +}; + +struct batch_op_data { + uint64_t lmt_addr; + struct batch_op_mem mem[RTE_MAX_LCORE] __rte_aligned(ROC_ALIGN); +}; + +static struct batch_op_data **batch_op_data_tbl; + +static int +batch_op_data_table_create(void) +{ + const struct rte_memzone *mz; + + /* If table is already set, nothing to do */ + if (batch_op_data_tbl) + return 0; + + mz = rte_memzone_lookup(BATCH_OP_DATA_TABLE_MZ_NAME); + if (mz == NULL) { + if (rte_eal_process_type() == RTE_PROC_PRIMARY) { + unsigned int maxpools, sz; + + maxpools = roc_idev_npa_maxpools_get(); + sz = maxpools * sizeof(struct batch_op_data *); + + mz = rte_memzone_reserve_aligned( + BATCH_OP_DATA_TABLE_MZ_NAME, sz, SOCKET_ID_ANY, + 0, ROC_ALIGN); + } + if (mz == NULL) { + plt_err("Failed to reserve batch op data table"); + return -ENOMEM; + } + } + batch_op_data_tbl = mz->addr; + rte_wmb(); + return 0; +} + +static inline struct batch_op_data * +batch_op_data_get(uint64_t pool_id) +{ + uint64_t aura = roc_npa_aura_handle_to_aura(pool_id); + + return batch_op_data_tbl[aura]; +} + +static inline void +batch_op_data_set(uint64_t pool_id, struct batch_op_data *op_data) +{ + uint64_t aura = roc_npa_aura_handle_to_aura(pool_id); + + batch_op_data_tbl[aura] = op_data; +} + +static int +batch_op_init(struct rte_mempool *mp) +{ + struct batch_op_data *op_data; + int i; + + op_data = batch_op_data_get(mp->pool_id); + /* The data should not have been allocated previously */ + RTE_ASSERT(op_data == NULL); + + op_data = rte_zmalloc(NULL, sizeof(struct batch_op_data), ROC_ALIGN); + if (op_data == NULL) + return -ENOMEM; + + for (i = 0; i < RTE_MAX_LCORE; i++) { + op_data->mem[i].sz = 0; + op_data->mem[i].status = BATCH_ALLOC_OP_NOT_ISSUED; + } + + op_data->lmt_addr = roc_idev_lmt_base_addr_get(); + batch_op_data_set(mp->pool_id, op_data); + rte_wmb(); + + return 0; +} + +static void +batch_op_fini(struct rte_mempool *mp) +{ + struct batch_op_data *op_data; + int i; + + op_data = batch_op_data_get(mp->pool_id); + + rte_wmb(); + for (i = 0; i < RTE_MAX_LCORE; i++) { + struct batch_op_mem *mem = &op_data->mem[i]; + + if (mem->status == BATCH_ALLOC_OP_ISSUED) { + mem->sz = roc_npa_aura_batch_alloc_extract( + mem->objs, mem->objs, BATCH_ALLOC_SZ); + mem->status = BATCH_ALLOC_OP_DONE; + } + if (mem->status == BATCH_ALLOC_OP_DONE) { + roc_npa_aura_op_bulk_free(mp->pool_id, mem->objs, + mem->sz, 1); + mem->status = BATCH_ALLOC_OP_NOT_ISSUED; + } + } + + rte_free(op_data); + batch_op_data_set(mp->pool_id, NULL); + rte_wmb(); +} + static int cn10k_mempool_alloc(struct rte_mempool *mp) { uint32_t block_size; size_t padding; + int rc; block_size = mp->elt_size + mp->header_size + mp->trailer_size; /* Align header size to ROC_ALIGN */ @@ -29,15 +154,35 @@ cn10k_mempool_alloc(struct rte_mempool *mp) block_size += padding; } - return cnxk_mempool_alloc(mp); + rc = cnxk_mempool_alloc(mp); + if (rc) + return rc; + + rc = batch_op_init(mp); + if (rc) { + plt_err("Failed to init batch alloc mem rc=%d", rc); + goto error; + } + + return 0; +error: + cnxk_mempool_free(mp); + return rc; } static void cn10k_mempool_free(struct rte_mempool *mp) { + batch_op_fini(mp); cnxk_mempool_free(mp); } +int +cn10k_mempool_plt_init(void) +{ + return batch_op_data_table_create(); +} + static struct rte_mempool_ops cn10k_mempool_ops = { .name = "cn10k_mempool_ops", .alloc = cn10k_mempool_alloc, diff --git a/drivers/mempool/cnxk/cnxk_mempool.h b/drivers/mempool/cnxk/cnxk_mempool.h index 099b7f6998..3405aa7663 100644 --- a/drivers/mempool/cnxk/cnxk_mempool.h +++ b/drivers/mempool/cnxk/cnxk_mempool.h @@ -23,4 +23,6 @@ int __rte_hot cnxk_mempool_enq(struct rte_mempool *mp, void *const *obj_table, int __rte_hot cnxk_mempool_deq(struct rte_mempool *mp, void **obj_table, unsigned int n); +int cn10k_mempool_plt_init(void); + #endif diff --git a/drivers/mempool/cnxk/cnxk_mempool_ops.c b/drivers/mempool/cnxk/cnxk_mempool_ops.c index 42c02bf14e..c7b75f026d 100644 --- a/drivers/mempool/cnxk/cnxk_mempool_ops.c +++ b/drivers/mempool/cnxk/cnxk_mempool_ops.c @@ -174,12 +174,15 @@ cnxk_mempool_populate(struct rte_mempool *mp, unsigned int max_objs, static int cnxk_mempool_plt_init(void) { - if (roc_model_is_cn9k()) + int rc = 0; + + if (roc_model_is_cn9k()) { rte_mbuf_set_platform_mempool_ops("cn9k_mempool_ops"); - else if (roc_model_is_cn10k()) + } else if (roc_model_is_cn10k()) { rte_mbuf_set_platform_mempool_ops("cn10k_mempool_ops"); - - return 0; + rc = cn10k_mempool_plt_init(); + } + return rc; } RTE_INIT(cnxk_mempool_ops_init) From patchwork Thu Apr 8 09:50:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90869 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 47940A0579; Thu, 8 Apr 2021 11:51:58 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7D3691410BF; Thu, 8 Apr 2021 11:51:28 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 027D31410B9 for ; Thu, 8 Apr 2021 11:51:27 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1389pJ4I020776 for ; Thu, 8 Apr 2021 02:51:27 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=9z9awlTkX7mV62t3OZnm8/UldOgQxqaEUSwDUvz/BpE=; b=IuvWowmo75htSYqItH4aQlw9xdjA/XP/74WShgzlVH+psJm/neUOkRNiyK+9AeU4/sxZ iEf3cOOYkmNumkCJpAfGULPxZWScFal00tRNo22Q6XvqJV05yiQ6Pyhlep9TEVQjNE0R GGXm97iQmKpyIFMWt6T4TEeS2LXyO/UpuilrCWXqVsXK0hJaC3eetITbK0Cqi4F2gwiG YdJCmWt8wYl4wIr1pt+A36JIaJCeLHjqm8DFz+IpzDJtRzxRDUhB3T6ekwcKAsjQLfEp 5q4jlt8e1gzObOg7cmuObjKhwfHUZR0UGYGrW7ev9lu1VGRJW8OYFMGwCRtw/TDEdFg0 dg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37swewgfad-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 08 Apr 2021 02:51:27 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 8 Apr 2021 02:51:25 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 8 Apr 2021 02:51:25 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id F1B273F7045; Thu, 8 Apr 2021 02:51:22 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Thu, 8 Apr 2021 15:20:47 +0530 Message-ID: <20210408095049.3100322-10-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210408095049.3100322-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210408095049.3100322-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: MLcQCR9xhWFpLE8lMBfKWmZk_go5u3ST X-Proofpoint-GUID: MLcQCR9xhWFpLE8lMBfKWmZk_go5u3ST X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-04-08_02:2021-04-08, 2021-04-08 signatures=0 Subject: [dpdk-dev] [PATCH v4 09/11] mempool/cnxk: add cn10k batch enqueue op X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the implementation for Marvell CN10k mempool batch enqueue op. Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cn10k_mempool_ops.c | 28 +++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c index a3aef0ddb2..c225c227df 100644 --- a/drivers/mempool/cnxk/cn10k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -131,6 +131,32 @@ batch_op_fini(struct rte_mempool *mp) rte_wmb(); } +static int __rte_hot +cn10k_mempool_enq(struct rte_mempool *mp, void *const *obj_table, + unsigned int n) +{ + const uint64_t *ptr = (const uint64_t *)obj_table; + uint64_t lmt_addr = 0, lmt_id = 0; + struct batch_op_data *op_data; + + /* Ensure mbuf init changes are written before the free pointers are + * enqueued to the stack. + */ + rte_io_wmb(); + + if (n == 1) { + roc_npa_aura_op_free(mp->pool_id, 1, ptr[0]); + return 0; + } + + op_data = batch_op_data_get(mp->pool_id); + lmt_addr = op_data->lmt_addr; + ROC_LMT_BASE_ID_GET(lmt_addr, lmt_id); + roc_npa_aura_op_batch_free(mp->pool_id, ptr, n, 1, lmt_addr, lmt_id); + + return 0; +} + static int cn10k_mempool_alloc(struct rte_mempool *mp) { @@ -187,7 +213,7 @@ static struct rte_mempool_ops cn10k_mempool_ops = { .name = "cn10k_mempool_ops", .alloc = cn10k_mempool_alloc, .free = cn10k_mempool_free, - .enqueue = cnxk_mempool_enq, + .enqueue = cn10k_mempool_enq, .dequeue = cnxk_mempool_deq, .get_count = cnxk_mempool_get_count, .calc_mem_size = cnxk_mempool_calc_mem_size, From patchwork Thu Apr 8 09:50:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90870 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5D2E5A0579; Thu, 8 Apr 2021 11:52:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 56F721410DF; Thu, 8 Apr 2021 11:51:32 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id DA48A1410D0 for ; Thu, 8 Apr 2021 11:51:30 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1389pJ4J020776 for ; Thu, 8 Apr 2021 02:51:30 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=Yo4EdKWZyn6sdv5wu7Szwvf44Rp7ZMUo0Irel90M6wU=; b=VRAfez0i0bNB50gcA/fWtRPsaGWtmGu6r/13VfW+nssds1EkiJcGlV/TYvAEC2JugM8p QQgkW1imjYNDtISXj38zVQTW9TK8eWkb06xTiqkn4ZAj6HTalN3DUOCHkDy6DjPtFZ6h p3W9ACYV9FdMEgUxKfato2QHe/xR19HdORP4ZSPGgAyCHksFTvcknDQHFJaXf3np6uIH qaFS5DwOdcnK4TQWcZzK7g93lXoAblM6pE8Va66IOi/uHT8b2VvH4c8qQBKwF5LEdYre FEgDRhjngNMGSJYAp3N4PQWb/QYfsLCjkU2w8+jmOrsdEDT+3JFbAr3aMwRUkEQhgh95 zQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com with ESMTP id 37swewgfbn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 08 Apr 2021 02:51:30 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 8 Apr 2021 02:51:28 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 8 Apr 2021 02:51:28 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id E5F963F7040; Thu, 8 Apr 2021 02:51:25 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Thu, 8 Apr 2021 15:20:48 +0530 Message-ID: <20210408095049.3100322-11-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210408095049.3100322-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210408095049.3100322-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: VyKbYOlROuzj5CBIWxWSOlhpPLWgb6aF X-Proofpoint-GUID: VyKbYOlROuzj5CBIWxWSOlhpPLWgb6aF X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-04-08_02:2021-04-08, 2021-04-08 signatures=0 Subject: [dpdk-dev] [PATCH v4 10/11] mempool/cnxk: add cn10k get count op X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the implementation for Marvell CN10k get count op. Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cn10k_mempool_ops.c | 28 +++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c index c225c227df..d244a5e90f 100644 --- a/drivers/mempool/cnxk/cn10k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -157,6 +157,32 @@ cn10k_mempool_enq(struct rte_mempool *mp, void *const *obj_table, return 0; } +static unsigned int +cn10k_mempool_get_count(const struct rte_mempool *mp) +{ + struct batch_op_data *op_data; + unsigned int count = 0; + int i; + + op_data = batch_op_data_get(mp->pool_id); + + rte_wmb(); + for (i = 0; i < RTE_MAX_LCORE; i++) { + struct batch_op_mem *mem = &op_data->mem[i]; + + if (mem->status == BATCH_ALLOC_OP_ISSUED) + count += roc_npa_aura_batch_alloc_count(mem->objs, + BATCH_ALLOC_SZ); + + if (mem->status == BATCH_ALLOC_OP_DONE) + count += mem->sz; + } + + count += cnxk_mempool_get_count(mp); + + return count; +} + static int cn10k_mempool_alloc(struct rte_mempool *mp) { @@ -215,7 +241,7 @@ static struct rte_mempool_ops cn10k_mempool_ops = { .free = cn10k_mempool_free, .enqueue = cn10k_mempool_enq, .dequeue = cnxk_mempool_deq, - .get_count = cnxk_mempool_get_count, + .get_count = cn10k_mempool_get_count, .calc_mem_size = cnxk_mempool_calc_mem_size, .populate = cnxk_mempool_populate, }; From patchwork Thu Apr 8 09:50:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ashwin Sekhar T K X-Patchwork-Id: 90871 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7E3A5A0579; Thu, 8 Apr 2021 11:52:16 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 84B5C1410D9; Thu, 8 Apr 2021 11:51:35 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id AFCC01410E4 for ; Thu, 8 Apr 2021 11:51:33 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 1389ocvs026882 for ; Thu, 8 Apr 2021 02:51:32 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=K8SAd5qHCm7tqNs2/mmzRpMNzc/Y+z2mlYaJWuiSyfU=; b=eARrIN3o2kfQJgTQuzVWGnH/BprHEisB1dPL9CMWGwKI4XVCRKv3xiaHC8TFnPfo//M/ 4yFlfwtODch0/TYG6+V3UBkGX85JctAgRyi9VLMh+2Uh9KbZbRpcphfAs4jjUPOFS0ca aMpp1SYvbdvkUJ9YH231crDjRXmYB1W5hsKhH9B9EseEiQaKoow9hwv58OWnLnGyf+0H rAvyjo96cZhil2UboKa9l2gz0PJMk6uRTxMfTdsjTmuhECdQlnJUPSKtzkfyWnf4PY2j +fck9xVMhSjQuSoi7Bf+6vpdaecRXR8jc4WqkHhvrhowvwdlvCCS7O9x4j0IPLRXWiGy gw== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com with ESMTP id 37shqxjj1a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 08 Apr 2021 02:51:32 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 8 Apr 2021 02:51:31 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Thu, 8 Apr 2021 02:51:31 -0700 Received: from lab-ci-142.marvell.com (unknown [10.28.36.142]) by maili.marvell.com (Postfix) with ESMTP id D98A23F7041; Thu, 8 Apr 2021 02:51:28 -0700 (PDT) From: Ashwin Sekhar T K To: CC: , , , , , , Date: Thu, 8 Apr 2021 15:20:49 +0530 Message-ID: <20210408095049.3100322-12-asekhar@marvell.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <20210408095049.3100322-1-asekhar@marvell.com> References: <20210305162149.2196166-1-asekhar@marvell.com> <20210408095049.3100322-1-asekhar@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: lzj4-6QgdgQvJNogGJTek25jWds86eFP X-Proofpoint-GUID: lzj4-6QgdgQvJNogGJTek25jWds86eFP X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391, 18.0.761 definitions=2021-04-08_02:2021-04-08, 2021-04-08 signatures=0 Subject: [dpdk-dev] [PATCH v4 11/11] mempool/cnxk: add cn10k batch dequeue op X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the implementation for Marvell CN10k mempool batch dequeue op. Signed-off-by: Ashwin Sekhar T K --- drivers/mempool/cnxk/cn10k_mempool_ops.c | 72 +++++++++++++++++++++++- 1 file changed, 71 insertions(+), 1 deletion(-) diff --git a/drivers/mempool/cnxk/cn10k_mempool_ops.c b/drivers/mempool/cnxk/cn10k_mempool_ops.c index d244a5e90f..95458b34b7 100644 --- a/drivers/mempool/cnxk/cn10k_mempool_ops.c +++ b/drivers/mempool/cnxk/cn10k_mempool_ops.c @@ -183,6 +183,76 @@ cn10k_mempool_get_count(const struct rte_mempool *mp) return count; } +static int __rte_hot +cn10k_mempool_deq(struct rte_mempool *mp, void **obj_table, unsigned int n) +{ + struct batch_op_data *op_data; + struct batch_op_mem *mem; + unsigned int count = 0; + int tid, rc, retry; + bool loop = true; + + op_data = batch_op_data_get(mp->pool_id); + tid = rte_lcore_id(); + mem = &op_data->mem[tid]; + + /* Issue batch alloc */ + if (mem->status == BATCH_ALLOC_OP_NOT_ISSUED) { + rc = roc_npa_aura_batch_alloc_issue(mp->pool_id, mem->objs, + BATCH_ALLOC_SZ, 0, 1); + /* If issue fails, try falling back to default alloc */ + if (unlikely(rc)) + return cn10k_mempool_enq(mp, obj_table, n); + mem->status = BATCH_ALLOC_OP_ISSUED; + } + + retry = 4; + while (loop) { + unsigned int cur_sz; + + if (mem->status == BATCH_ALLOC_OP_ISSUED) { + mem->sz = roc_npa_aura_batch_alloc_extract( + mem->objs, mem->objs, BATCH_ALLOC_SZ); + + /* If partial alloc reduce the retry count */ + retry -= (mem->sz != BATCH_ALLOC_SZ); + /* Break the loop if retry count exhausted */ + loop = !!retry; + mem->status = BATCH_ALLOC_OP_DONE; + } + + cur_sz = n - count; + if (cur_sz > mem->sz) + cur_sz = mem->sz; + + /* Dequeue the pointers */ + memcpy(&obj_table[count], &mem->objs[mem->sz - cur_sz], + cur_sz * sizeof(uintptr_t)); + mem->sz -= cur_sz; + count += cur_sz; + + /* Break loop if the required pointers has been dequeued */ + loop &= (count != n); + + /* Issue next batch alloc if pointers are exhausted */ + if (mem->sz == 0) { + rc = roc_npa_aura_batch_alloc_issue( + mp->pool_id, mem->objs, BATCH_ALLOC_SZ, 0, 1); + /* Break loop if issue failed and set status */ + loop &= !rc; + mem->status = !rc; + } + } + + if (unlikely(count != n)) { + /* No partial alloc allowed. Free up allocated pointers */ + cn10k_mempool_enq(mp, obj_table, count); + return -ENOENT; + } + + return 0; +} + static int cn10k_mempool_alloc(struct rte_mempool *mp) { @@ -240,7 +310,7 @@ static struct rte_mempool_ops cn10k_mempool_ops = { .alloc = cn10k_mempool_alloc, .free = cn10k_mempool_free, .enqueue = cn10k_mempool_enq, - .dequeue = cnxk_mempool_deq, + .dequeue = cn10k_mempool_deq, .get_count = cn10k_mempool_get_count, .calc_mem_size = cnxk_mempool_calc_mem_size, .populate = cnxk_mempool_populate,