From patchwork Mon Jan 10 05:26:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rohit Raj X-Patchwork-Id: 105703 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 07307A0351; Mon, 10 Jan 2022 06:26:31 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 803E040E0F; Mon, 10 Jan 2022 06:26:31 +0100 (CET) Received: from inva021.nxp.com (inva021.nxp.com [92.121.34.21]) by mails.dpdk.org (Postfix) with ESMTP id 415844013F for ; Mon, 10 Jan 2022 06:26:30 +0100 (CET) Received: from inva021.nxp.com (localhost [127.0.0.1]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id F2B14201138; Mon, 10 Jan 2022 06:26:29 +0100 (CET) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 90E01201137; Mon, 10 Jan 2022 06:26:29 +0100 (CET) Received: from lsv03196.swis.in-blr01.nxp.com (lsv03196.swis.in-blr01.nxp.com [92.120.146.192]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id 64EC3183AC4C; Mon, 10 Jan 2022 13:26:28 +0800 (+08) From: rohit.raj@nxp.com To: Bruce Richardson , Ray Kinsella , Dmitry Kozlyuk , Narcisa Ana Maria Vasile , Dmitry Malloy , Pallavi Kadam Cc: dev@dpdk.org, nipun.gupta@nxp.com, sachin.saxena@nxp.com, hemant.agrawal@nxp.com, Rohit Raj Subject: [PATCH v5 1/2] eal: add API for bus close Date: Mon, 10 Jan 2022 10:56:26 +0530 Message-Id: <20220110052627.22577-1-rohit.raj@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201008153048.19369-1-rohit.raj@nxp.com> References: <20201008153048.19369-1-rohit.raj@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Rohit Raj As per the current code we have API for bus probe, but the bus close API is missing. This breaks the multi process scenarios as objects are not cleaned while terminating the secondary processes. This patch adds a new API rte_bus_close() for cleanup of bus objects which were acquired during probe. Signed-off-by: Rohit Raj --- Rebased on this patch series: https://patches.dpdk.org/project/dpdk/list/?series=21049 v5: * Updated release notes for new feature and API change. * Added support for error checking while closing bus. * Added experimental banner for new API. * Squashed changes related to freebsd and windows into single patch. * Discarded patch to fix a bug which is already fixed on latest release. v4: * Added comments to clarify responsibility of rte_bus_close. * Added support for rte_bus_close on freebsd. * Added support for rte_bus_close on windows. v3: * nit: combined nested if statements. v2: * Moved rte_bus_close call to rte_eal_cleanup path. doc/guides/rel_notes/release_22_03.rst | 8 +++++++ lib/eal/common/eal_common_bus.c | 33 +++++++++++++++++++++++++- lib/eal/freebsd/eal.c | 1 + lib/eal/include/rte_bus.h | 30 ++++++++++++++++++++++- lib/eal/linux/eal.c | 8 +++++++ lib/eal/version.map | 3 +++ lib/eal/windows/eal.c | 1 + 7 files changed, 82 insertions(+), 2 deletions(-) diff --git a/doc/guides/rel_notes/release_22_03.rst b/doc/guides/rel_notes/release_22_03.rst index 6d99d1eaa9..7417606a2a 100644 --- a/doc/guides/rel_notes/release_22_03.rst +++ b/doc/guides/rel_notes/release_22_03.rst @@ -55,6 +55,11 @@ New Features Also, make sure to start the actual text at the margin. ======================================================= + * **Added support to close bus.** + + Added capability to allow a user to do cleanup of bus objects which + were acquired during bus probe. + Removed Items ------------- @@ -84,6 +89,9 @@ API Changes Also, make sure to start the actual text at the margin. ======================================================= + * eal: Added new API ``rte_bus_close`` to perform cleanup bus objects which + were acquired during bus probe. + ABI Changes ----------- diff --git a/lib/eal/common/eal_common_bus.c b/lib/eal/common/eal_common_bus.c index baa5b532af..2c3c0a90d2 100644 --- a/lib/eal/common/eal_common_bus.c +++ b/lib/eal/common/eal_common_bus.c @@ -1,5 +1,5 @@ /* SPDX-License-Identifier: BSD-3-Clause - * Copyright 2016 NXP + * Copyright 2016,2022 NXP */ #include @@ -85,6 +85,37 @@ rte_bus_probe(void) return 0; } +/* Close all devices of all buses */ +int +rte_bus_close(void) +{ + int ret; + struct rte_bus *bus, *vbus = NULL; + + TAILQ_FOREACH(bus, &rte_bus_list, next) { + if (!strcmp(bus->name, "vdev")) { + vbus = bus; + continue; + } + + if (bus->close) { + ret = bus->close(); + if (ret) + RTE_LOG(ERR, EAL, "Bus (%s) close failed.\n", + bus->name); + } + } + + if (vbus && vbus->close) { + ret = vbus->close(); + if (ret) + RTE_LOG(ERR, EAL, "Bus (%s) close failed.\n", + vbus->name); + } + + return 0; +} + /* Dump information of a single bus */ static int bus_dump_one(FILE *f, struct rte_bus *bus) diff --git a/lib/eal/freebsd/eal.c b/lib/eal/freebsd/eal.c index a1cd2462db..87d70c6898 100644 --- a/lib/eal/freebsd/eal.c +++ b/lib/eal/freebsd/eal.c @@ -984,6 +984,7 @@ rte_eal_cleanup(void) { struct internal_config *internal_conf = eal_get_internal_configuration(); + rte_bus_close(); rte_service_finalize(); rte_mp_channel_cleanup(); /* after this point, any DPDK pointers will become dangling */ diff --git a/lib/eal/include/rte_bus.h b/lib/eal/include/rte_bus.h index bbbb6efd28..c6211bbd95 100644 --- a/lib/eal/include/rte_bus.h +++ b/lib/eal/include/rte_bus.h @@ -1,5 +1,5 @@ /* SPDX-License-Identifier: BSD-3-Clause - * Copyright 2016 NXP + * Copyright 2016,2022 NXP */ #ifndef _RTE_BUS_H_ @@ -66,6 +66,23 @@ typedef int (*rte_bus_scan_t)(void); */ typedef int (*rte_bus_probe_t)(void); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Implementation specific close function which is responsible for resetting all + * detected devices on the bus to a default state, closing UIO nodes or VFIO + * groups and also freeing any memory allocated during rte_bus_probe like + * private resources for device list. + * + * This is called while iterating over each registered bus. + * + * @return + * 0 for successful close + * !0 for any error while closing + */ +typedef int (*rte_bus_close_t)(void); + /** * Device iterator to find a device on a bus. * @@ -263,6 +280,7 @@ struct rte_bus { const char *name; /**< Name of the bus */ rte_bus_scan_t scan; /**< Scan for devices attached to bus */ rte_bus_probe_t probe; /**< Probe devices on bus */ + rte_bus_close_t close; /**< Close devices on bus */ rte_bus_find_device_t find_device; /**< Find a device on the bus */ rte_bus_plug_t plug; /**< Probe single device for drivers */ rte_bus_unplug_t unplug; /**< Remove single device from driver */ @@ -317,6 +335,16 @@ int rte_bus_scan(void); */ int rte_bus_probe(void); +/** + * For each device on the buses, call the device specific close. + * + * @return + * 0 for successful close + * !0 otherwise + */ +__rte_experimental +int rte_bus_close(void); + /** * Dump information of all the buses registered with EAL. * diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c index 60b4924838..5c60131e46 100644 --- a/lib/eal/linux/eal.c +++ b/lib/eal/linux/eal.c @@ -1362,6 +1362,14 @@ rte_eal_cleanup(void) if (rte_eal_process_type() == RTE_PROC_PRIMARY) rte_memseg_walk(mark_freeable, NULL); + + /* Close all the buses and devices/drivers on them */ + if (rte_bus_close()) { + rte_eal_init_alert("Cannot close devices"); + rte_errno = ENOTSUP; + return -1; + } + rte_service_finalize(); rte_mp_channel_cleanup(); /* after this point, any DPDK pointers will become dangling */ diff --git a/lib/eal/version.map b/lib/eal/version.map index ab28c22791..39882dbbd5 100644 --- a/lib/eal/version.map +++ b/lib/eal/version.map @@ -420,6 +420,9 @@ EXPERIMENTAL { rte_intr_instance_free; rte_intr_type_get; rte_intr_type_set; + + # added in 22.03 + rte_bus_close; }; INTERNAL { diff --git a/lib/eal/windows/eal.c b/lib/eal/windows/eal.c index 67db7f099a..5915ab6291 100644 --- a/lib/eal/windows/eal.c +++ b/lib/eal/windows/eal.c @@ -260,6 +260,7 @@ rte_eal_cleanup(void) struct internal_config *internal_conf = eal_get_internal_configuration(); + rte_bus_close(); eal_intr_thread_cancel(); eal_mem_virt2iova_cleanup(); /* after this point, any DPDK pointers will become dangling */ From patchwork Mon Jan 10 05:26:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rohit Raj X-Patchwork-Id: 105704 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 30BFCA0351; Mon, 10 Jan 2022 06:26:38 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 81332411B2; Mon, 10 Jan 2022 06:26:32 +0100 (CET) Received: from inva021.nxp.com (inva021.nxp.com [92.121.34.21]) by mails.dpdk.org (Postfix) with ESMTP id CCA064013F for ; Mon, 10 Jan 2022 06:26:30 +0100 (CET) Received: from inva021.nxp.com (localhost [127.0.0.1]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id A60C320113E; Mon, 10 Jan 2022 06:26:30 +0100 (CET) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 439B1201137; Mon, 10 Jan 2022 06:26:30 +0100 (CET) Received: from lsv03196.swis.in-blr01.nxp.com (lsv03196.swis.in-blr01.nxp.com [92.120.146.192]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id 2D182183AD0B; Mon, 10 Jan 2022 13:26:29 +0800 (+08) From: rohit.raj@nxp.com To: Hemant Agrawal , Sachin Saxena , Ray Kinsella , Nipun Gupta , Anatoly Burakov Cc: dev@dpdk.org, sachin.saxena@nxp.com, Rohit Raj Subject: [PATCH v5 2/2] bus/fslmc: support bus close API Date: Mon, 10 Jan 2022 10:56:27 +0530 Message-Id: <20220110052627.22577-2-rohit.raj@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220110052627.22577-1-rohit.raj@nxp.com> References: <20201008153048.19369-1-rohit.raj@nxp.com> <20220110052627.22577-1-rohit.raj@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Rohit Raj This patch add support for closing the bus objects which were acquired In the bus probe. Some devices need to be cleaned while in both primary and secondary process and while some need to be cleaned only in case of primary process. The devices are closed as per the white list used while creating the objects in a particular process. Signed-off-by: Rohit Raj --- drivers/bus/fslmc/fslmc_bus.c | 15 +++- drivers/bus/fslmc/fslmc_vfio.c | 89 +++++++++++++++++++++++- drivers/bus/fslmc/fslmc_vfio.h | 3 +- drivers/bus/fslmc/mc/fsl_dpcon.h | 3 +- drivers/bus/fslmc/portal/dpaa2_hw_dpbp.c | 30 +++++++- drivers/bus/fslmc/portal/dpaa2_hw_dpci.c | 30 +++++++- drivers/bus/fslmc/portal/dpaa2_hw_dpio.c | 36 +++++++++- drivers/bus/fslmc/portal/dpaa2_hw_dprc.c | 31 ++++++++- drivers/bus/fslmc/rte_fslmc.h | 5 +- drivers/bus/fslmc/version.map | 1 + drivers/event/dpaa2/dpaa2_hw_dpcon.c | 31 ++++++++- drivers/net/dpaa2/dpaa2_mux.c | 20 +++++- 12 files changed, 282 insertions(+), 12 deletions(-) diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c index a3c0d838c4..79053233a1 100644 --- a/drivers/bus/fslmc/fslmc_bus.c +++ b/drivers/bus/fslmc/fslmc_bus.c @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: BSD-3-Clause * - * Copyright 2016,2018-2021 NXP + * Copyright 2016,2018-2022 NXP * */ @@ -493,6 +493,18 @@ rte_fslmc_probe(void) return 0; } +static int +rte_fslmc_close(void) +{ + int ret = 0; + + ret = fslmc_vfio_close_group(); + if (ret) + DPAA2_BUS_ERR("Unable to close devices %d", ret); + + return 0; +} + static struct rte_device * rte_fslmc_find_device(const struct rte_device *start, rte_dev_cmp_t cmp, const void *data) @@ -668,6 +680,7 @@ struct rte_fslmc_bus rte_fslmc_bus = { .bus = { .scan = rte_fslmc_scan, .probe = rte_fslmc_probe, + .close = rte_fslmc_close, .parse = rte_fslmc_parse, .find_device = rte_fslmc_find_device, .get_iommu_class = rte_dpaa2_get_iommu_class, diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fslmc/fslmc_vfio.c index 1b89a56bbc..59bb0f0171 100644 --- a/drivers/bus/fslmc/fslmc_vfio.c +++ b/drivers/bus/fslmc/fslmc_vfio.c @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause * * Copyright (c) 2015-2016 Freescale Semiconductor, Inc. All rights reserved. - * Copyright 2016-2021 NXP + * Copyright 2016-2022 NXP * */ @@ -1066,3 +1066,90 @@ fslmc_vfio_setup_group(void) return 0; } + +static void +fslmc_close_iodevices(struct rte_dpaa2_device *dev) +{ + struct rte_dpaa2_object *object = NULL; + struct rte_dpaa2_driver *drv; + int ret, probe_all; + + switch (dev->dev_type) { + case DPAA2_IO: + case DPAA2_CON: + case DPAA2_CI: + case DPAA2_BPOOL: + case DPAA2_MUX: + TAILQ_FOREACH(object, &dpaa2_obj_list, next) { + if (dev->dev_type == object->dev_type) + object->close(dev->object_id); + else + continue; + } + break; + case DPAA2_ETH: + case DPAA2_CRYPTO: + case DPAA2_QDMA: + probe_all = rte_fslmc_bus.bus.conf.scan_mode != + RTE_BUS_SCAN_ALLOWLIST; + TAILQ_FOREACH(drv, &rte_fslmc_bus.driver_list, next) { + if (drv->drv_type != dev->dev_type) + continue; + if (rte_dev_is_probed(&dev->device)) + continue; + if (probe_all || + (dev->device.devargs && + dev->device.devargs->policy == + RTE_DEV_ALLOWED)) { + ret = drv->remove(dev); + if (ret) + DPAA2_BUS_ERR("Unable to remove"); + } + } + break; + default: + break; + } + + DPAA2_BUS_LOG(DEBUG, "Device (%s) Closed", + dev->device.name); +} + +int +fslmc_vfio_close_group(void) +{ + struct rte_dpaa2_device *dev, *dev_temp; + + RTE_TAILQ_FOREACH_SAFE(dev, &rte_fslmc_bus.device_list, next, dev_temp) { + if (dev->device.devargs && + dev->device.devargs->policy == RTE_DEV_BLOCKED) { + DPAA2_BUS_LOG(DEBUG, "%s Blacklisted, skipping", + dev->device.name); + TAILQ_REMOVE(&rte_fslmc_bus.device_list, dev, next); + continue; + } + switch (dev->dev_type) { + case DPAA2_ETH: + case DPAA2_CRYPTO: + case DPAA2_QDMA: + case DPAA2_IO: + fslmc_close_iodevices(dev); + break; + case DPAA2_CON: + case DPAA2_CI: + case DPAA2_BPOOL: + case DPAA2_MUX: + if (rte_eal_process_type() == RTE_PROC_SECONDARY) + continue; + + fslmc_close_iodevices(dev); + break; + case DPAA2_DPRTC: + default: + DPAA2_BUS_DEBUG("Device cannot be closed: Not supported (%s)", + dev->device.name); + } + } + + return 0; +} diff --git a/drivers/bus/fslmc/fslmc_vfio.h b/drivers/bus/fslmc/fslmc_vfio.h index 133606a9fd..5f2f8119fb 100644 --- a/drivers/bus/fslmc/fslmc_vfio.h +++ b/drivers/bus/fslmc/fslmc_vfio.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause * * Copyright (c) 2015-2016 Freescale Semiconductor, Inc. All rights reserved. - * Copyright 2016,2019 NXP + * Copyright 2016,2019,2021-2022 NXP * */ @@ -55,6 +55,7 @@ int rte_dpaa2_vfio_setup_intr(struct rte_intr_handle *intr_handle, int fslmc_vfio_setup_group(void); int fslmc_vfio_process_group(void); +int fslmc_vfio_close_group(void); char *fslmc_get_container(void); int fslmc_get_container_group(int *gropuid); int rte_fslmc_vfio_dmamap(void); diff --git a/drivers/bus/fslmc/mc/fsl_dpcon.h b/drivers/bus/fslmc/mc/fsl_dpcon.h index 7caa6c68a1..a158f576b5 100644 --- a/drivers/bus/fslmc/mc/fsl_dpcon.h +++ b/drivers/bus/fslmc/mc/fsl_dpcon.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: (BSD-3-Clause OR GPL-2.0) * * Copyright 2013-2016 Freescale Semiconductor Inc. - * Copyright 2017-2019 NXP + * Copyright 2017-2019,2021-2022 NXP * */ #ifndef __FSL_DPCON_H @@ -26,6 +26,7 @@ int dpcon_open(struct fsl_mc_io *mc_io, int dpcon_id, uint16_t *token); +__rte_internal int dpcon_close(struct fsl_mc_io *mc_io, uint32_t cmd_flags, uint16_t token); diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpbp.c b/drivers/bus/fslmc/portal/dpaa2_hw_dpbp.c index 122aa1740d..4cde4890d9 100644 --- a/drivers/bus/fslmc/portal/dpaa2_hw_dpbp.c +++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpbp.c @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause * * Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved. - * Copyright 2016 NXP + * Copyright 2016,2021-2022 NXP * */ @@ -33,6 +33,19 @@ TAILQ_HEAD(dpbp_dev_list, dpaa2_dpbp_dev); static struct dpbp_dev_list dpbp_dev_list = TAILQ_HEAD_INITIALIZER(dpbp_dev_list); /*!< DPBP device list */ +static struct dpaa2_dpbp_dev *get_dpbp_from_id(uint32_t dpbp_id) +{ + struct dpaa2_dpbp_dev *dpbp_dev = NULL; + + /* Get DPBP dev handle from list using index */ + TAILQ_FOREACH(dpbp_dev, &dpbp_dev_list, next) { + if (dpbp_dev->dpbp_id == dpbp_id) + break; + } + + return dpbp_dev; +} + static int dpaa2_create_dpbp_device(int vdev_fd __rte_unused, struct vfio_device_info *obj_info __rte_unused, @@ -109,6 +122,20 @@ void dpaa2_free_dpbp_dev(struct dpaa2_dpbp_dev *dpbp) } } +static void dpaa2_close_dpbp_device(int object_id) +{ + struct dpaa2_dpbp_dev *dpbp_dev = NULL; + + dpbp_dev = get_dpbp_from_id((uint32_t)object_id); + + if (dpbp_dev) { + dpaa2_free_dpbp_dev(dpbp_dev); + dpbp_close(&dpbp_dev->dpbp, CMD_PRI_LOW, dpbp_dev->token); + TAILQ_REMOVE(&dpbp_dev_list, dpbp_dev, next); + rte_free(dpbp_dev); + } +} + int dpaa2_dpbp_supported(void) { if (TAILQ_EMPTY(&dpbp_dev_list)) @@ -119,6 +146,7 @@ int dpaa2_dpbp_supported(void) static struct rte_dpaa2_object rte_dpaa2_dpbp_obj = { .dev_type = DPAA2_BPOOL, .create = dpaa2_create_dpbp_device, + .close = dpaa2_close_dpbp_device, }; RTE_PMD_REGISTER_DPAA2_OBJECT(dpbp, rte_dpaa2_dpbp_obj); diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpci.c b/drivers/bus/fslmc/portal/dpaa2_hw_dpci.c index 8ed969c7c0..5eae63a6fd 100644 --- a/drivers/bus/fslmc/portal/dpaa2_hw_dpci.c +++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpci.c @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: BSD-3-Clause * - * Copyright 2017 NXP + * Copyright 2017,2021-2022 NXP * */ @@ -30,6 +30,19 @@ TAILQ_HEAD(dpci_dev_list, dpaa2_dpci_dev); static struct dpci_dev_list dpci_dev_list = TAILQ_HEAD_INITIALIZER(dpci_dev_list); /*!< DPCI device list */ +static struct dpaa2_dpci_dev *get_dpci_from_id(uint32_t dpci_id) +{ + struct dpaa2_dpci_dev *dpci_dev = NULL; + + /* Get DPCI dev handle from list using index */ + TAILQ_FOREACH(dpci_dev, &dpci_dev_list, next) { + if (dpci_dev->dpci_id == dpci_id) + break; + } + + return dpci_dev; +} + static int rte_dpaa2_create_dpci_device(int vdev_fd __rte_unused, struct vfio_device_info *obj_info __rte_unused, @@ -179,9 +192,24 @@ void rte_dpaa2_free_dpci_dev(struct dpaa2_dpci_dev *dpci) } } +static void rte_dpaa2_close_dpci_device(int object_id) +{ + struct dpaa2_dpci_dev *dpci_dev = NULL; + + dpci_dev = get_dpci_from_id((uint32_t)object_id); + + if (dpci_dev) { + rte_dpaa2_free_dpci_dev(dpci_dev); + dpci_close(&dpci_dev->dpci, CMD_PRI_LOW, dpci_dev->token); + TAILQ_REMOVE(&dpci_dev_list, dpci_dev, next); + rte_free(dpci_dev); + } +} + static struct rte_dpaa2_object rte_dpaa2_dpci_obj = { .dev_type = DPAA2_CI, .create = rte_dpaa2_create_dpci_device, + .close = rte_dpaa2_close_dpci_device, }; RTE_PMD_REGISTER_DPAA2_OBJECT(dpci, rte_dpaa2_dpci_obj); diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c index 2210a0fa4a..f8676ec53b 100644 --- a/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c +++ b/drivers/bus/fslmc/portal/dpaa2_hw_dpio.c @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: BSD-3-Clause * * Copyright (c) 2016 Freescale Semiconductor, Inc. All rights reserved. - * Copyright 2016-2019 NXP + * Copyright 2016-2019,2021-2022 NXP * */ #include @@ -86,6 +86,19 @@ static int dpaa2_cluster_sz = 2; * Cluster 4 (ID = x07) : CPU14, CPU15; */ +static struct dpaa2_dpio_dev *get_dpio_dev_from_id(int32_t dpio_id) +{ + struct dpaa2_dpio_dev *dpio_dev = NULL; + + /* Get DPIO dev handle from list using index */ + TAILQ_FOREACH(dpio_dev, &dpio_dev_list, next) { + if (dpio_dev->hw_id == dpio_id) + break; + } + + return dpio_dev; +} + static int dpaa2_get_core_id(void) { @@ -634,9 +647,30 @@ dpaa2_free_eq_descriptors(void) MAX_EQ_RESP_ENTRIES - 1; } +static void +dpaa2_close_dpio_device(int object_id) +{ + struct dpaa2_dpio_dev *dpio_dev = NULL; + + dpio_dev = get_dpio_dev_from_id((int32_t)object_id); + + if (dpio_dev) { + if (dpio_dev->dpio) { + dpio_disable(dpio_dev->dpio, CMD_PRI_LOW, + dpio_dev->token); + dpio_close(dpio_dev->dpio, CMD_PRI_LOW, + dpio_dev->token); + rte_free(dpio_dev->dpio); + } + TAILQ_REMOVE(&dpio_dev_list, dpio_dev, next); + rte_free(dpio_dev); + } +} + static struct rte_dpaa2_object rte_dpaa2_dpio_obj = { .dev_type = DPAA2_IO, .create = dpaa2_create_dpio_device, + .close = dpaa2_close_dpio_device, }; RTE_PMD_REGISTER_DPAA2_OBJECT(dpio, rte_dpaa2_dpio_obj); diff --git a/drivers/bus/fslmc/portal/dpaa2_hw_dprc.c b/drivers/bus/fslmc/portal/dpaa2_hw_dprc.c index ca1d0304d5..ec35fdc65a 100644 --- a/drivers/bus/fslmc/portal/dpaa2_hw_dprc.c +++ b/drivers/bus/fslmc/portal/dpaa2_hw_dprc.c @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: BSD-3-Clause * - * Copyright 2021 NXP + * Copyright 2021-2022 NXP * */ @@ -92,9 +92,38 @@ rte_dpaa2_create_dprc_device(int vdev_fd __rte_unused, return 0; } +static struct dpaa2_dprc_dev * +get_dprc_from_id(uint32_t dprc_id) +{ + struct dpaa2_dprc_dev *dprc_dev = NULL; + + /* Get DPRC dev handle from list using index */ + TAILQ_FOREACH(dprc_dev, &dprc_dev_list, next) { + if (dprc_dev->dprc_id == dprc_id) + break; + } + + return dprc_dev; +} + +static void +rte_dpaa2_close_dprc_device(int object_id) +{ + struct dpaa2_dprc_dev *dprc_dev = NULL; + + dprc_dev = get_dprc_from_id((uint32_t)object_id); + + if (dprc_dev) { + dprc_close(&dprc_dev->dprc, CMD_PRI_LOW, dprc_dev->token); + TAILQ_REMOVE(&dprc_dev_list, dprc_dev, next); + rte_free(dprc_dev); + } +} + static struct rte_dpaa2_object rte_dpaa2_dprc_obj = { .dev_type = DPAA2_DPRC, .create = rte_dpaa2_create_dprc_device, + .close = rte_dpaa2_close_dprc_device, }; RTE_PMD_REGISTER_DPAA2_OBJECT(dprc, rte_dpaa2_dprc_obj); diff --git a/drivers/bus/fslmc/rte_fslmc.h b/drivers/bus/fslmc/rte_fslmc.h index 12b586b13b..f9753d8af2 100644 --- a/drivers/bus/fslmc/rte_fslmc.h +++ b/drivers/bus/fslmc/rte_fslmc.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: BSD-3-Clause * - * Copyright 2016,2021 NXP + * Copyright 2016,2021-2022 NXP * */ @@ -104,6 +104,8 @@ typedef int (*rte_dpaa2_obj_create_t)(int vdev_fd, struct vfio_device_info *obj_info, int object_id); +typedef void (*rte_dpaa2_obj_close_t)(int object_id); + /** * A structure describing a DPAA2 object. */ @@ -112,6 +114,7 @@ struct rte_dpaa2_object { const char *name; /**< Name of Object. */ enum rte_dpaa2_dev_type dev_type; /**< Type of device */ rte_dpaa2_obj_create_t create; + rte_dpaa2_obj_close_t close; }; /** diff --git a/drivers/bus/fslmc/version.map b/drivers/bus/fslmc/version.map index 79b29746b6..465c4d10e6 100644 --- a/drivers/bus/fslmc/version.map +++ b/drivers/bus/fslmc/version.map @@ -34,6 +34,7 @@ INTERNAL { dpci_get_opr; dpci_set_opr; dpci_set_rx_queue; + dpcon_close; dpcon_get_attributes; dpcon_open; dpdmai_close; diff --git a/drivers/event/dpaa2/dpaa2_hw_dpcon.c b/drivers/event/dpaa2/dpaa2_hw_dpcon.c index acc1fde771..1de3ec9a81 100644 --- a/drivers/event/dpaa2/dpaa2_hw_dpcon.c +++ b/drivers/event/dpaa2/dpaa2_hw_dpcon.c @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: BSD-3-Clause * - * Copyright 2017 NXP + * Copyright 2017,2022 NXP * */ @@ -30,6 +30,19 @@ TAILQ_HEAD(dpcon_dev_list, dpaa2_dpcon_dev); static struct dpcon_dev_list dpcon_dev_list = TAILQ_HEAD_INITIALIZER(dpcon_dev_list); /*!< DPCON device list */ +static struct dpaa2_dpcon_dev *get_dpcon_from_id(uint32_t dpcon_id) +{ + struct dpaa2_dpcon_dev *dpcon_dev = NULL; + + /* Get DPCONC dev handle from list using index */ + TAILQ_FOREACH(dpcon_dev, &dpcon_dev_list, next) { + if (dpcon_dev->dpcon_id == dpcon_id) + break; + } + + return dpcon_dev; +} + static int rte_dpaa2_create_dpcon_device(int dev_fd __rte_unused, struct vfio_device_info *obj_info __rte_unused, @@ -105,9 +118,25 @@ void rte_dpaa2_free_dpcon_dev(struct dpaa2_dpcon_dev *dpcon) } } +static void +rte_dpaa2_close_dpcon_device(int object_id) +{ + struct dpaa2_dpcon_dev *dpcon_dev = NULL; + + dpcon_dev = get_dpcon_from_id((uint32_t)object_id); + + if (dpcon_dev) { + rte_dpaa2_free_dpcon_dev(dpcon_dev); + dpcon_close(&dpcon_dev->dpcon, CMD_PRI_LOW, dpcon_dev->token); + TAILQ_REMOVE(&dpcon_dev_list, dpcon_dev, next); + rte_free(dpcon_dev); + } +} + static struct rte_dpaa2_object rte_dpaa2_dpcon_obj = { .dev_type = DPAA2_CON, .create = rte_dpaa2_create_dpcon_device, + .close = rte_dpaa2_close_dpcon_device, }; RTE_PMD_REGISTER_DPAA2_OBJECT(dpcon, rte_dpaa2_dpcon_obj); diff --git a/drivers/net/dpaa2/dpaa2_mux.c b/drivers/net/dpaa2/dpaa2_mux.c index d347f4df51..e9289ca808 100644 --- a/drivers/net/dpaa2/dpaa2_mux.c +++ b/drivers/net/dpaa2/dpaa2_mux.c @@ -1,5 +1,5 @@ /* SPDX-License-Identifier: BSD-3-Clause - * Copyright 2018-2021 NXP + * Copyright 2018-2022 NXP */ #include @@ -44,7 +44,7 @@ static struct dpaa2_dpdmux_dev *get_dpdmux_from_id(uint32_t dpdmux_id) { struct dpaa2_dpdmux_dev *dpdmux_dev = NULL; - /* Get DPBP dev handle from list using index */ + /* Get DPDMUX dev handle from list using index */ TAILQ_FOREACH(dpdmux_dev, &dpdmux_dev_list, next) { if (dpdmux_dev->dpdmux_id == dpdmux_id) break; @@ -359,9 +359,25 @@ dpaa2_create_dpdmux_device(int vdev_fd __rte_unused, return -1; } +static void +dpaa2_close_dpdmux_device(int object_id) +{ + struct dpaa2_dpdmux_dev *dpdmux_dev; + + dpdmux_dev = get_dpdmux_from_id((uint32_t)object_id); + + if (dpdmux_dev) { + dpdmux_close(&dpdmux_dev->dpdmux, CMD_PRI_LOW, + dpdmux_dev->token); + TAILQ_REMOVE(&dpdmux_dev_list, dpdmux_dev, next); + rte_free(dpdmux_dev); + } +} + static struct rte_dpaa2_object rte_dpaa2_dpdmux_obj = { .dev_type = DPAA2_MUX, .create = dpaa2_create_dpdmux_device, + .close = dpaa2_close_dpdmux_device, }; RTE_PMD_REGISTER_DPAA2_OBJECT(dpdmux, rte_dpaa2_dpdmux_obj);