From patchwork Wed Aug 2 03:15:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "lihuisong (C)" X-Patchwork-Id: 129803 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 85E7042FB3; Wed, 2 Aug 2023 05:19:13 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 066D643268; Wed, 2 Aug 2023 05:18:52 +0200 (CEST) Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by mails.dpdk.org (Postfix) with ESMTP id 8C2074161A for ; Wed, 2 Aug 2023 05:18:45 +0200 (CEST) Received: from kwepemm600004.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4RFxtd3WYtztRlP; Wed, 2 Aug 2023 11:15:21 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by kwepemm600004.china.huawei.com (7.193.23.242) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Wed, 2 Aug 2023 11:18:42 +0800 From: Huisong Li To: CC: , , , , , , Subject: [PATCH RESEND v6 1/5] drivers/bus: restore driver assignment at front of probing Date: Wed, 2 Aug 2023 11:15:56 +0800 Message-ID: <20230802031600.23631-2-lihuisong@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20230802031600.23631-1-lihuisong@huawei.com> References: <20220825024425.10534-1-lihuisong@huawei.com> <20230802031600.23631-1-lihuisong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemm600004.china.huawei.com (7.193.23.242) X-CFilter-Loop: Reflected X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The driver assignment was moved back at the end of the device probing because there is no something to use rte_driver during the phase of probing. See commit 391797f04208 ("drivers/bus: move driver assignment to end of probing") However, it is necessary for probing callback to reference rte_driver before probing. For example, probing callback may call some APIs which access the rte_pci_driver::driver by the device::driver pointer to get driver information. In this case, a segment fault will occur in probing callback if there is not this assignment. Further, some comments in code need to be updated if we do that. The driver pointer in rte_device is set before probing and needs to be reset if probing failed. And rte_dev_is_probed can not be called inside probing. Fixes: 391797f04208 ("drivers/bus: move driver assignment to end of probing") Cc: stable@dpdk.org Signed-off-by: Huisong Li Acked-by: Chengwen Feng --- drivers/bus/auxiliary/auxiliary_common.c | 9 +++++++-- drivers/bus/dpaa/dpaa_bus.c | 9 +++++++-- drivers/bus/fslmc/fslmc_bus.c | 8 +++++++- drivers/bus/ifpga/ifpga_bus.c | 12 +++++++++--- drivers/bus/pci/pci_common.c | 9 +++++++-- drivers/bus/vdev/vdev.c | 10 ++++++++-- drivers/bus/vmbus/vmbus_common.c | 9 +++++++-- 7 files changed, 52 insertions(+), 14 deletions(-) diff --git a/drivers/bus/auxiliary/auxiliary_common.c b/drivers/bus/auxiliary/auxiliary_common.c index 29f99342a7..6313684b8f 100644 --- a/drivers/bus/auxiliary/auxiliary_common.c +++ b/drivers/bus/auxiliary/auxiliary_common.c @@ -132,16 +132,21 @@ rte_auxiliary_probe_one_driver(struct rte_auxiliary_driver *drv, } dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &drv->driver; AUXILIARY_LOG(INFO, "Probe auxiliary driver: %s device: %s (NUMA node %i)", drv->driver.name, dev->name, dev->device.numa_node); ret = drv->probe(drv, dev); if (ret != 0) { dev->driver = NULL; + dev->device.driver = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &drv->driver; } return ret; diff --git a/drivers/bus/dpaa/dpaa_bus.c b/drivers/bus/dpaa/dpaa_bus.c index e57159f5d8..f1b817e58c 100644 --- a/drivers/bus/dpaa/dpaa_bus.c +++ b/drivers/bus/dpaa/dpaa_bus.c @@ -693,17 +693,22 @@ rte_dpaa_bus_probe(void) (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_BLOCKED)) continue; - + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA_BUS_ERR("unable to probe:%s", dev->name); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c index 57bfb5111a..4bc0c6d3d4 100644 --- a/drivers/bus/fslmc/fslmc_bus.c +++ b/drivers/bus/fslmc/fslmc_bus.c @@ -471,15 +471,21 @@ rte_fslmc_probe(void) continue; } + /* + * Reference rte_driver before probing so as to this + * pointer can be used to get driver information in case + * of segment fault in probing callback. + */ + dev->device.driver = &drv->driver; if (probe_all || (dev->device.devargs && dev->device.devargs->policy == RTE_DEV_ALLOWED)) { ret = drv->probe(drv, dev); if (ret) { + dev->device.driver = NULL; DPAA2_BUS_ERR("Unable to probe"); } else { dev->driver = drv; - dev->device.driver = &drv->driver; } } break; diff --git a/drivers/bus/ifpga/ifpga_bus.c b/drivers/bus/ifpga/ifpga_bus.c index ffb0c61214..bfe7077645 100644 --- a/drivers/bus/ifpga/ifpga_bus.c +++ b/drivers/bus/ifpga/ifpga_bus.c @@ -294,13 +294,19 @@ ifpga_probe_one_driver(struct rte_afu_driver *drv, /* reference driver structure */ afu_dev->driver = drv; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + afu_dev->device.driver = &drv->driver; /* call the driver probe() function */ ret = drv->probe(afu_dev); - if (ret) + if (ret) { afu_dev->driver = NULL; - else - afu_dev->device.driver = &drv->driver; + afu_dev->device.driver = NULL; + } return ret; } diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c index 52404ab0fe..c0d9e959ea 100644 --- a/drivers/bus/pci/pci_common.c +++ b/drivers/bus/pci/pci_common.c @@ -302,6 +302,12 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; } } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; } RTE_LOG(INFO, EAL, "Probe PCI driver: %s (%x:%x) device: "PCI_PRI_FMT" (socket %i)\n", @@ -314,6 +320,7 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return ret; /* no rollback if already succeeded earlier */ if (ret) { dev->driver = NULL; + dev->device.driver = NULL; if ((dr->drv_flags & RTE_PCI_DRV_NEED_MAPPING) && /* Don't unmap if device is unsupported and * driver needs mapped resources. @@ -325,8 +332,6 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, dev->vfio_req_intr_handle = NULL; rte_intr_instance_free(dev->intr_handle); dev->intr_handle = NULL; - } else { - dev->device.driver = &dr->driver; } return ret; diff --git a/drivers/bus/vdev/vdev.c b/drivers/bus/vdev/vdev.c index 7974b27295..286ba1fbf9 100644 --- a/drivers/bus/vdev/vdev.c +++ b/drivers/bus/vdev/vdev.c @@ -207,9 +207,15 @@ vdev_probe_all_drivers(struct rte_vdev_device *dev) return -1; } + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &driver->driver; ret = driver->probe(dev); - if (ret == 0) - dev->device.driver = &driver->driver; + if (ret != 0) + dev->device.driver = NULL; return ret; } diff --git a/drivers/bus/vmbus/vmbus_common.c b/drivers/bus/vmbus/vmbus_common.c index 95f3ad78bc..df63eaeaf9 100644 --- a/drivers/bus/vmbus/vmbus_common.c +++ b/drivers/bus/vmbus/vmbus_common.c @@ -110,6 +110,12 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, /* reference driver structure */ dev->driver = dr; + /* + * Reference rte_driver before probing so as to this pointer can + * be used to get driver information in case of segment fault in + * probing callback. + */ + dev->device.driver = &dr->driver; if (dev->device.numa_node < 0 && rte_socket_count() > 1) VMBUS_LOG(INFO, "Device %s is not NUMA-aware", guid); @@ -119,9 +125,8 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, ret = dr->probe(dr, dev); if (ret) { dev->driver = NULL; + dev->device.driver = NULL; rte_vmbus_unmap_device(dev); - } else { - dev->device.driver = &dr->driver; } return ret; From patchwork Wed Aug 2 03:15:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "lihuisong (C)" X-Patchwork-Id: 129802 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3395142FB3; Wed, 2 Aug 2023 05:19:08 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0C09943262; Wed, 2 Aug 2023 05:18:51 +0200 (CEST) Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by mails.dpdk.org (Postfix) with ESMTP id 629DE41153 for ; Wed, 2 Aug 2023 05:18:45 +0200 (CEST) Received: from kwepemm600004.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RFxtY4sPVzNmh7; Wed, 2 Aug 2023 11:15:17 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by kwepemm600004.china.huawei.com (7.193.23.242) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Wed, 2 Aug 2023 11:18:42 +0800 From: Huisong Li To: CC: , , , , , , Subject: [PATCH RESEND v6 2/5] ethdev: fix skip valid port in probing callback Date: Wed, 2 Aug 2023 11:15:57 +0800 Message-ID: <20230802031600.23631-3-lihuisong@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20230802031600.23631-1-lihuisong@huawei.com> References: <20220825024425.10534-1-lihuisong@huawei.com> <20230802031600.23631-1-lihuisong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemm600004.china.huawei.com (7.193.23.242) X-CFilter-Loop: Reflected X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The event callback in application may use the macro RTE_ETH_FOREACH_DEV to iterate over all enabled ports to do something(like, verifying the port id validity) when receive a probing event. If the ethdev state of a port is not RTE_ETH_DEV_UNUSED, this port will be considered as a valid port. However, this state is set to RTE_ETH_DEV_ATTACHED after pushing probing event. It means that probing callback will skip this port. But this assignment can not move to front of probing notification. See commit be8cd210379a ("ethdev: fix port probing notification") So this patch has to add a new state, RTE_ETH_DEV_ALLOCATED. Set the ethdev state to RTE_ETH_DEV_ALLOCATED before pushing probing event and set it to RTE_ETH_DEV_ATTACHED after definitely probed. And this port is valid if its device state is 'ALLOCATED' or 'ATTACHED'. In addition, the new state has to be placed behind 'REMOVED' to avoid ABI break. Fortunately, this ethdev state is internal and applications can not access it directly. So this patch encapsulates an API, rte_eth_dev_is_used, for ethdev or PMD to call and eliminate concerns about using this state enum value comparison. Fixes: be8cd210379a ("ethdev: fix port probing notification") Cc: stable@dpdk.org Signed-off-by: Huisong Li Acked-by: Chengwen Feng --- drivers/net/bnxt/bnxt_ethdev.c | 3 ++- drivers/net/mlx5/mlx5.c | 2 +- lib/ethdev/ethdev_driver.c | 13 ++++++++++--- lib/ethdev/ethdev_driver.h | 12 ++++++++++++ lib/ethdev/ethdev_pci.h | 2 +- lib/ethdev/rte_class_eth.c | 2 +- lib/ethdev/rte_ethdev.c | 4 ++-- lib/ethdev/rte_ethdev.h | 4 +++- lib/ethdev/version.map | 1 + 9 files changed, 33 insertions(+), 10 deletions(-) diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index ee1552452a..bf1910709b 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -6108,7 +6108,8 @@ bnxt_dev_uninit(struct rte_eth_dev *eth_dev) PMD_DRV_LOG(DEBUG, "Calling Device uninit\n"); - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + + if (rte_eth_dev_is_used(eth_dev->state)) bnxt_dev_close_op(eth_dev); return 0; diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index b373306f98..54c6fff889 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -3152,7 +3152,7 @@ mlx5_eth_find_next(uint16_t port_id, struct rte_device *odev) while (port_id < RTE_MAX_ETHPORTS) { struct rte_eth_dev *dev = &rte_eth_devices[port_id]; - if (dev->state != RTE_ETH_DEV_UNUSED && + if (rte_eth_dev_is_used(dev->state) && dev->device && (dev->device == odev || (dev->device->driver && diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c index 0be1e8ca04..29e9417bea 100644 --- a/lib/ethdev/ethdev_driver.c +++ b/lib/ethdev/ethdev_driver.c @@ -50,8 +50,8 @@ eth_dev_find_free_port(void) for (i = 0; i < RTE_MAX_ETHPORTS; i++) { /* Using shared name field to find a free port. */ if (eth_dev_shared_data->data[i].name[0] == '\0') { - RTE_ASSERT(rte_eth_devices[i].state == - RTE_ETH_DEV_UNUSED); + RTE_ASSERT(!rte_eth_dev_is_used( + rte_eth_devices[i].state)); return i; } } @@ -208,11 +208,18 @@ rte_eth_dev_probing_finish(struct rte_eth_dev *dev) if (rte_eal_process_type() == RTE_PROC_SECONDARY) eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); + dev->state = RTE_ETH_DEV_ALLOCATED; rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_NEW, NULL); dev->state = RTE_ETH_DEV_ATTACHED; } +bool rte_eth_dev_is_used(uint16_t dev_state) +{ + return dev_state == RTE_ETH_DEV_ALLOCATED || + dev_state == RTE_ETH_DEV_ATTACHED; +} + int rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) { @@ -221,7 +228,7 @@ rte_eth_dev_release_port(struct rte_eth_dev *eth_dev) eth_dev_shared_data_prepare(); - if (eth_dev->state != RTE_ETH_DEV_UNUSED) + if (rte_eth_dev_is_used(eth_dev->state)) rte_eth_dev_callback_process(eth_dev, RTE_ETH_EVENT_DESTROY, NULL); diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 980f837ab6..5bd2780643 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1582,6 +1582,18 @@ int rte_eth_dev_callback_process(struct rte_eth_dev *dev, __rte_internal void rte_eth_dev_probing_finish(struct rte_eth_dev *dev); +/** + * Check if a Ethernet device state is used or not + * + * @param dev_state + * The state of the Ethernet device + * @return + * - true if the state of the Ethernet device is allocated or attached + * - false if this state is neither allocated nor attached + */ +__rte_internal +bool rte_eth_dev_is_used(uint16_t dev_state); + /** * Create memzone for HW rings. * malloc can't be used as the physical address is needed. diff --git a/lib/ethdev/ethdev_pci.h b/lib/ethdev/ethdev_pci.h index 320e3e0093..efe20be1a7 100644 --- a/lib/ethdev/ethdev_pci.h +++ b/lib/ethdev/ethdev_pci.h @@ -165,7 +165,7 @@ rte_eth_dev_pci_generic_remove(struct rte_pci_device *pci_dev, * eth device has been released. */ if (rte_eal_process_type() == RTE_PROC_SECONDARY && - eth_dev->state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_is_used(eth_dev->state)) return 0; if (dev_uninit) { diff --git a/lib/ethdev/rte_class_eth.c b/lib/ethdev/rte_class_eth.c index b61dae849d..88e56dd9a4 100644 --- a/lib/ethdev/rte_class_eth.c +++ b/lib/ethdev/rte_class_eth.c @@ -118,7 +118,7 @@ eth_dev_match(const struct rte_eth_dev *edev, const struct rte_kvargs *kvlist = arg->kvlist; unsigned int pair; - if (edev->state == RTE_ETH_DEV_UNUSED) + if (!rte_eth_dev_is_used(edev->state)) return -1; if (arg->device != NULL && arg->device != edev->device) return -1; diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index 0840d2b594..ec44490cde 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -338,7 +338,7 @@ uint16_t rte_eth_find_next(uint16_t port_id) { while (port_id < RTE_MAX_ETHPORTS && - rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED) + !rte_eth_dev_is_used(rte_eth_devices[port_id].state)) port_id++; if (port_id >= RTE_MAX_ETHPORTS) @@ -397,7 +397,7 @@ rte_eth_dev_is_valid_port(uint16_t port_id) int is_valid; if (port_id >= RTE_MAX_ETHPORTS || - (rte_eth_devices[port_id].state == RTE_ETH_DEV_UNUSED)) + !rte_eth_dev_is_used(rte_eth_devices[port_id].state)) is_valid = 0; else is_valid = 1; diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index 04a2564f22..e7e521efc4 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -2003,10 +2003,12 @@ typedef uint16_t (*rte_tx_callback_fn)(uint16_t port_id, uint16_t queue, enum rte_eth_dev_state { /** Device is unused before being probed. */ RTE_ETH_DEV_UNUSED = 0, - /** Device is attached when allocated in probing. */ + /** Device is attached when definitely probed. */ RTE_ETH_DEV_ATTACHED, /** Device is in removed state when plug-out is detected. */ RTE_ETH_DEV_REMOVED, + /** Device is allocated and is set before reporting new event. */ + RTE_ETH_DEV_ALLOCATED, }; struct rte_eth_dev_sriov { diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index b965d6aa52..ad95329b57 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -326,6 +326,7 @@ INTERNAL { rte_eth_dev_get_by_name; rte_eth_dev_is_rx_hairpin_queue; rte_eth_dev_is_tx_hairpin_queue; + rte_eth_dev_is_used; rte_eth_dev_probing_finish; rte_eth_dev_release_port; rte_eth_dev_internal_reset; From patchwork Wed Aug 2 03:15:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "lihuisong (C)" X-Patchwork-Id: 129799 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DA1EE42FB3; Wed, 2 Aug 2023 05:18:46 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B0450410DC; Wed, 2 Aug 2023 05:18:46 +0200 (CEST) Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by mails.dpdk.org (Postfix) with ESMTP id 517DF40DDB for ; Wed, 2 Aug 2023 05:18:45 +0200 (CEST) Received: from kwepemm600004.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4RFxtf1rYrztRmF; Wed, 2 Aug 2023 11:15:22 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by kwepemm600004.china.huawei.com (7.193.23.242) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Wed, 2 Aug 2023 11:18:43 +0800 From: Huisong Li To: CC: , , , , , , Subject: [PATCH RESEND v6 3/5] app/testpmd: check the validity of the port Date: Wed, 2 Aug 2023 11:15:58 +0800 Message-ID: <20230802031600.23631-4-lihuisong@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20230802031600.23631-1-lihuisong@huawei.com> References: <20220825024425.10534-1-lihuisong@huawei.com> <20230802031600.23631-1-lihuisong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemm600004.china.huawei.com (7.193.23.242) X-CFilter-Loop: Reflected X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch checks the validity of port id for all events in 'eth_event_callback()'. Signed-off-by: Huisong Li Acked-by: Aman Singh Acked-by: Chengwen Feng --- app/test-pmd/testpmd.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 938ca035d4..f934f6b418 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3945,14 +3945,15 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, fflush(stdout); } + if (port_id_is_invalid(port_id, DISABLED_WARN)) + return 0; + switch (type) { case RTE_ETH_EVENT_NEW: ports[port_id].need_setup = 1; ports[port_id].port_status = RTE_PORT_HANDLING; break; case RTE_ETH_EVENT_INTR_RMV: - if (port_id_is_invalid(port_id, DISABLED_WARN)) - break; if (rte_eal_alarm_set(100000, rmv_port_callback, (void *)(intptr_t)port_id)) fprintf(stderr, From patchwork Wed Aug 2 03:15:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "lihuisong (C)" X-Patchwork-Id: 129801 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B0C7042FB3; Wed, 2 Aug 2023 05:19:02 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DF5134325E; Wed, 2 Aug 2023 05:18:49 +0200 (CEST) Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by mails.dpdk.org (Postfix) with ESMTP id 5D845410DC for ; Wed, 2 Aug 2023 05:18:45 +0200 (CEST) Received: from kwepemm600004.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4RFxtZ32bpzNmhK; Wed, 2 Aug 2023 11:15:18 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by kwepemm600004.china.huawei.com (7.193.23.242) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Wed, 2 Aug 2023 11:18:43 +0800 From: Huisong Li To: CC: , , , , , , Subject: [PATCH RESEND v6 4/5] app/testpmd: add attach and detach port for multiple process Date: Wed, 2 Aug 2023 11:15:59 +0800 Message-ID: <20230802031600.23631-5-lihuisong@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20230802031600.23631-1-lihuisong@huawei.com> References: <20220825024425.10534-1-lihuisong@huawei.com> <20230802031600.23631-1-lihuisong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemm600004.china.huawei.com (7.193.23.242) X-CFilter-Loop: Reflected X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The port information needs to be updated due to attaching and detaching port. Currently, it is done in the same thread as removing or probing device, which doesn't satisfy the operation of attaching and detaching device in multiple process. If this operation is performed in one process, the other process can receive 'new' or 'destroy' event. So we can move updating port information to event callback to support attaching and detaching port in primary and secondary process. The reason for adding an alarm callback in 'destroy' event is that the ethdev state is changed from 'ATTACHED' to 'UNUSED' only after the event callback finished. But the remove_invalid_ports() function removes invalid port only if ethdev state is 'UNUSED'. If we don't add alarm callback, this detached port information can not be removed. Signed-off-by: Huisong Li Signed-off-by: Dongdong Liu Acked-by: Chengwen Feng --- app/test-pmd/testpmd.c | 38 ++++++++++++++++----------- app/test-pmd/testpmd.h | 1 - drivers/net/bonding/bonding_testpmd.c | 1 - 3 files changed, 22 insertions(+), 18 deletions(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index f934f6b418..a5124c3231 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3610,15 +3610,12 @@ attach_port(char *identifier) return; } - /* first attach mode: event */ - if (setup_on_probe_event) { - /* new ports are detected on RTE_ETH_EVENT_NEW event */ - for (pi = 0; pi < RTE_MAX_ETHPORTS; pi++) - if (ports[pi].port_status == RTE_PORT_HANDLING && - ports[pi].need_setup != 0) - setup_attached_port(pi); + /* + * first attach mode: event, setting up attached port is done in + * probing callback. + */ + if (setup_on_probe_event) return; - } /* second attach mode: iterator */ RTE_ETH_FOREACH_MATCHING_DEV(pi, identifier, &iterator) { @@ -3649,7 +3646,6 @@ setup_attached_port(portid_t pi) ports_ids[nb_ports++] = pi; fwd_ports_ids[nb_fwd_ports++] = pi; nb_cfg_ports = nb_fwd_ports; - ports[pi].need_setup = 0; ports[pi].port_status = RTE_PORT_STOPPED; printf("Port %d is attached. Now total ports is %d\n", pi, nb_ports); @@ -3683,10 +3679,8 @@ detach_device(struct rte_device *dev) TESTPMD_LOG(ERR, "Failed to detach device %s\n", rte_dev_name(dev)); return; } - remove_invalid_ports(); printf("Device is detached\n"); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); return; } @@ -3753,11 +3747,9 @@ detach_devargs(char *identifier) return; } - remove_invalid_ports(); - printf("Device %s is detached\n", identifier); - printf("Now total ports is %d\n", nb_ports); printf("Done\n"); + rte_devargs_reset(&da); } @@ -3921,11 +3913,22 @@ rmv_port_callback(void *arg) struct rte_device *device = dev_info.device; close_port(port_id); detach_device(device); /* might be already removed or have more ports */ + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); } if (need_to_start) start_packet_forwarding(0); } +static void +remove_invalid_ports_callback(void *arg) +{ + RTE_SET_USED(arg); + + remove_invalid_ports(); + printf("Now total ports is %d\n", nb_ports); +} + /* This function is used by the interrupt thread */ static int eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, @@ -3950,8 +3953,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: - ports[port_id].need_setup = 1; - ports[port_id].port_status = RTE_PORT_HANDLING; + if (setup_on_probe_event) + setup_attached_port(port_id); break; case RTE_ETH_EVENT_INTR_RMV: if (rte_eal_alarm_set(100000, @@ -3962,6 +3965,9 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, case RTE_ETH_EVENT_DESTROY: ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); + if (rte_eal_alarm_set(100000, remove_invalid_ports_callback, + (void *)(intptr_t)port_id)) + fprintf(stderr, "Could not set up deferred device released\n"); break; case RTE_ETH_EVENT_RX_AVAIL_THRESH: { uint16_t rxq_id; diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index f1df6a8faf..daacdd7ee4 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -324,7 +324,6 @@ struct rte_port { uint16_t tx_vlan_id;/**< The tag ID */ uint16_t tx_vlan_id_outer;/**< The outer tag ID */ volatile uint16_t port_status; /**< port started or not */ - uint8_t need_setup; /**< port just attached */ uint8_t need_reconfig; /**< need reconfiguring port or not */ uint8_t need_reconfig_queues; /**< need reconfiguring queues or not */ uint8_t rss_flag; /**< enable rss or not */ diff --git a/drivers/net/bonding/bonding_testpmd.c b/drivers/net/bonding/bonding_testpmd.c index b3c12cada0..160f638e3a 100644 --- a/drivers/net/bonding/bonding_testpmd.c +++ b/drivers/net/bonding/bonding_testpmd.c @@ -493,7 +493,6 @@ static void cmd_create_bonded_device_parsed(void *parsed_result, ports[port_id].update_conf = 1; ports[port_id].bond_flag = 1; - ports[port_id].need_setup = 0; ports[port_id].port_status = RTE_PORT_STOPPED; } From patchwork Wed Aug 2 03:16:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "lihuisong (C)" X-Patchwork-Id: 129800 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9D2FD42FB3; Wed, 2 Aug 2023 05:18:51 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BDE8743253; Wed, 2 Aug 2023 05:18:47 +0200 (CEST) Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by mails.dpdk.org (Postfix) with ESMTP id 910E243251 for ; Wed, 2 Aug 2023 05:18:45 +0200 (CEST) Received: from kwepemm600004.china.huawei.com (unknown [172.30.72.56]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4RFxtg0G5hztRm7; Wed, 2 Aug 2023 11:15:23 +0800 (CST) Received: from localhost.localdomain (10.69.192.56) by kwepemm600004.china.huawei.com (7.193.23.242) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Wed, 2 Aug 2023 11:18:44 +0800 From: Huisong Li To: CC: , , , , , , Subject: [PATCH RESEND v6 5/5] app/testpmd: stop forwarding in new or destroy event Date: Wed, 2 Aug 2023 11:16:00 +0800 Message-ID: <20230802031600.23631-6-lihuisong@huawei.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20230802031600.23631-1-lihuisong@huawei.com> References: <20220825024425.10534-1-lihuisong@huawei.com> <20230802031600.23631-1-lihuisong@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.69.192.56] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemm600004.china.huawei.com (7.193.23.242) X-CFilter-Loop: Reflected X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org When testpmd receives the new or destroy event, the port related information will be updated. Testpmd must stop packet forwarding before updating the information to avoid some serious problems. Signed-off-by: Huisong Li Acked-by: Chengwen Feng --- app/test-pmd/testpmd.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index a5124c3231..15cbede69b 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3953,6 +3953,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, switch (type) { case RTE_ETH_EVENT_NEW: + if (test_done == 0) + stop_packet_forwarding(); if (setup_on_probe_event) setup_attached_port(port_id); break; @@ -3963,6 +3965,8 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, "Could not set up deferred device removal\n"); break; case RTE_ETH_EVENT_DESTROY: + if (test_done == 0) + stop_packet_forwarding(); ports[port_id].port_status = RTE_PORT_CLOSED; printf("Port %u is closed\n", port_id); if (rte_eal_alarm_set(100000, remove_invalid_ports_callback,