From patchwork Wed Mar 1 03:06:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: fengchengwen X-Patchwork-Id: 124600 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5676241DA4; Wed, 1 Mar 2023 04:12:59 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F1C8042B8B; Wed, 1 Mar 2023 04:12:45 +0100 (CET) Received: from szxga03-in.huawei.com (szxga03-in.huawei.com [45.249.212.189]) by mails.dpdk.org (Postfix) with ESMTP id 1C20341151 for ; Wed, 1 Mar 2023 04:12:41 +0100 (CET) Received: from dggpeml500024.china.huawei.com (unknown [172.30.72.57]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4PRK0r4gjTzKmZ8; Wed, 1 Mar 2023 11:07:40 +0800 (CST) Received: from localhost.localdomain (10.50.163.32) by dggpeml500024.china.huawei.com (7.185.36.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Wed, 1 Mar 2023 11:12:38 +0800 From: Chengwen Feng To: , , , Andrew Rybchenko , Kalesh AP , Ajit Khaparde CC: Subject: [PATCH 1/5] ethdev: fix race-condition of proactive error handling mode Date: Wed, 1 Mar 2023 03:06:06 +0000 Message-ID: <20230301030610.49468-2-fengchengwen@huawei.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230301030610.49468-1-fengchengwen@huawei.com> References: <20230301030610.49468-1-fengchengwen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.50.163.32] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpeml500024.china.huawei.com (7.185.36.10) X-CFilter-Loop: Reflected X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org In the proactive error handling mode, the PMD will set the data path pointers to dummy functions and then try recovery, in this period the application may still invoking data path API. This will introduce a race-condition with data path which may lead to crash [1]. Although the PMD added delay after setting data path pointers to cover the above race-condition, it reduces the probability, but it doesn't solve the problem. To solve the race-condition problem fundamentally, the following requirements are added: 1. The PMD should set the data path pointers to dummy functions after report RTE_ETH_EVENT_ERR_RECOVERING event. 2. The application should stop data path API invocation when process the RTE_ETH_EVENT_ERR_RECOVERING event. 3. The PMD should set the data path pointers to valid functions before report RTE_ETH_EVENT_RECOVERY_SUCCESS event. 4. The application should enable data path API invocation when process the RTE_ETH_EVENT_RECOVERY_SUCCESS event. Also, this patch introduce a driver internal function rte_eth_fp_ops_setup which used as an help function for PMD. [1] http://patchwork.dpdk.org/project/dpdk/patch/20230220060839.1267349-2-ashok.k.kaladi@intel.com/ Fixes: eb0d471a8941 ("ethdev: add proactive error handling mode") Cc: stable@dpdk.org Signed-off-by: Chengwen Feng Acked-by: Konstantin Ananyev --- doc/guides/prog_guide/poll_mode_drv.rst | 20 +++++++--------- lib/ethdev/ethdev_driver.c | 8 +++++++ lib/ethdev/ethdev_driver.h | 10 ++++++++ lib/ethdev/rte_ethdev.h | 32 +++++++++++++++---------- lib/ethdev/version.map | 1 + 5 files changed, 46 insertions(+), 25 deletions(-) diff --git a/doc/guides/prog_guide/poll_mode_drv.rst b/doc/guides/prog_guide/poll_mode_drv.rst index c145a9066c..e380ff135a 100644 --- a/doc/guides/prog_guide/poll_mode_drv.rst +++ b/doc/guides/prog_guide/poll_mode_drv.rst @@ -638,14 +638,9 @@ different from the application invokes recovery in PASSIVE mode, the PMD automatically recovers from error in PROACTIVE mode, and only a small amount of work is required for the application. -During error detection and automatic recovery, -the PMD sets the data path pointers to dummy functions -(which will prevent the crash), -and also make sure the control path operations fail with a return code ``-EBUSY``. - -Because the PMD recovers automatically, -the application can only sense that the data flow is disconnected for a while -and the control API returns an error in this period. +During error detection and automatic recovery, the PMD sets the data path +pointers to dummy functions and also make sure the control path operations +failed with a return code ``-EBUSY``. In order to sense the error happening/recovering, as well as to restore some additional configuration, @@ -653,9 +648,9 @@ three events are available: ``RTE_ETH_EVENT_ERR_RECOVERING`` Notify the application that an error is detected - and the recovery is being started. + and the recovery is about to start. Upon receiving the event, the application should not invoke - any control path function until receiving + any control and data path API until receiving ``RTE_ETH_EVENT_RECOVERY_SUCCESS`` or ``RTE_ETH_EVENT_RECOVERY_FAILED`` event. .. note:: @@ -666,8 +661,9 @@ three events are available: ``RTE_ETH_EVENT_RECOVERY_SUCCESS`` Notify the application that the recovery from error is successful, - the PMD already re-configures the port, - and the effect is the same as a restart operation. + the PMD already re-configures the port. + The application should restore some additional configuration, and then + enable data path API invocation. ``RTE_ETH_EVENT_RECOVERY_FAILED`` Notify the application that the recovery from error failed, diff --git a/lib/ethdev/ethdev_driver.c b/lib/ethdev/ethdev_driver.c index 0be1e8ca04..f994653fe9 100644 --- a/lib/ethdev/ethdev_driver.c +++ b/lib/ethdev/ethdev_driver.c @@ -515,6 +515,14 @@ rte_eth_dma_zone_free(const struct rte_eth_dev *dev, const char *ring_name, return rc; } +void +rte_eth_fp_ops_setup(struct rte_eth_dev *dev) +{ + if (dev == NULL) + return; + eth_dev_fp_ops_setup(rte_eth_fp_ops + dev->data->port_id, dev); +} + const struct rte_memzone * rte_eth_dma_zone_reserve(const struct rte_eth_dev *dev, const char *ring_name, uint16_t queue_id, size_t size, unsigned int align, diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 2c9d615fb5..0d964d1f67 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1621,6 +1621,16 @@ int rte_eth_dma_zone_free(const struct rte_eth_dev *eth_dev, const char *name, uint16_t queue_id); +/** + * @internal + * Setup eth fast-path API to ethdev values. + * + * @param dev + * Pointer to struct rte_eth_dev. + */ +__rte_internal +void rte_eth_fp_ops_setup(struct rte_eth_dev *dev); + /** * @internal * Atomically set the link status for the specific device. diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index 049641d57c..44ee7229c1 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -3944,25 +3944,28 @@ enum rte_eth_event_type { */ RTE_ETH_EVENT_RX_AVAIL_THRESH, /** Port recovering from a hardware or firmware error. - * If PMD supports proactive error recovery, - * it should trigger this event to notify application - * that it detected an error and the recovery is being started. - * Upon receiving the event, the application should not invoke any control path API - * (such as rte_eth_dev_configure/rte_eth_dev_stop...) until receiving - * RTE_ETH_EVENT_RECOVERY_SUCCESS or RTE_ETH_EVENT_RECOVERY_FAILED event. - * The PMD will set the data path pointers to dummy functions, - * and re-set the data path pointers to non-dummy functions - * before reporting RTE_ETH_EVENT_RECOVERY_SUCCESS event. - * It means that the application cannot send or receive any packets - * during this period. + * + * If PMD supports proactive error recovery, it should trigger this + * event to notify application that it detected an error and the + * recovery is about to start. + * + * Upon receiving the event, the application should not invoke any + * control and data path API until receiving + * RTE_ETH_EVENT_RECOVERY_SUCCESS or RTE_ETH_EVENT_RECOVERY_FAILED + * event. + * + * Once this event is reported, the PMD will set the data path pointers + * to dummy functions, and re-set the data path pointers to valid + * functions before reporting RTE_ETH_EVENT_RECOVERY_SUCCESS event. + * * @note Before the PMD reports the recovery result, * the PMD may report the RTE_ETH_EVENT_ERR_RECOVERING event again, * because a larger error may occur during the recovery. */ RTE_ETH_EVENT_ERR_RECOVERING, /** Port recovers successfully from the error. - * The PMD already re-configured the port, - * and the effect is the same as a restart operation. + * + * The PMD already re-configured the port: * a) The following operation will be retained: (alphabetically) * - DCB configuration * - FEC configuration @@ -3989,6 +3992,9 @@ enum rte_eth_event_type { * (@see RTE_ETH_DEV_CAPA_FLOW_SHARED_OBJECT_KEEP) * c) Any other configuration will not be stored * and will need to be re-configured. + * + * The application should restore some additional configuration + * (see above case b/c), and then enable data path API invocation. */ RTE_ETH_EVENT_RECOVERY_SUCCESS, /** Port recovery failed. diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index 357d1a88c0..c273e0bdae 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -320,6 +320,7 @@ INTERNAL { rte_eth_devices; rte_eth_dma_zone_free; rte_eth_dma_zone_reserve; + rte_eth_fp_ops_setup; rte_eth_hairpin_queue_peer_bind; rte_eth_hairpin_queue_peer_unbind; rte_eth_hairpin_queue_peer_update; From patchwork Wed Mar 1 03:06:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: fengchengwen X-Patchwork-Id: 124597 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D9D4F41DA4; Wed, 1 Mar 2023 04:12:42 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B711D41151; Wed, 1 Mar 2023 04:12:42 +0100 (CET) Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by mails.dpdk.org (Postfix) with ESMTP id E5AFB4067B for ; Wed, 1 Mar 2023 04:12:40 +0100 (CET) Received: from dggpeml500024.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4PRK5r37lvzrRwH; Wed, 1 Mar 2023 11:12:00 +0800 (CST) Received: from localhost.localdomain (10.50.163.32) by dggpeml500024.china.huawei.com (7.185.36.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Wed, 1 Mar 2023 11:12:39 +0800 From: Chengwen Feng To: , , , Dongdong Liu , Yisen Zhuang CC: Subject: [PATCH 2/5] net/hns3: replace fp ops config function Date: Wed, 1 Mar 2023 03:06:07 +0000 Message-ID: <20230301030610.49468-3-fengchengwen@huawei.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230301030610.49468-1-fengchengwen@huawei.com> References: <20230301030610.49468-1-fengchengwen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.50.163.32] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpeml500024.china.huawei.com (7.185.36.10) X-CFilter-Loop: Reflected X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch replace hns3_eth_dev_fp_ops_config() with rte_eth_fp_ops_setup(). Cc: stable@dpdk.org Signed-off-by: Chengwen Feng Acked-by: Dongdong Liu --- drivers/net/hns3/hns3_rxtx.c | 21 +++------------------ 1 file changed, 3 insertions(+), 18 deletions(-) diff --git a/drivers/net/hns3/hns3_rxtx.c b/drivers/net/hns3/hns3_rxtx.c index 4065c519c3..6d02b4ee9f 100644 --- a/drivers/net/hns3/hns3_rxtx.c +++ b/drivers/net/hns3/hns3_rxtx.c @@ -4382,21 +4382,6 @@ hns3_trace_rxtx_function(struct rte_eth_dev *dev) rx_mode.info, tx_mode.info); } -static void -hns3_eth_dev_fp_ops_config(const struct rte_eth_dev *dev) -{ - struct rte_eth_fp_ops *fpo = rte_eth_fp_ops; - uint16_t port_id = dev->data->port_id; - - fpo[port_id].rx_pkt_burst = dev->rx_pkt_burst; - fpo[port_id].tx_pkt_burst = dev->tx_pkt_burst; - fpo[port_id].tx_pkt_prepare = dev->tx_pkt_prepare; - fpo[port_id].rx_descriptor_status = dev->rx_descriptor_status; - fpo[port_id].tx_descriptor_status = dev->tx_descriptor_status; - fpo[port_id].rxq.data = dev->data->rx_queues; - fpo[port_id].txq.data = dev->data->tx_queues; -} - void hns3_set_rxtx_function(struct rte_eth_dev *eth_dev) { @@ -4419,7 +4404,7 @@ hns3_set_rxtx_function(struct rte_eth_dev *eth_dev) } hns3_trace_rxtx_function(eth_dev); - hns3_eth_dev_fp_ops_config(eth_dev); + rte_eth_fp_ops_setup(eth_dev); } void @@ -4741,7 +4726,7 @@ hns3_stop_tx_datapath(struct rte_eth_dev *dev) { dev->tx_pkt_burst = rte_eth_pkt_burst_dummy; dev->tx_pkt_prepare = NULL; - hns3_eth_dev_fp_ops_config(dev); + rte_eth_fp_ops_setup(dev); if (rte_eal_process_type() == RTE_PROC_SECONDARY) return; @@ -4758,7 +4743,7 @@ hns3_start_tx_datapath(struct rte_eth_dev *dev) { dev->tx_pkt_burst = hns3_get_tx_function(dev); dev->tx_pkt_prepare = hns3_get_tx_prepare(dev); - hns3_eth_dev_fp_ops_config(dev); + rte_eth_fp_ops_setup(dev); if (rte_eal_process_type() == RTE_PROC_SECONDARY) return; From patchwork Wed Mar 1 03:06:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: fengchengwen X-Patchwork-Id: 124601 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AAE1341DA4; Wed, 1 Mar 2023 04:13:04 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0581D42BD9; Wed, 1 Mar 2023 04:12:47 +0100 (CET) Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by mails.dpdk.org (Postfix) with ESMTP id 8C8784067B for ; Wed, 1 Mar 2023 04:12:42 +0100 (CET) Received: from dggpeml500024.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4PRK3F4YdKzSkVy; Wed, 1 Mar 2023 11:09:45 +0800 (CST) Received: from localhost.localdomain (10.50.163.32) by dggpeml500024.china.huawei.com (7.185.36.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Wed, 1 Mar 2023 11:12:39 +0800 From: Chengwen Feng To: , , , Ajit Khaparde , Somnath Kotur , Kalesh AP CC: Subject: [PATCH 3/5] net/bnxt: fix race-condition when report error recovery Date: Wed, 1 Mar 2023 03:06:08 +0000 Message-ID: <20230301030610.49468-4-fengchengwen@huawei.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230301030610.49468-1-fengchengwen@huawei.com> References: <20230301030610.49468-1-fengchengwen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.50.163.32] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpeml500024.china.huawei.com (7.185.36.10) X-CFilter-Loop: Reflected X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org If set data path functions to dummy functions before reports error recovering event, there maybe a race-condition with data path threads, this patch fixes it by setting data path functions to dummy functions only after reports such event. Fixes: e11052f3a46f ("net/bnxt: support proactive error handling mode") Cc: stable@dpdk.org Signed-off-by: Chengwen Feng Acked-by: Konstantin Ananyev --- drivers/net/bnxt/bnxt_cpr.c | 13 +++++++------ drivers/net/bnxt/bnxt_ethdev.c | 4 ++-- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/drivers/net/bnxt/bnxt_cpr.c b/drivers/net/bnxt/bnxt_cpr.c index 5bb376d4d5..3950840600 100644 --- a/drivers/net/bnxt/bnxt_cpr.c +++ b/drivers/net/bnxt/bnxt_cpr.c @@ -168,14 +168,9 @@ void bnxt_handle_async_event(struct bnxt *bp, PMD_DRV_LOG(INFO, "Port conn async event\n"); break; case HWRM_ASYNC_EVENT_CMPL_EVENT_ID_RESET_NOTIFY: - /* - * Avoid any rx/tx packet processing during firmware reset - * operation. - */ - bnxt_stop_rxtx(bp->eth_dev); - /* Ignore reset notify async events when stopping the port */ if (!bp->eth_dev->data->dev_started) { + bnxt_stop_rxtx(bp->eth_dev); bp->flags |= BNXT_FLAG_FATAL_ERROR; return; } @@ -184,6 +179,12 @@ void bnxt_handle_async_event(struct bnxt *bp, RTE_ETH_EVENT_ERR_RECOVERING, NULL); + /* + * Avoid any rx/tx packet processing during firmware reset + * operation. + */ + bnxt_stop_rxtx(bp->eth_dev); + pthread_mutex_lock(&bp->err_recovery_lock); event_data = data1; /* timestamp_lo/hi values are in units of 100ms */ diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index 753e86b4b2..4083a69d02 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -4562,14 +4562,14 @@ static void bnxt_check_fw_health(void *arg) bp->flags |= BNXT_FLAG_FATAL_ERROR; bp->flags |= BNXT_FLAG_FW_RESET; - bnxt_stop_rxtx(bp->eth_dev); - PMD_DRV_LOG(ERR, "Detected FW dead condition\n"); rte_eth_dev_callback_process(bp->eth_dev, RTE_ETH_EVENT_ERR_RECOVERING, NULL); + bnxt_stop_rxtx(bp->eth_dev); + if (bnxt_is_primary_func(bp)) wait_msec = info->primary_func_wait_period; else From patchwork Wed Mar 1 03:06:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: fengchengwen X-Patchwork-Id: 124599 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 90C4D41DA4; Wed, 1 Mar 2023 04:12:53 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id DB7F0427F2; Wed, 1 Mar 2023 04:12:44 +0100 (CET) Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by mails.dpdk.org (Postfix) with ESMTP id 11E4E4114A for ; Wed, 1 Mar 2023 04:12:41 +0100 (CET) Received: from dggpeml500024.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4PRK3X3vTQznWLS; Wed, 1 Mar 2023 11:10:00 +0800 (CST) Received: from localhost.localdomain (10.50.163.32) by dggpeml500024.china.huawei.com (7.185.36.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Wed, 1 Mar 2023 11:12:39 +0800 From: Chengwen Feng To: , , , Ajit Khaparde , Somnath Kotur CC: Subject: [PATCH 4/5] net/bnxt: use fp ops setup function Date: Wed, 1 Mar 2023 03:06:09 +0000 Message-ID: <20230301030610.49468-5-fengchengwen@huawei.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230301030610.49468-1-fengchengwen@huawei.com> References: <20230301030610.49468-1-fengchengwen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.50.163.32] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpeml500024.china.huawei.com (7.185.36.10) X-CFilter-Loop: Reflected X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Use rte_eth_fp_ops_setup() instead of directly manipulating rte_eth_fp_ops variable. Cc: stable@dpdk.org Signed-off-by: Chengwen Feng --- drivers/net/bnxt/bnxt_cpr.c | 5 +---- drivers/net/bnxt/bnxt_ethdev.c | 5 +---- 2 files changed, 2 insertions(+), 8 deletions(-) diff --git a/drivers/net/bnxt/bnxt_cpr.c b/drivers/net/bnxt/bnxt_cpr.c index 3950840600..a3f33c24c3 100644 --- a/drivers/net/bnxt/bnxt_cpr.c +++ b/drivers/net/bnxt/bnxt_cpr.c @@ -416,10 +416,7 @@ void bnxt_stop_rxtx(struct rte_eth_dev *eth_dev) eth_dev->rx_pkt_burst = rte_eth_pkt_burst_dummy; eth_dev->tx_pkt_burst = rte_eth_pkt_burst_dummy; - rte_eth_fp_ops[eth_dev->data->port_id].rx_pkt_burst = - eth_dev->rx_pkt_burst; - rte_eth_fp_ops[eth_dev->data->port_id].tx_pkt_burst = - eth_dev->tx_pkt_burst; + rte_eth_fp_ops_setup(eth_dev); rte_mb(); /* Allow time for threads to exit the real burst functions. */ diff --git a/drivers/net/bnxt/bnxt_ethdev.c b/drivers/net/bnxt/bnxt_ethdev.c index 4083a69d02..d6064ceea4 100644 --- a/drivers/net/bnxt/bnxt_ethdev.c +++ b/drivers/net/bnxt/bnxt_ethdev.c @@ -4374,10 +4374,7 @@ static void bnxt_dev_recover(void *arg) if (rc) goto err_start; - rte_eth_fp_ops[bp->eth_dev->data->port_id].rx_pkt_burst = - bp->eth_dev->rx_pkt_burst; - rte_eth_fp_ops[bp->eth_dev->data->port_id].tx_pkt_burst = - bp->eth_dev->tx_pkt_burst; + rte_eth_fp_ops_setup(bp->eth_dev); rte_mb(); PMD_DRV_LOG(INFO, "Port: %u Recovered from FW reset\n", From patchwork Wed Mar 1 03:06:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: fengchengwen X-Patchwork-Id: 124602 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 18EF541DA4; Wed, 1 Mar 2023 04:13:10 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0FBAC42C24; Wed, 1 Mar 2023 04:12:48 +0100 (CET) Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by mails.dpdk.org (Postfix) with ESMTP id 9FA8640A8A for ; Wed, 1 Mar 2023 04:12:42 +0100 (CET) Received: from dggpeml500024.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4PRK3Y3zB2z16Nvm; Wed, 1 Mar 2023 11:10:01 +0800 (CST) Received: from localhost.localdomain (10.50.163.32) by dggpeml500024.china.huawei.com (7.185.36.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Wed, 1 Mar 2023 11:12:39 +0800 From: Chengwen Feng To: , , , Aman Singh , Yuying Zhang CC: Subject: [PATCH 5/5] app/testpmd: add error recovery usage demo Date: Wed, 1 Mar 2023 03:06:10 +0000 Message-ID: <20230301030610.49468-6-fengchengwen@huawei.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230301030610.49468-1-fengchengwen@huawei.com> References: <20230301030610.49468-1-fengchengwen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.50.163.32] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpeml500024.china.huawei.com (7.185.36.10) X-CFilter-Loop: Reflected X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds error recovery usage demo which will: 1. stop packet forwarding when the RTE_ETH_EVENT_ERR_RECOVERING event is received. 2. restart packet forwarding when the RTE_ETH_EVENT_RECOVERY_SUCCESS event is received. 3. prompt the ports that fail to recovery and need to be removed when the RTE_ETH_EVENT_RECOVERY_FAILED event is received. In addition, a message is added to the printed information, requiring no command to be executed during the error recovery. Signed-off-by: Chengwen Feng Acked-by: Konstantin Ananyev --- app/test-pmd/testpmd.c | 80 ++++++++++++++++++++++++++++++++++++++++++ app/test-pmd/testpmd.h | 4 ++- 2 files changed, 83 insertions(+), 1 deletion(-) diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c index 0c14325b8d..fdc3ae604b 100644 --- a/app/test-pmd/testpmd.c +++ b/app/test-pmd/testpmd.c @@ -3823,6 +3823,77 @@ rmv_port_callback(void *arg) start_packet_forwarding(0); } +static int need_start_when_recovery_over; + +static bool +has_port_in_err_recovering(void) +{ + struct rte_port *port; + portid_t pid; + + RTE_ETH_FOREACH_DEV(pid) { + port = &ports[pid]; + if (port->err_recovering) + return true; + } + + return false; +} + +static void +err_recovering_callback(portid_t port_id) +{ + if (!has_port_in_err_recovering()) + printf("Please stop executing any commands until recovery result events are received!\n"); + + ports[port_id].err_recovering = 1; + ports[port_id].recover_failed = 0; + + /* To simplify implementation, stop forwarding regardless of whether the port is used. */ + if (!test_done) { + printf("Stop packet forwarding because some ports are in error recovering!\n"); + stop_packet_forwarding(); + need_start_when_recovery_over = 1; + } +} + +static void +recover_success_callback(portid_t port_id) +{ + ports[port_id].err_recovering = 0; + if (has_port_in_err_recovering()) + return; + + if (need_start_when_recovery_over) { + printf("Recovery success! Restart packet forwarding!\n"); + start_packet_forwarding(0); + need_start_when_recovery_over = 0; + } else { + printf("Recovery success!\n"); + } +} + +static void +recover_failed_callback(portid_t port_id) +{ + struct rte_port *port; + portid_t pid; + + ports[port_id].err_recovering = 0; + ports[port_id].recover_failed = 1; + if (has_port_in_err_recovering()) + return; + + need_start_when_recovery_over = 0; + printf("The ports:"); + RTE_ETH_FOREACH_DEV(pid) { + port = &ports[pid]; + if (port->recover_failed) + printf(" %u", pid); + } + printf(" recovery failed! Please remove them!\n"); +} + /* This function is used by the interrupt thread */ static int eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, @@ -3878,6 +3949,15 @@ eth_event_callback(portid_t port_id, enum rte_eth_event_type type, void *param, } break; } + case RTE_ETH_EVENT_ERR_RECOVERING: + err_recovering_callback(port_id); + break; + case RTE_ETH_EVENT_RECOVERY_SUCCESS: + recover_success_callback(port_id); + break; + case RTE_ETH_EVENT_RECOVERY_FAILED: + recover_failed_callback(port_id); + break; default: break; } diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h index 329a6378a1..1bbf82a96c 100644 --- a/app/test-pmd/testpmd.h +++ b/app/test-pmd/testpmd.h @@ -323,7 +323,9 @@ struct rte_port { uint8_t slave_flag : 1, /**< bonding slave port */ bond_flag : 1, /**< port is bond device */ fwd_mac_swap : 1, /**< swap packet MAC before forward */ - update_conf : 1; /**< need to update bonding device configuration */ + update_conf : 1, /**< need to update bonding device configuration */ + err_recovering : 1, /**< port is in error recovering */ + recover_failed : 1; /**< port recover failed */ struct port_template *pattern_templ_list; /**< Pattern templates. */ struct port_template *actions_templ_list; /**< Actions templates. */ struct port_table *table_list; /**< Flow tables. */