From patchwork Thu May 6 03:49:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Haiyue" X-Patchwork-Id: 92982 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A8501A0524; Thu, 6 May 2021 06:10:26 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 799ED410F6; Thu, 6 May 2021 06:10:21 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id 3435B410DB for ; Thu, 6 May 2021 06:10:19 +0200 (CEST) IronPort-SDR: n4VUA9d+WcEZ2YzdZeAUA4+jDDnv5EYVfXBaQplHEru9bShK/M4N4z8jmMvoKGXfzWRcekFRGb SYBP66SaLBXA== X-IronPort-AV: E=McAfee;i="6200,9189,9975"; a="195251306" X-IronPort-AV: E=Sophos;i="5.82,276,1613462400"; d="scan'208";a="195251306" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2021 21:10:18 -0700 IronPort-SDR: FTXkuCHDT+6oLTvnk5qpqX5OUQGDgUOYuea+1eeiWi74B/zoIyHCy3jcZbvOzBbxz0thQFu4kU nLHx12avGjng== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,276,1613462400"; d="scan'208";a="434147068" Received: from npg-dpdk-haiyue-1.sh.intel.com ([10.67.118.220]) by orsmga008.jf.intel.com with ESMTP; 05 May 2021 21:10:15 -0700 From: Haiyue Wang To: dev@dpdk.org Cc: qi.z.zhang@intel.com, liang-min.wang@intel.com, david.marchand@redhat.com, Haiyue Wang , Ray Kinsella , Neil Horman , Gaetan Rivet Date: Thu, 6 May 2021 11:49:55 +0800 Message-Id: <20210506034957.46458-2-haiyue.wang@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210506034957.46458-1-haiyue.wang@intel.com> References: <20210421050243.130585-1-haiyue.wang@intel.com> <20210506034957.46458-1-haiyue.wang@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 1/3] bus/pci: set PCI master in command register X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add the API to set 'Bus Master Enable' bit to be enabled or disabled in the PCI command register. Signed-off-by: Haiyue Wang Acked-by: Ray Kinsella --- drivers/bus/pci/pci_common.c | 28 ++++++++++++++++++++++++++++ drivers/bus/pci/rte_bus_pci.h | 14 ++++++++++++++ drivers/bus/pci/version.map | 3 +++ lib/pci/rte_pci.h | 4 ++++ 4 files changed, 49 insertions(+) diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c index ee7f96635..35d7d092d 100644 --- a/drivers/bus/pci/pci_common.c +++ b/drivers/bus/pci/pci_common.c @@ -746,6 +746,34 @@ rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap) return 0; } +int +rte_pci_set_bus_master(struct rte_pci_device *dev, bool enable) +{ + uint16_t old_cmd, cmd; + + if (rte_pci_read_config(dev, &old_cmd, sizeof(old_cmd), + RTE_PCI_COMMAND) < 0) { + RTE_LOG(ERR, EAL, "error in reading PCI command register\n"); + return -1; + } + + if (enable) + cmd = old_cmd | RTE_PCI_COMMAND_MASTER; + else + cmd = old_cmd & ~RTE_PCI_COMMAND_MASTER; + + if (cmd == old_cmd) + return 0; + + if (rte_pci_write_config(dev, &cmd, sizeof(cmd), + RTE_PCI_COMMAND) < 0) { + RTE_LOG(ERR, EAL, "error in writing PCI command register\n"); + return -1; + } + + return 0; +} + struct rte_pci_bus rte_pci_bus = { .bus = { .scan = rte_pci_scan, diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h index 64886b473..976c33c92 100644 --- a/drivers/bus/pci/rte_bus_pci.h +++ b/drivers/bus/pci/rte_bus_pci.h @@ -249,6 +249,20 @@ void rte_pci_dump(FILE *f); __rte_experimental off_t rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap); +/** + * Enables/Disables Bus Master for device's PCI command register. + * + * @param dev + * A pointer to rte_pci_device structure. + * @param enable + * Enable or disable Bus Master. + * + * @return + * 0 on success, -1 on error in PCI config space read/write. + */ +__rte_experimental +int rte_pci_set_bus_master(struct rte_pci_device *dev, bool enable); + /** * Register a PCI driver. * diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map index f33ed0abd..c6e5f797c 100644 --- a/drivers/bus/pci/version.map +++ b/drivers/bus/pci/version.map @@ -21,4 +21,7 @@ EXPERIMENTAL { global: rte_pci_find_ext_capability; + + # added in 21.05 + rte_pci_set_bus_master; }; diff --git a/lib/pci/rte_pci.h b/lib/pci/rte_pci.h index a8f8e404a..1f33d687f 100644 --- a/lib/pci/rte_pci.h +++ b/lib/pci/rte_pci.h @@ -32,6 +32,10 @@ extern "C" { #define RTE_PCI_VENDOR_ID 0x00 /* 16 bits */ #define RTE_PCI_DEVICE_ID 0x02 /* 16 bits */ +#define RTE_PCI_COMMAND 0x04 /* 16 bits */ + +/* PCI Command Register */ +#define RTE_PCI_COMMAND_MASTER 0x4 /* Bus Master Enable */ /* PCI Express capability registers */ #define RTE_PCI_EXP_DEVCTL 8 /* Device Control */ From patchwork Thu May 6 03:49:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Haiyue" X-Patchwork-Id: 92983 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5D4EFA0524; Thu, 6 May 2021 06:10:32 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 99595410FE; Thu, 6 May 2021 06:10:22 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id DE1C0410F3 for ; Thu, 6 May 2021 06:10:20 +0200 (CEST) IronPort-SDR: SrHm3pUFa+8Lt+ullg2DbG/4Gj70k02nGz4z9F+doARxbDa0haArHTwuMil38Wig//YIAl/MCD xEIAF2zXRsng== X-IronPort-AV: E=McAfee;i="6200,9189,9975"; a="195251311" X-IronPort-AV: E=Sophos;i="5.82,276,1613462400"; d="scan'208";a="195251311" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2021 21:10:20 -0700 IronPort-SDR: TQ0EbJw8ALwbDmMp50F/b7u4qNgsdmrsm0MmOMNxelFdFd6CHFiUFQReBkgex3RQk22SzXPdYt kUvf1bfz5QsA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,276,1613462400"; d="scan'208";a="434147085" Received: from npg-dpdk-haiyue-1.sh.intel.com ([10.67.118.220]) by orsmga008.jf.intel.com with ESMTP; 05 May 2021 21:10:18 -0700 From: Haiyue Wang To: dev@dpdk.org Cc: qi.z.zhang@intel.com, liang-min.wang@intel.com, david.marchand@redhat.com, Haiyue Wang , Jingjing Wu , Beilei Xing Date: Thu, 6 May 2021 11:49:56 +0800 Message-Id: <20210506034957.46458-3-haiyue.wang@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210506034957.46458-1-haiyue.wang@intel.com> References: <20210421050243.130585-1-haiyue.wang@intel.com> <20210506034957.46458-1-haiyue.wang@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 2/3] net/iavf: enable PCI bus master after reset X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The VF reset can be triggerred by the PF reset event, in this case, the PCI bus master will be cleared, then the VF is not allowed to issue any Memory or I/O Requests. So after the reset event is detected, always enable the PCI bus master. And if failed, the device or system may be in an invalid state, so keep the VF reset state to mark it as I/O error. Signed-off-by: Haiyue Wang --- drivers/net/iavf/iavf_ethdev.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c index c06873d26..0084a083b 100644 --- a/drivers/net/iavf/iavf_ethdev.c +++ b/drivers/net/iavf/iavf_ethdev.c @@ -2356,7 +2356,8 @@ iavf_dev_close(struct rte_eth_dev *dev) rte_free(vf->aq_resp); vf->aq_resp = NULL; - vf->vf_reset = false; + if (vf->vf_reset && !rte_pci_set_bus_master(pci_dev, true)) + vf->vf_reset = false; return ret; } From patchwork Thu May 6 03:49:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Wang, Haiyue" X-Patchwork-Id: 92984 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 062C9A0524; Thu, 6 May 2021 06:10:38 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BFFE241106; Thu, 6 May 2021 06:10:23 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id 56408410FB for ; Thu, 6 May 2021 06:10:22 +0200 (CEST) IronPort-SDR: 7z7KI/NQA4H3O/H/RkRML7iCOXGtYpIdMYvkDqCZjcqvilly28tE5UqJl1ErMKlHcKpscL700P oJfjufullMBg== X-IronPort-AV: E=McAfee;i="6200,9189,9975"; a="195251313" X-IronPort-AV: E=Sophos;i="5.82,276,1613462400"; d="scan'208";a="195251313" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2021 21:10:21 -0700 IronPort-SDR: l9++MPzEbac/vmdN8uydr6C5LxCmONa9h7KykaF/YoUEy7hm9Pg2BtliW1NKSOvqRYe6dUZWb9 XYbvsJYD6FWw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,276,1613462400"; d="scan'208";a="434147104" Received: from npg-dpdk-haiyue-1.sh.intel.com ([10.67.118.220]) by orsmga008.jf.intel.com with ESMTP; 05 May 2021 21:10:20 -0700 From: Haiyue Wang To: dev@dpdk.org Cc: qi.z.zhang@intel.com, liang-min.wang@intel.com, david.marchand@redhat.com, Haiyue Wang , Beilei Xing Date: Thu, 6 May 2021 11:49:57 +0800 Message-Id: <20210506034957.46458-4-haiyue.wang@intel.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210506034957.46458-1-haiyue.wang@intel.com> References: <20210421050243.130585-1-haiyue.wang@intel.com> <20210506034957.46458-1-haiyue.wang@intel.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v5 3/3] net/i40e: enable PCI bus master after reset X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The VF reset can be triggerred by the PF reset event, in this case, the PCI bus master will be cleared, then the VF is not allowed to issue any Memory or I/O Requests. So after the reset event is detected, always enable the PCI bus master. And if failed, the device or system may be in an invalid state, so keep the VF reset state to mark it as I/O error. Signed-off-by: Haiyue Wang --- drivers/net/i40e/i40e_ethdev_vf.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c index cb898bdb6..bded64842 100644 --- a/drivers/net/i40e/i40e_ethdev_vf.c +++ b/drivers/net/i40e/i40e_ethdev_vf.c @@ -1213,7 +1213,6 @@ i40evf_check_vf_reset_done(struct rte_eth_dev *dev) if (i >= MAX_RESET_WAIT_CNT) return -1; - vf->vf_reset = false; vf->pend_msg &= ~PFMSG_RESET_IMPENDING; return 0; @@ -1392,6 +1391,7 @@ i40evf_handle_pf_event(struct rte_eth_dev *dev, uint8_t *msg, switch (pf_msg->event) { case VIRTCHNL_EVENT_RESET_IMPENDING: PMD_DRV_LOG(DEBUG, "VIRTCHNL_EVENT_RESET_IMPENDING event"); + vf->vf_reset = true; rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_RESET, NULL); break; @@ -2490,6 +2490,10 @@ i40evf_dev_close(struct rte_eth_dev *dev) i40e_shutdown_adminq(hw); i40evf_disable_irq0(hw); + if (vf->vf_reset && + !rte_pci_set_bus_master(RTE_ETH_DEV_TO_PCI(dev), true)) + vf->vf_reset = false; + rte_free(vf->vf_res); vf->vf_res = NULL; rte_free(vf->aq_resp);