From patchwork Fri Nov 3 22:03:34 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ferruh Yigit X-Patchwork-Id: 31168 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7DEF61B6BA; Fri, 3 Nov 2017 23:03:40 +0100 (CET) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id D61FB1B6A3; Fri, 3 Nov 2017 23:03:38 +0100 (CET) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga104.jf.intel.com with ESMTP; 03 Nov 2017 15:03:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.44,339,1505804400"; d="scan'208"; a="1239518333" Received: from fyigit-mobl1.ger.corp.intel.com (HELO [10.241.225.48]) ([10.241.225.48]) by fmsmga002.fm.intel.com with ESMTP; 03 Nov 2017 15:03:34 -0700 To: "Roberts, Lee A." , "Tan, Jianfeng" , Thomas Monjalon Cc: "dev@dpdk.org" , "stable@dpdk.org" , "Wu, Jingjing" , Shijith Thotton , Gregory Etelson , Harish Patil , George Prekas , "Gonzalez Monroy, Sergio" , Rasesh Mody References: <20171103003005.44339-1-ferruh.yigit@intel.com> <20171103005100.44633-1-ferruh.yigit@intel.com> From: Ferruh Yigit Message-ID: Date: Fri, 3 Nov 2017 15:03:34 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Subject: Re: [dpdk-dev] [PATCH v2] igb_uio: add config option to control reset X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 11/3/2017 12:42 PM, Roberts, Lee A. wrote: > > >> -----Original Message----- >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Tan, Jianfeng >> Sent: Thursday, November 02, 2017 8:57 PM >> To: Ferruh Yigit ; Thomas Monjalon >> Cc: dev@dpdk.org; stable@dpdk.org; Jingjing Wu ; Shijith Thotton >> ; Gregory Etelson ; Harish Patil >> ; George Prekas ; Sergio Gonzalez Monroy >> ; Rasesh Mody >> Subject: Re: [dpdk-dev] [PATCH v2] igb_uio: add config option to control reset >> >> >> >> On 11/3/2017 8:51 AM, Ferruh Yigit wrote: >>> Adding a compile time configuration option to control device reset done >>> during DPDK application exit. >>> >>> Config option is CONFIG_RTE_EAL_IGB_UIO_RESET and enabled by default, >>> so by default reset will happen. Having this reset is safer to be sure >>> device left in a proper case. >>> >>> But for special cases [1] it is possible to disable the config option >>> to prevent the device reset. >>> >>> [1] >>> http://dpdk.org/ml/archives/dev/2017-November/080927.html >>> >>> Fixes: b58eedfc7dd5 ("igb_uio: issue FLR during open and release of device file") >>> Cc: stable@dpdk.org >>> >>> Signed-off-by: Ferruh Yigit >> >> Realize that we do have a pci_clear_master() in the release() to disable >> the DMA from device until the next open() will enable the DMA again . >> Here is my: >> >> Reviewed-by: Jianfeng Tan >> >> Thanks, >> Jianfeng >> >>> --- >>> Cc: Jianfeng Tan >>> Cc: Jingjing Wu >>> Cc: Shijith Thotton >>> Cc: Gregory Etelson >>> Cc: Harish Patil >>> Cc: George Prekas >>> Cc: Sergio Gonzalez Monroy >>> Cc: Rasesh Mody >>> >>> v2: >>> * fix typo in commit log >>> --- >>> config/common_base | 1 + >>> config/common_linuxapp | 1 + >>> lib/librte_eal/linuxapp/igb_uio/igb_uio.c | 2 ++ >>> 3 files changed, 4 insertions(+) >>> >>> diff --git a/config/common_base b/config/common_base >>> index 82ee75456..2a9947420 100644 >>> --- a/config/common_base >>> +++ b/config/common_base >>> @@ -102,6 +102,7 @@ CONFIG_RTE_LIBEAL_USE_HPET=n >>> CONFIG_RTE_EAL_ALLOW_INV_SOCKET_ID=n >>> CONFIG_RTE_EAL_ALWAYS_PANIC_ON_ERROR=n >>> CONFIG_RTE_EAL_IGB_UIO=n >>> +CONFIG_RTE_EAL_IGB_UIO_RESET=n >>> CONFIG_RTE_EAL_VFIO=n >>> CONFIG_RTE_MALLOC_DEBUG=n >>> CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n >>> diff --git a/config/common_linuxapp b/config/common_linuxapp >>> index 74c7d64ec..b3a602909 100644 >>> --- a/config/common_linuxapp >>> +++ b/config/common_linuxapp >>> @@ -37,6 +37,7 @@ CONFIG_RTE_EXEC_ENV_LINUXAPP=y >>> >>> CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=y >>> CONFIG_RTE_EAL_IGB_UIO=y >>> +CONFIG_RTE_EAL_IGB_UIO_RESET=y >>> CONFIG_RTE_EAL_VFIO=y >>> CONFIG_RTE_KNI_KMOD=y >>> CONFIG_RTE_LIBRTE_KNI=y >>> diff --git a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c >>> index fd320d87d..0325722c0 100644 >>> --- a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c >>> +++ b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c >>> @@ -360,7 +360,9 @@ igbuio_pci_release(struct uio_info *info, struct inode *inode) >>> /* stop the device from further DMA */ >>> pci_clear_master(dev); >>> >>> +#ifdef RTE_EAL_IGB_UIO_RESET >>> pci_reset_function(dev); >>> +#endif >>> >>> return 0; >>> } > > A compile time configuration option makes life very difficult for application providers. > > Consider the case where an application such as Open vSwitch with DPDK support is being provided > with a Linux distribution. One would want the Open vSwitch binary to support as many vendor NICs > as possible---without the need to recompile. With a change such as this, one would need to have > different versions of the kernel igb_uio module to support different NICs. Agreed, I am against adding more compile time options although I am end up sending a few of them these days. > The Linux kernel is already aware of, and provides work-arounds for, various PCI quirks. > For example, see linux/drivers/pci/quirks.c (http://elixir.free-electrons.com/linux/latest/source/drivers/pci/quirks.c). > > At this point in igb_uio.c, one is aware of the struct pci_dev "dev" for the device in question. > Access to the vendor and device information should be simple: > > struct pci_dev { > struct list_head bus_list;/* node in per-bus list */ > struct pci_bus*bus;/* bus this device is on */ > struct pci_bus*subordinate;/* bus this device bridges to */ > > void*sysdata;/* hook for sys-specific extension */ > struct proc_dir_entry *procent;/* device entry in /proc/bus/pci */ > struct pci_slot*slot;/* Physical slot this device is in */ > > unsigned intdevfn;/* encoded device & function index */ > unsigned shortvendor; > unsigned shortdevice; > unsigned shortsubsystem_vendor; > unsigned shortsubsystem_device; > ... > > One could imagine using logic to implement corresponding PCI quirks that can be evaluated > at runtime. For example (in pseudocode), > > if not (vendor = "Cavium" and device = "bnx2x") > then pci_reset_function(dev); It wouldn't be nice to add device specific checks into generic igb_uio module, but also it is not nice to add compile time option, comparing two I would be OK to device checks. What do you think about following? If there is no objection and Rasesh confirms that patch is working, I can send a proper patch for it. > > There are other possible implementations. If there are enough quirks, one might have action > functions defined---and a table of function pointers associated with each PMD to select the > proper action. > > - Lee Roberts > > > > diff --git a/lib/librte_eal/linuxapp/igb_uio/compat.h b/lib/librte_eal/linuxapp/igb_uio/compat.h index 30508f35c..264206af3 100644 --- a/lib/librte_eal/linuxapp/igb_uio/compat.h +++ b/lib/librte_eal/linuxapp/igb_uio/compat.h @@ -134,3 +134,21 @@ static bool pci_check_and_mask_intx(struct pci_dev *pdev) #endif +#define BROADCOM_PCI_VENDOR_ID 0x14E4 +static const struct pci_device_id no_reset_pci_tbl[] = { + { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x168a) }, /* 57800 */ + { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x164f) }, /* 57711 */ + { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x168e) }, /* 57810 */ + { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x163d) }, /* 57811 */ + { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x168d) }, /* 57840_OBS */ + { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x16a1) }, /* 57840_4_10 */ + { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x16a2) }, /* 57840_2_20 */ + { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x16ae) }, /* 57810_MF */ + { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x163e) }, /* 57811_MF */ + { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x16a4) }, /* 57840_MF */ + { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x16a9) }, /* 57800_VF */ + { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x16af) }, /* 57810_VF */ + { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x163f) }, /* 57811_VF */ + { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x16ad) }, /* 57840_VF */ + { 0 }, +}; diff --git a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c index fd320d87d..b0d92b51e 100644 --- a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c +++ b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c @@ -348,6 +348,14 @@ igbuio_pci_open(struct uio_info *info, struct inode *inode) return 0; } +static int is_device_excluded_from_reset(struct pci_dev *pdev) +{ + if (pci_match_id(no_reset_pci_tbl, pdev)) + return 1; + + return 0; +} + static int igbuio_pci_release(struct uio_info *info, struct inode *inode) { @@ -360,7 +368,8 @@ igbuio_pci_release(struct uio_info *info, struct inode *inode) /* stop the device from further DMA */ pci_clear_master(dev); - pci_reset_function(dev); + if (!is_device_excluded_from_reset(dev)) + pci_reset_function(dev); return 0; }