From patchwork Wed Apr 11 14:07:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rafal Kozik X-Patchwork-Id: 37948 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F0B0F1BC2C; Wed, 11 Apr 2018 16:07:43 +0200 (CEST) Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com [74.125.82.68]) by dpdk.org (Postfix) with ESMTP id 7DA3D1BC2A for ; Wed, 11 Apr 2018 16:07:42 +0200 (CEST) Received: by mail-wm0-f68.google.com with SMTP id x4so4017001wmh.5 for ; Wed, 11 Apr 2018 07:07:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=LOIs2o//SoA4Dfua0A2wrUDbKblMtegGmtCxZnCpGVk=; b=Q7EJyvzZoipfztUwR788EZ8elamGwIT+UHacVabdDhZeror/1ktLgxDgaB+UlAl0Gj nXiALQOT1oU5SthQzxaOawKmE5LUoVH3Q2XXfuiR9dBRUhi2FFncV/XxHXmoQcQBn5mx Zar2oHqp876yUA53aQhjLxqGDlOYVWSxyMrDM+EG3TxgPgwCUd/X09apadDiV0bUZ/wq eQ6jZWqvfSwuXB3ywOuL+0/Q4Td1K5eB/TGtCZWT/WhCSHHauelB/OFwpEninqlzOKvg FcS2OOeuLMbAJvstBF+u9bGUibSwBWIu1KsTYzmZyD/FGSnfWEBFBKaIVosn9nVTzUF9 5Saw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=LOIs2o//SoA4Dfua0A2wrUDbKblMtegGmtCxZnCpGVk=; b=hXw+qoGSaNc1kYX6RvRVHO8IHkeku6+m//WS7QmSVJXoD+MPnUPXtSOSiGrvo36H7v cP6/UAsFkpismw0JyQm9pQIJkPmowlNagrAmxQFQwTjz87+Okm2NFlDrqoEUaLIBWjeT nG7+vtvEmyGVaCZm5d3RoytTL1qfuXUrVveC2dXp1of8VJ68Dgn1avQBZOrj2y8Bct4v mwXzK5M6wwSkja4UHjMksJ9ON9AXgCNq8hofDugZB+0eUd8pHeJCZ/Wv+CGiuRrqqQzE Djh1NsRX9OKkWYwMevCPjwvlSSkW5/gmXik86QqVHcwe12HizojqP7DGOs8hlu1/ClZT tlKw== X-Gm-Message-State: ALQs6tB5hSEh7ziTHOjIMaJqlChOkKORIqj/OIBM42NYWEhFDxw9FYPo rJdIFUPibd/0SRAMRWKE2SZQ5uojZP7ewA== X-Google-Smtp-Source: AIpwx49eI6O8F1R+1Zw1Wxnv2ihc5v3ta3b9zxl59hUEYu+GQ1lNOzzzthgd7ytBtRrV9ooWsE7nbg== X-Received: by 10.28.232.202 with SMTP id f71mr3099649wmi.136.1523455661960; Wed, 11 Apr 2018 07:07:41 -0700 (PDT) Received: from rafalkozik.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id 6sm1607761wmf.31.2018.04.11.07.07.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 11 Apr 2018 07:07:41 -0700 (PDT) From: Rafal Kozik To: dev@dpdk.org Cc: mw@semihalf.com, mk@semihalf.com, gtzalik@amazon.com, evgenys@amazon.com, matua@amazon.com, igorch@amazon.com, ferruh.yigit@intel.com, Rafal Kozik Date: Wed, 11 Apr 2018 16:07:16 +0200 Message-Id: <1523455637-31719-4-git-send-email-rk@semihalf.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1523455637-31719-1-git-send-email-rk@semihalf.com> References: <1523455637-31719-1-git-send-email-rk@semihalf.com> Subject: [dpdk-dev] [PATCH 3/4] eal: enable WC during resources mapping X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Write combining (wc) increase NIC performenca by making better utilization of PCI bus, but cannot be used by all PMD. It will be enable only if RTE_PCI_DRV_WC_ACTIVATE will be set in drivers flags. For proper work also igb driver must be loaded with wc_activate set to 1. When mapping PCI resources firstly try to us wc. If it is not supported it will fallback to normal mode. Signed-off-by: Rafal Kozik --- drivers/bus/pci/linux/pci_uio.c | 39 ++++++++++++++++++++++++++++----------- drivers/bus/pci/rte_bus_pci.h | 2 ++ 2 files changed, 30 insertions(+), 11 deletions(-) diff --git a/drivers/bus/pci/linux/pci_uio.c b/drivers/bus/pci/linux/pci_uio.c index d423e4b..a1570c9 100644 --- a/drivers/bus/pci/linux/pci_uio.c +++ b/drivers/bus/pci/linux/pci_uio.c @@ -287,17 +287,14 @@ pci_uio_map_resource_by_index(struct rte_pci_device *dev, int res_idx, void *mapaddr; struct rte_pci_addr *loc; struct pci_map *maps; + int wc_activate = 0; + + if (dev->driver != NULL) + wc_activate = dev->driver->drv_flags & RTE_PCI_DRV_WC_ACTIVATE; loc = &dev->addr; maps = uio_res->maps; - /* update devname for mmap */ - snprintf(devname, sizeof(devname), - "%s/" PCI_PRI_FMT "/resource%d", - rte_pci_get_sysfs_path(), - loc->domain, loc->bus, loc->devid, - loc->function, res_idx); - /* allocate memory to keep path */ maps[map_idx].path = rte_malloc(NULL, strlen(devname) + 1, 0); if (maps[map_idx].path == NULL) { @@ -309,11 +306,31 @@ pci_uio_map_resource_by_index(struct rte_pci_device *dev, int res_idx, /* * open resource file, to mmap it */ - fd = open(devname, O_RDWR); - if (fd < 0) { - RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", + if (wc_activate) { + /* update devname for mmap */ + snprintf(devname, sizeof(devname), + "%s/" PCI_PRI_FMT "/resource%d_wc", + rte_pci_get_sysfs_path(), + loc->domain, loc->bus, loc->devid, + loc->function, res_idx); + + fd = open(devname, O_RDWR); + } + + if (!wc_activate || fd < 0) { + snprintf(devname, sizeof(devname), + "%s/" PCI_PRI_FMT "/resource%d", + rte_pci_get_sysfs_path(), + loc->domain, loc->bus, loc->devid, + loc->function, res_idx); + + /* then try to map resource file */ + fd = open(devname, O_RDWR); + if (fd < 0) { + RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", devname, strerror(errno)); - goto error; + goto error; + } } /* try mapping somewhere close to the end of hugepages */ diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h index 357afb9..b7bcce3 100644 --- a/drivers/bus/pci/rte_bus_pci.h +++ b/drivers/bus/pci/rte_bus_pci.h @@ -132,6 +132,8 @@ struct rte_pci_bus { /** Device needs PCI BAR mapping (done with either IGB_UIO or VFIO) */ #define RTE_PCI_DRV_NEED_MAPPING 0x0001 +/** Device needs PCI BAR mapping with enabled write combining (wc) */ +#define RTE_PCI_DRV_WC_ACTIVATE 0x0002 /** Device driver supports link state interrupt */ #define RTE_PCI_DRV_INTR_LSC 0x0008 /** Device driver supports device removal interrupt */