From patchwork Thu Jun 28 13:15:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rafal Kozik X-Patchwork-Id: 41844 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3D4B71B3A8; Thu, 28 Jun 2018 15:16:19 +0200 (CEST) Received: from mail-lf0-f67.google.com (mail-lf0-f67.google.com [209.85.215.67]) by dpdk.org (Postfix) with ESMTP id 6DDD28DAF for ; Thu, 28 Jun 2018 15:16:15 +0200 (CEST) Received: by mail-lf0-f67.google.com with SMTP id j26-v6so4178659lfb.11 for ; Thu, 28 Jun 2018 06:16:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=kBGocbdujXKlEP/+cvIvqTMZ+v6g682daABO2jktBhE=; b=YG+t7fyPwIOzegi9LRj4sA82ch6A9fqBTSYg8WpOb2RCJvvHY70UoJ3A0S2rR8dYo1 5aPb8VnsQXY1J6ZS8tmJbvFtvpDWMSl86sadusB2im8da+O9f5+FZHIku7LM/7Nht1B3 4L+kyDhfy2LXpenycTgfAPkCFUrs5mJya2Fk61gK3bm6Juq/hN6BbvE8HzzucF+XMlhA gtHTbQ4sV0HypC9utndc6QBMXumgt9MbGIFpRhhYNN55IfMfMCQexyO3b64YYrf9fvj6 ViMzzsCwT5/3y1yI1gzkH7oaF06Dh5tZqnJPsWdKwm8aeVzTwoHpq7nHYh1wWedxtuG3 RJZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=kBGocbdujXKlEP/+cvIvqTMZ+v6g682daABO2jktBhE=; b=JTa9Zv6wWWenS0A3pkjLLi1k1+WLGdUzn58B7L8qEv8KgNmgc5gAXCJm9SfP3vusAb POP077m8Hy3SXV9FhWWvjqwYvZYt7Hx2L3gAI3+9qB4+t1TjDdFLF0wqQq5A+KjwMER+ 6iTyev1CJQyZ/gaplC9o3j4uYLrHFJRCSG77Zsj2mG9rBNHmsaEmWhowNJ22OK+jqjWo m5adO7kdOHPaH6CCT0hDTrrK03LST3hdyzCaJcnNM200Vx9hBCTQbMBlTKX6XBNm5XSf A33nWulASul4B3eWsZQFBP5um4fVXbAfGYnrpYHWfEvKLcFLtBSihXKI5S+m+wOZK9f8 /tBw== X-Gm-Message-State: APt69E2E7DVZp+6z0MqZcKGt4XQGMBnhJRKP1IQkFlmBPMHMMPKyjct2 ugZQ0zouAq3RJZ4/Dl4GncvBS0SQ2MPB+g== X-Google-Smtp-Source: AAOMgpd/Z54njFqZ3DkX+5zJZURWke/UNgAp8tvhjtK/C8ZkB+jnzmwPnaVBSqumC4dahEMRQXCZmA== X-Received: by 2002:a19:e546:: with SMTP id c67-v6mr7309057lfh.135.1530191774880; Thu, 28 Jun 2018 06:16:14 -0700 (PDT) Received: from rafalkozik.semihalf.local (31-172-191-173.noc.fibertech.net.pl. [31.172.191.173]) by smtp.gmail.com with ESMTPSA id o196-v6sm1303246lfe.95.2018.06.28.06.16.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 28 Jun 2018 06:16:13 -0700 (PDT) From: Rafal Kozik To: dev@dpdk.org Cc: mw@semihalf.com, mk@semihalf.com, gtzalik@amazon.com, evgenys@amazon.com, matua@amazon.com, igorch@amazon.com, thomas@monjalon.net, ferruh.yigit@intel.com, Rafal Kozik Date: Thu, 28 Jun 2018 15:15:52 +0200 Message-Id: <1530191753-18689-4-git-send-email-rk@semihalf.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1530191753-18689-1-git-send-email-rk@semihalf.com> References: <1530191753-18689-1-git-send-email-rk@semihalf.com> Subject: [dpdk-dev] [PATCH v2 3/4] eal: enable WC during resources mapping X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Write combining (WC) increases NIC performance by making better utilization of PCI bus, but cannot be used by all PMDs. It will be enabled only if RTE_PCI_DRV_WC_ACTIVATE will be set in drivers flags. For proper work also igb_uio driver must be loaded with wc_activate set to 1. When mapping PCI resources, firstly try to us WC. If it is not supported it will fallback to normal mode. Signed-off-by: Rafal Kozik Acked-by: Bruce Richardson --- drivers/bus/pci/linux/pci_uio.c | 41 +++++++++++++++++++++++++++++------------ drivers/bus/pci/rte_bus_pci.h | 2 ++ 2 files changed, 31 insertions(+), 12 deletions(-) diff --git a/drivers/bus/pci/linux/pci_uio.c b/drivers/bus/pci/linux/pci_uio.c index d423e4b..e3947c2 100644 --- a/drivers/bus/pci/linux/pci_uio.c +++ b/drivers/bus/pci/linux/pci_uio.c @@ -282,22 +282,19 @@ int pci_uio_map_resource_by_index(struct rte_pci_device *dev, int res_idx, struct mapped_pci_resource *uio_res, int map_idx) { - int fd; + int fd = -1; char devname[PATH_MAX]; void *mapaddr; struct rte_pci_addr *loc; struct pci_map *maps; + int wc_activate = 0; + + if (dev->driver != NULL) + wc_activate = dev->driver->drv_flags & RTE_PCI_DRV_WC_ACTIVATE; loc = &dev->addr; maps = uio_res->maps; - /* update devname for mmap */ - snprintf(devname, sizeof(devname), - "%s/" PCI_PRI_FMT "/resource%d", - rte_pci_get_sysfs_path(), - loc->domain, loc->bus, loc->devid, - loc->function, res_idx); - /* allocate memory to keep path */ maps[map_idx].path = rte_malloc(NULL, strlen(devname) + 1, 0); if (maps[map_idx].path == NULL) { @@ -309,11 +306,31 @@ pci_uio_map_resource_by_index(struct rte_pci_device *dev, int res_idx, /* * open resource file, to mmap it */ - fd = open(devname, O_RDWR); - if (fd < 0) { - RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", + if (wc_activate) { + /* update devname for mmap */ + snprintf(devname, sizeof(devname), + "%s/" PCI_PRI_FMT "/resource%d_wc", + rte_pci_get_sysfs_path(), + loc->domain, loc->bus, loc->devid, + loc->function, res_idx); + + fd = open(devname, O_RDWR); + } + + if (!wc_activate || fd < 0) { + snprintf(devname, sizeof(devname), + "%s/" PCI_PRI_FMT "/resource%d", + rte_pci_get_sysfs_path(), + loc->domain, loc->bus, loc->devid, + loc->function, res_idx); + + /* then try to map resource file */ + fd = open(devname, O_RDWR); + if (fd < 0) { + RTE_LOG(ERR, EAL, "Cannot open %s: %s\n", devname, strerror(errno)); - goto error; + goto error; + } } /* try mapping somewhere close to the end of hugepages */ diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h index 458e6d0..828acc5 100644 --- a/drivers/bus/pci/rte_bus_pci.h +++ b/drivers/bus/pci/rte_bus_pci.h @@ -135,6 +135,8 @@ struct rte_pci_bus { /** Device needs PCI BAR mapping (done with either IGB_UIO or VFIO) */ #define RTE_PCI_DRV_NEED_MAPPING 0x0001 +/** Device needs PCI BAR mapping with enabled write combining (wc) */ +#define RTE_PCI_DRV_WC_ACTIVATE 0x0002 /** Device driver supports link state interrupt */ #define RTE_PCI_DRV_INTR_LSC 0x0008 /** Device driver supports device removal interrupt */