From patchwork Wed Jul 17 09:04:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vamsi Krishna Attunuru X-Patchwork-Id: 56585 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 34263276C; Wed, 17 Jul 2019 11:04:57 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 1B6E61DBE for ; Wed, 17 Jul 2019 11:04:54 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x6H93ZBH001991; Wed, 17 Jul 2019 02:04:54 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0818; bh=bhIk2ix3UfThvL5mxEF5ava1Ge5pp9WCNPgjkg/Lo9s=; b=qoSZx69RN16YxJpISwmol19LIoQ2P6ZnRdmGAfHkRxusgW/GajXl0wJCu8Roi1in7PUs wm9u9y//Zs3O/s2Ilt8fvQVkFxYFFObMVlC9iJTH9jb8P5MNiTx0m/y/7b/l8UlXaCF8 wABcSchZC9Tr1LjmpsKTVaqbzh1NoBsDKzt7IAks1ae3bn6rRIkk2JOg/tl+Ca4K1lU0 cu/DhWLzPPt3tl7GnDDZh5hEXIL0bVREkukifTcT9JhdJyEK+qVEg56QUEgyesjkiSNA f1lbiw4G9ivO/14mDYeUn3TXuQ7lBu+Tlts1B2i/FnytsRKmJC1Qq00C2cYgmGLLu0+N 9g== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 2ts0a26x0k-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 17 Jul 2019 02:04:53 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 17 Jul 2019 02:04:52 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Wed, 17 Jul 2019 02:04:52 -0700 Received: from hyd1vattunuru-dt.caveonetworks.com (unknown [10.29.52.72]) by maili.marvell.com (Postfix) with ESMTP id D82A43F7040; Wed, 17 Jul 2019 02:04:49 -0700 (PDT) From: To: CC: , , , , , , Vamsi Attunuru Date: Wed, 17 Jul 2019 14:34:05 +0530 Message-ID: <20190717090408.13717-2-vattunuru@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20190717090408.13717-1-vattunuru@marvell.com> References: <20190625035700.2953-1-vattunuru@marvell.com> <20190717090408.13717-1-vattunuru@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:5.22.84,1.0.8 definitions=2019-07-17_03:2019-07-16,2019-07-17 signatures=0 Subject: [dpdk-dev] [PATCH v7 1/4] mempool: modify mempool populate() to skip objects from page boundaries X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Vamsi Attunuru Currently the phys address of a mempool object populated by the mempool populate default() routine may not be contiguous with in that mbuf range. Patch ensures that each object's phys address is contiguous by modifying default behaviour of mempool populate() to prevent objects from being across 2 pages, expect if the size of object is bigger than size of page. Since the overhead after this modification will be very minimal considering the hugepage sizes of 512M & 1G, default behaviour is modified except for the object sizes bigger than the page size. Signed-off-by: Vamsi Attunuru Signed-off-by: Kiran Kumar K --- lib/librte_mempool/rte_mempool.c | 2 +- lib/librte_mempool/rte_mempool_ops_default.c | 33 ++++++++++++++++++++++++++-- 2 files changed, 32 insertions(+), 3 deletions(-) diff --git a/lib/librte_mempool/rte_mempool.c b/lib/librte_mempool/rte_mempool.c index 7260ce0..1c48325 100644 --- a/lib/librte_mempool/rte_mempool.c +++ b/lib/librte_mempool/rte_mempool.c @@ -339,7 +339,7 @@ rte_mempool_populate_iova(struct rte_mempool *mp, char *vaddr, i = rte_mempool_ops_populate(mp, mp->size - mp->populated_size, (char *)vaddr + off, (iova == RTE_BAD_IOVA) ? RTE_BAD_IOVA : (iova + off), - len - off, mempool_add_elem, NULL); + len - off, mempool_add_elem, opaque); /* not enough room to store one object */ if (i == 0) { diff --git a/lib/librte_mempool/rte_mempool_ops_default.c b/lib/librte_mempool/rte_mempool_ops_default.c index 4e2bfc8..85da264 100644 --- a/lib/librte_mempool/rte_mempool_ops_default.c +++ b/lib/librte_mempool/rte_mempool_ops_default.c @@ -45,19 +45,48 @@ rte_mempool_op_calc_mem_size_default(const struct rte_mempool *mp, return mem_size; } +/* Returns -1 if object falls on a page boundary, else returns 0 */ +static inline int +mempool_check_obj_bounds(void *obj, uint64_t hugepage_sz, size_t elt_sz) +{ + uintptr_t page_end, elt_addr = (uintptr_t)obj; + uint32_t pg_shift = rte_bsf32(hugepage_sz); + uint64_t page_mask; + + page_mask = ~((1ull << pg_shift) - 1); + page_end = (elt_addr & page_mask) + hugepage_sz; + + if (elt_addr + elt_sz > page_end) + return -1; + + return 0; +} + int rte_mempool_op_populate_default(struct rte_mempool *mp, unsigned int max_objs, void *vaddr, rte_iova_t iova, size_t len, rte_mempool_populate_obj_cb_t *obj_cb, void *obj_cb_arg) { - size_t total_elt_sz; - size_t off; + struct rte_memzone *mz = obj_cb_arg; + size_t total_elt_sz, off; unsigned int i; void *obj; total_elt_sz = mp->header_size + mp->elt_size + mp->trailer_size; for (off = 0, i = 0; off + total_elt_sz <= len && i < max_objs; i++) { + + /* Skip page boundary check if element is bigger than page */ + if (mz->hugepage_sz >= total_elt_sz) { + if (mempool_check_obj_bounds((char *)vaddr + off, + mz->hugepage_sz, + total_elt_sz) < 0) { + i--; /* Decrement count & skip this obj */ + off += total_elt_sz; + continue; + } + } + off += mp->header_size; obj = (char *)vaddr + off; obj_cb(mp, obj_cb_arg, obj, From patchwork Wed Jul 17 09:04:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vamsi Krishna Attunuru X-Patchwork-Id: 56586 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 33FB92BBE; Wed, 17 Jul 2019 11:05:03 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 5F4352082 for ; Wed, 17 Jul 2019 11:05:01 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x6H94wtj030026; Wed, 17 Jul 2019 02:04:58 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0818; bh=m2znQ6SI8vJMqKSUjM2FNii3DlSWYj9Rxf257Ax4Z9E=; b=kQOARILgT6YqT1dOvzQ6fLjoEKuwYhXXao10VCa21HYe8WNBpMSzwDjC0iBBMimo6hd4 KPcTpGfGtc7wtysDPF+/xa637XDsuLaobTUqCtM70+uk8zGYkHqKxopZuwhujHpBhoM9 HSbInKA9SAPidlrDgWqJDCLFjRnft8/yi12lDcBmqreMm37orJTkTk6RZrpK1WVCyf4q 6FqLt2LQwv9RVcd2JYb5qxqbgrRzR8xktX7N0Rk9nCwsSs3/hYZQDjZqtcrl+6iX14Kb WAH12lgrSsWUu07sDDIT1uNiUMHolxNc9f+9Dl9GqgEti7SAGAkvTtP5Fkc82PHKyOsh 7A== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0a-0016f401.pphosted.com with ESMTP id 2ts07vfg9p-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 17 Jul 2019 02:04:58 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 17 Jul 2019 02:04:55 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Wed, 17 Jul 2019 02:04:55 -0700 Received: from hyd1vattunuru-dt.caveonetworks.com (unknown [10.29.52.72]) by maili.marvell.com (Postfix) with ESMTP id 169DD3F703F; Wed, 17 Jul 2019 02:04:52 -0700 (PDT) From: To: CC: , , , , , , Vamsi Attunuru Date: Wed, 17 Jul 2019 14:34:06 +0530 Message-ID: <20190717090408.13717-3-vattunuru@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20190717090408.13717-1-vattunuru@marvell.com> References: <20190625035700.2953-1-vattunuru@marvell.com> <20190717090408.13717-1-vattunuru@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:5.22.84,1.0.8 definitions=2019-07-17_03:2019-07-16,2019-07-17 signatures=0 Subject: [dpdk-dev] [PATCH v7 2/4] kni: add IOVA = VA support in KNI lib X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Vamsi Attunuru Current KNI implmentation only operates in IOVA=PA mode, patch adds required functionality in KNI lib to support IOVA=VA mode. KNI kernel module requires device info to get iommu domain related information for IOVA addr related translations. Patch defines device related info in rte_kni_device_info struct and passes device info to kernel KNI module when IOVA=VA mode is enabled. Signed-off-by: Vamsi Attunuru Signed-off-by: Kiran Kumar K --- lib/librte_eal/linux/eal/include/rte_kni_common.h | 8 ++++ lib/librte_kni/Makefile | 1 + lib/librte_kni/meson.build | 1 + lib/librte_kni/rte_kni.c | 54 +++++++++++++++++++++++ 4 files changed, 64 insertions(+) diff --git a/lib/librte_eal/linux/eal/include/rte_kni_common.h b/lib/librte_eal/linux/eal/include/rte_kni_common.h index 37d9ee8..4fd8a90 100644 --- a/lib/librte_eal/linux/eal/include/rte_kni_common.h +++ b/lib/librte_eal/linux/eal/include/rte_kni_common.h @@ -111,6 +111,13 @@ struct rte_kni_device_info { void * mbuf_va; phys_addr_t mbuf_phys; + /* PCI info */ + uint16_t vendor_id; /**< Vendor ID or PCI_ANY_ID. */ + uint16_t device_id; /**< Device ID or PCI_ANY_ID. */ + uint8_t bus; /**< Device bus */ + uint8_t devid; /**< Device ID */ + uint8_t function; /**< Device function. */ + uint16_t group_id; /**< Group ID */ uint32_t core_id; /**< core ID to bind for kernel thread */ @@ -121,6 +128,7 @@ struct rte_kni_device_info { unsigned mbuf_size; unsigned int mtu; uint8_t mac_addr[6]; + uint8_t iova_mode; }; #define KNI_DEVICE "kni" diff --git a/lib/librte_kni/Makefile b/lib/librte_kni/Makefile index cbd6599..ab15d10 100644 --- a/lib/librte_kni/Makefile +++ b/lib/librte_kni/Makefile @@ -7,6 +7,7 @@ include $(RTE_SDK)/mk/rte.vars.mk LIB = librte_kni.a CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 -fno-strict-aliasing +CFLAGS += -I$(RTE_SDK)/drivers/bus/pci LDLIBS += -lrte_eal -lrte_mempool -lrte_mbuf -lrte_ethdev EXPORT_MAP := rte_kni_version.map diff --git a/lib/librte_kni/meson.build b/lib/librte_kni/meson.build index 41fa2e3..fd46f87 100644 --- a/lib/librte_kni/meson.build +++ b/lib/librte_kni/meson.build @@ -9,3 +9,4 @@ version = 2 sources = files('rte_kni.c') headers = files('rte_kni.h') deps += ['ethdev', 'pci'] +includes += include_directories('../../drivers/bus/pci') diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c index 4b51fb4..2cb653e 100644 --- a/lib/librte_kni/rte_kni.c +++ b/lib/librte_kni/rte_kni.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -199,6 +200,26 @@ kni_release_mz(struct rte_kni *kni) rte_memzone_free(kni->m_sync_addr); } +static void +kni_dev_pci_addr_get(struct rte_pci_addr *addr, + struct rte_pci_id *id, uint16_t port_id) +{ + const struct rte_pci_device *pci_dev; + const struct rte_bus *bus = NULL; + struct rte_eth_dev_info dev_info; + + memset(&dev_info, 0, sizeof(dev_info)); + rte_eth_dev_info_get(port_id, &dev_info); + + if (dev_info.device) + bus = rte_bus_find_by_device(dev_info.device); + if (bus && !strcmp(bus->name, "pci")) { + pci_dev = RTE_DEV_TO_PCI(dev_info.device); + *addr = pci_dev->addr; + *id = pci_dev->id; + } +} + struct rte_kni * rte_kni_alloc(struct rte_mempool *pktmbuf_pool, const struct rte_kni_conf *conf, @@ -247,6 +268,37 @@ rte_kni_alloc(struct rte_mempool *pktmbuf_pool, kni->ops.port_id = UINT16_MAX; memset(&dev_info, 0, sizeof(dev_info)); + + if (rte_eal_iova_mode() == RTE_IOVA_VA) { + uint64_t page_sz = pktmbuf_pool->mz->hugepage_sz; + uint16_t port_id = conf->group_id; + struct rte_pci_addr addr = { 0 }; + struct rte_pci_id id = { 0 }; + size_t buf_sz; + + kni_dev_pci_addr_get(&addr, &id, port_id); + dev_info.bus = addr.bus; + dev_info.devid = addr.devid; + dev_info.function = addr.function; + dev_info.vendor_id = id.vendor_id; + dev_info.device_id = id.device_id; + + buf_sz = pktmbuf_pool->header_size + pktmbuf_pool->elt_size + + pktmbuf_pool->trailer_size; + + /* Return failure when mbuf size is bigger than page size, + * because phys address of those mbuf might not be physically + * contiguous and KNI kernal module can not translate those + * mbuf's IOVA addresses. + */ + if (buf_sz > page_sz) { + RTE_LOG(ERR, KNI, + "KNI does not work in IOVA=VA mode when mbuf_sz > page_sz\n"); + RTE_LOG(ERR, KNI, "buf_sz:0x%" PRIx64 " > ", buf_sz); + RTE_LOG(ERR, KNI, "page_sz:0x%" PRIx64 "\n", page_sz); + goto kni_fail; + } + } dev_info.core_id = conf->core_id; dev_info.force_bind = conf->force_bind; dev_info.group_id = conf->group_id; @@ -300,6 +352,8 @@ rte_kni_alloc(struct rte_mempool *pktmbuf_pool, kni->group_id = conf->group_id; kni->mbuf_size = conf->mbuf_size; + dev_info.iova_mode = (rte_eal_iova_mode() == RTE_IOVA_VA) ? 1 : 0; + ret = ioctl(kni_fd, RTE_KNI_IOCTL_CREATE, &dev_info); if (ret < 0) goto ioctl_fail; From patchwork Wed Jul 17 09:04:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vamsi Krishna Attunuru X-Patchwork-Id: 56587 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 568EE3977; Wed, 17 Jul 2019 11:05:05 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id B6F0D2BBB for ; Wed, 17 Jul 2019 11:05:02 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x6H93MIf001836; Wed, 17 Jul 2019 02:05:01 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0818; bh=OAeZYqB7+SelaSPks7xucgxswXjN47KDTGWjzMyrXrI=; b=kCI915Xk0xOgMqjgAYm1F9+OoX4rk9CbNn2L6gUZjdux5QtiULa70mpvLYkjcYkz3PFl dIeHu/+ywIHPPj+BWtUHPOahqEtDSii4RYYbHFbh/5n+TOlyM8/XUbWTkPyk6lyHKe0a b1oj6r+gKMckNY1eu9xeNU8H3FUrLoJpsX+ktUEzPMmKFRfqco1Tl2V+yNp8ax8zZfn6 znakpdU9dSymgeKCtQxOxH6Cv+iCgDdBc8/oIioL+BsMHMZrO8qlgofpBkFjBCVDX/l2 6/zm/HOR6MaVox8EUIGPFsD1tFCeTij+ljQyOw3JaF/pq65QQ2n0MaehwuoOJ6HS7x9L gg== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0b-0016f401.pphosted.com with ESMTP id 2ts0a26x1a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 17 Jul 2019 02:05:01 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 17 Jul 2019 02:04:59 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Wed, 17 Jul 2019 02:04:59 -0700 Received: from hyd1vattunuru-dt.caveonetworks.com (unknown [10.29.52.72]) by maili.marvell.com (Postfix) with ESMTP id 48FE03F7040; Wed, 17 Jul 2019 02:04:56 -0700 (PDT) From: To: CC: , , , , , , Vamsi Attunuru Date: Wed, 17 Jul 2019 14:34:07 +0530 Message-ID: <20190717090408.13717-4-vattunuru@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20190717090408.13717-1-vattunuru@marvell.com> References: <20190625035700.2953-1-vattunuru@marvell.com> <20190717090408.13717-1-vattunuru@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:5.22.84,1.0.8 definitions=2019-07-17_03:2019-07-16,2019-07-17 signatures=0 Subject: [dpdk-dev] [PATCH v7 3/4] kni: add IOVA=VA support in KNI module X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Kiran Kumar K Patch adds support for kernel module to work in IOVA = VA mode, the idea is to get physical address from IOVA address using iommu_iova_to_phys API and later use phys_to_virt API to convert the physical address to kernel virtual address. When compared with IOVA = PA mode, there is no performance drop with this approach. This approach does not work with the kernel versions less than 4.4.0 because of API compatibility issues. Patch also updates these support details in KNI documentation. Signed-off-by: Kiran Kumar K Signed-off-by: Vamsi Attunuru --- doc/guides/prog_guide/kernel_nic_interface.rst | 8 +++ kernel/linux/kni/compat.h | 4 ++ kernel/linux/kni/kni_dev.h | 4 ++ kernel/linux/kni/kni_misc.c | 71 ++++++++++++++++++++++---- kernel/linux/kni/kni_net.c | 59 ++++++++++++++++----- 5 files changed, 124 insertions(+), 22 deletions(-) diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst index 38369b3..fd2ce63 100644 --- a/doc/guides/prog_guide/kernel_nic_interface.rst +++ b/doc/guides/prog_guide/kernel_nic_interface.rst @@ -291,6 +291,14 @@ The sk_buff is then freed and the mbuf sent in the tx_q FIFO. The DPDK TX thread dequeues the mbuf and sends it to the PMD via ``rte_eth_tx_burst()``. It then puts the mbuf back in the cache. +IOVA = VA: Support +------------------ + +KNI can be operated in IOVA as VA scheme when following criteria are fullfilled + +- LINUX_VERSION_CODE >= KERNEL_VERSION(4, 4, 0) +- eal param `--iova-mode va` is passed or bus IOVA scheme is set to RTE_IOVA_VA + Ethtool ------- diff --git a/kernel/linux/kni/compat.h b/kernel/linux/kni/compat.h index 562d8bf..ee997a6 100644 --- a/kernel/linux/kni/compat.h +++ b/kernel/linux/kni/compat.h @@ -121,3 +121,7 @@ #if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 11, 0) #define HAVE_SIGNAL_FUNCTIONS_OWN_HEADER #endif + +#if KERNEL_VERSION(4, 4, 0) <= LINUX_VERSION_CODE +#define HAVE_IOVA_AS_VA_SUPPORT +#endif diff --git a/kernel/linux/kni/kni_dev.h b/kernel/linux/kni/kni_dev.h index c1ca678..d5898f3 100644 --- a/kernel/linux/kni/kni_dev.h +++ b/kernel/linux/kni/kni_dev.h @@ -25,6 +25,7 @@ #include #include #include +#include #include #define KNI_KTHREAD_RESCHEDULE_INTERVAL 5 /* us */ @@ -41,6 +42,9 @@ struct kni_dev { /* kni list */ struct list_head list; + uint8_t iova_mode; + struct iommu_domain *domain; + uint32_t core_id; /* Core ID to bind */ char name[RTE_KNI_NAMESIZE]; /* Network device name */ struct task_struct *pthread; diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c index 2b75502..8660205 100644 --- a/kernel/linux/kni/kni_misc.c +++ b/kernel/linux/kni/kni_misc.c @@ -295,6 +295,9 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_num, struct rte_kni_device_info dev_info; struct net_device *net_dev = NULL; struct kni_dev *kni, *dev, *n; + struct pci_dev *pci = NULL; + struct iommu_domain *domain = NULL; + phys_addr_t phys_addr; pr_info("Creating kni...\n"); /* Check the buffer size, to avoid warning */ @@ -348,15 +351,65 @@ kni_ioctl_create(struct net *net, uint32_t ioctl_num, strncpy(kni->name, dev_info.name, RTE_KNI_NAMESIZE); /* Translate user space info into kernel space info */ - kni->tx_q = phys_to_virt(dev_info.tx_phys); - kni->rx_q = phys_to_virt(dev_info.rx_phys); - kni->alloc_q = phys_to_virt(dev_info.alloc_phys); - kni->free_q = phys_to_virt(dev_info.free_phys); - - kni->req_q = phys_to_virt(dev_info.req_phys); - kni->resp_q = phys_to_virt(dev_info.resp_phys); - kni->sync_va = dev_info.sync_va; - kni->sync_kva = phys_to_virt(dev_info.sync_phys); + if (dev_info.iova_mode) { +#ifdef HAVE_IOVA_AS_VA_SUPPORT + pci = pci_get_device(dev_info.vendor_id, + dev_info.device_id, NULL); + if (pci == NULL) { + pr_err("pci dev does not exist\n"); + return -ENODEV; + } + + while (pci) { + if ((pci->bus->number == dev_info.bus) && + (PCI_SLOT(pci->devfn) == dev_info.devid) && + (PCI_FUNC(pci->devfn) == dev_info.function)) { + domain = iommu_get_domain_for_dev(&pci->dev); + break; + } + pci = pci_get_device(dev_info.vendor_id, + dev_info.device_id, pci); + } + + if (domain == NULL) { + pr_err("Failed to get pci dev domain info\n"); + return -ENODEV; + } +#else + pr_err("Kernel version does not support IOVA as VA\n"); + return -EINVAL; +#endif + kni->domain = domain; + phys_addr = iommu_iova_to_phys(domain, dev_info.tx_phys); + kni->tx_q = phys_to_virt(phys_addr); + phys_addr = iommu_iova_to_phys(domain, dev_info.rx_phys); + kni->rx_q = phys_to_virt(phys_addr); + phys_addr = iommu_iova_to_phys(domain, dev_info.alloc_phys); + kni->alloc_q = phys_to_virt(phys_addr); + phys_addr = iommu_iova_to_phys(domain, dev_info.free_phys); + kni->free_q = phys_to_virt(phys_addr); + phys_addr = iommu_iova_to_phys(domain, dev_info.req_phys); + kni->req_q = phys_to_virt(phys_addr); + phys_addr = iommu_iova_to_phys(domain, dev_info.resp_phys); + kni->resp_q = phys_to_virt(phys_addr); + kni->sync_va = dev_info.sync_va; + phys_addr = iommu_iova_to_phys(domain, dev_info.sync_phys); + kni->sync_kva = phys_to_virt(phys_addr); + kni->iova_mode = 1; + + } else { + + kni->tx_q = phys_to_virt(dev_info.tx_phys); + kni->rx_q = phys_to_virt(dev_info.rx_phys); + kni->alloc_q = phys_to_virt(dev_info.alloc_phys); + kni->free_q = phys_to_virt(dev_info.free_phys); + + kni->req_q = phys_to_virt(dev_info.req_phys); + kni->resp_q = phys_to_virt(dev_info.resp_phys); + kni->sync_va = dev_info.sync_va; + kni->sync_kva = phys_to_virt(dev_info.sync_phys); + kni->iova_mode = 0; + } kni->mbuf_size = dev_info.mbuf_size; diff --git a/kernel/linux/kni/kni_net.c b/kernel/linux/kni/kni_net.c index a736407..9387e4c 100644 --- a/kernel/linux/kni/kni_net.c +++ b/kernel/linux/kni/kni_net.c @@ -36,6 +36,21 @@ static void kni_net_rx_normal(struct kni_dev *kni); /* kni rx function pointer, with default to normal rx */ static kni_net_rx_t kni_net_rx_func = kni_net_rx_normal; +/* iova to kernel virtual address */ +static inline void * +iova2kva(struct kni_dev *kni, void *pa) +{ + return phys_to_virt(iommu_iova_to_phys(kni->domain, + (uintptr_t)pa)); +} + +static inline void * +iova2data_kva(struct kni_dev *kni, struct rte_kni_mbuf *m) +{ + return phys_to_virt(iommu_iova_to_phys(kni->domain, + (uintptr_t)m->buf_physaddr) + m->data_off); +} + /* physical address to kernel virtual address */ static void * pa2kva(void *pa) @@ -62,6 +77,24 @@ kva2data_kva(struct rte_kni_mbuf *m) return phys_to_virt(m->buf_physaddr + m->data_off); } +static inline void * +get_kva(struct kni_dev *kni, void *pa) +{ + if (kni->iova_mode == 1) + return iova2kva(kni, pa); + + return pa2kva(pa); +} + +static inline void * +get_data_kva(struct kni_dev *kni, void *pkt_kva) +{ + if (kni->iova_mode == 1) + return iova2data_kva(kni, pkt_kva); + + return kva2data_kva(pkt_kva); +} + /* * It can be called to process the request. */ @@ -178,7 +211,7 @@ kni_fifo_trans_pa2va(struct kni_dev *kni, return; for (i = 0; i < num_rx; i++) { - kva = pa2kva(kni->pa[i]); + kva = get_kva(kni, kni->pa[i]); kni->va[i] = pa2va(kni->pa[i], kva); kva_nb_segs = kva->nb_segs; @@ -266,8 +299,8 @@ kni_net_tx(struct sk_buff *skb, struct net_device *dev) if (likely(ret == 1)) { void *data_kva; - pkt_kva = pa2kva(pkt_pa); - data_kva = kva2data_kva(pkt_kva); + pkt_kva = get_kva(kni, pkt_pa); + data_kva = get_data_kva(kni, pkt_kva); pkt_va = pa2va(pkt_pa, pkt_kva); len = skb->len; @@ -338,9 +371,9 @@ kni_net_rx_normal(struct kni_dev *kni) /* Transfer received packets to netif */ for (i = 0; i < num_rx; i++) { - kva = pa2kva(kni->pa[i]); + kva = get_kva(kni, kni->pa[i]); len = kva->pkt_len; - data_kva = kva2data_kva(kva); + data_kva = get_data_kva(kni, kva); kni->va[i] = pa2va(kni->pa[i], kva); skb = netdev_alloc_skb(dev, len); @@ -437,9 +470,9 @@ kni_net_rx_lo_fifo(struct kni_dev *kni) num = ret; /* Copy mbufs */ for (i = 0; i < num; i++) { - kva = pa2kva(kni->pa[i]); + kva = get_kva(kni, kni->pa[i]); len = kva->pkt_len; - data_kva = kva2data_kva(kva); + data_kva = get_data_kva(kni, kva); kni->va[i] = pa2va(kni->pa[i], kva); while (kva->next) { @@ -449,8 +482,8 @@ kni_net_rx_lo_fifo(struct kni_dev *kni) kva = next_kva; } - alloc_kva = pa2kva(kni->alloc_pa[i]); - alloc_data_kva = kva2data_kva(alloc_kva); + alloc_kva = get_kva(kni, kni->alloc_pa[i]); + alloc_data_kva = get_data_kva(kni, alloc_kva); kni->alloc_va[i] = pa2va(kni->alloc_pa[i], alloc_kva); memcpy(alloc_data_kva, data_kva, len); @@ -517,9 +550,9 @@ kni_net_rx_lo_fifo_skb(struct kni_dev *kni) /* Copy mbufs to sk buffer and then call tx interface */ for (i = 0; i < num; i++) { - kva = pa2kva(kni->pa[i]); + kva = get_kva(kni, kni->pa[i]); len = kva->pkt_len; - data_kva = kva2data_kva(kva); + data_kva = get_data_kva(kni, kva); kni->va[i] = pa2va(kni->pa[i], kva); skb = netdev_alloc_skb(dev, len); @@ -550,8 +583,8 @@ kni_net_rx_lo_fifo_skb(struct kni_dev *kni) break; prev_kva = kva; - kva = pa2kva(kva->next); - data_kva = kva2data_kva(kva); + kva = get_kva(kni, kva->next); + data_kva = get_data_kva(kni, kva); /* Convert physical address to virtual address */ prev_kva->next = pa2va(prev_kva->next, kva); } From patchwork Wed Jul 17 09:04:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vamsi Krishna Attunuru X-Patchwork-Id: 56588 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6BEB92F42; Wed, 17 Jul 2019 11:05:09 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 001784D27 for ; Wed, 17 Jul 2019 11:05:05 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x6H9535I030057; Wed, 17 Jul 2019 02:05:03 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0818; bh=7nR4JTgwZJIggO5h+vWhAxb7u3FyOWDTWqzEpEdlV38=; b=MtfZ4LP57kgXUYulIoDYDwkLaeLy1649fBy611wfXc5mPdZ3ZhlJqu/TWke6blnmww4t zC7FM5I+SUmYDoNqEikc/PVkkkbKXtoWDBP25TFO+6LhvbOjc4HrFkbXfBVaJkSxqvx6 KMDckHexyK8Vod/0qonfuv3gWt5Z35jZ6N3dJOUMcki5KcNpRK8o3b+EcTOC68ak58G1 9948Adjyx0PIl8z5Kor3ieZAxBSlwmwtOFVW37P8Oh92emofxHOQ4f1sC5951cvzYpKv yFAEeMILhlUUJrbn2om4L9GDLg5zSMoAA+eiO1FOCHFMaSyrkZkaUju/WpyKVBSn5Puk Zw== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0a-0016f401.pphosted.com with ESMTP id 2ts07vfgaa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 17 Jul 2019 02:05:03 -0700 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 17 Jul 2019 02:05:02 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Wed, 17 Jul 2019 02:05:02 -0700 Received: from hyd1vattunuru-dt.caveonetworks.com (unknown [10.29.52.72]) by maili.marvell.com (Postfix) with ESMTP id 803F23F703F; Wed, 17 Jul 2019 02:04:59 -0700 (PDT) From: To: CC: , , , , , , Vamsi Attunuru Date: Wed, 17 Jul 2019 14:34:08 +0530 Message-ID: <20190717090408.13717-5-vattunuru@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20190717090408.13717-1-vattunuru@marvell.com> References: <20190625035700.2953-1-vattunuru@marvell.com> <20190717090408.13717-1-vattunuru@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:5.22.84,1.0.8 definitions=2019-07-17_03:2019-07-16,2019-07-17 signatures=0 Subject: [dpdk-dev] [PATCH v7 4/4] kni: modify IOVA mode checks to support VA X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Vamsi Attunuru Patch addresses checks in KNI and eal that enforce IOVA=PA when IOVA=VA mode is enabled, since KNI kernel module supports VA mode for kernel versions >= 4.4.0. Signed-off-by: Vamsi Attunuru Signed-off-by: Kiran Kumar K --- lib/librte_eal/linux/eal/eal.c | 4 +++- lib/librte_kni/rte_kni.c | 5 ----- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/lib/librte_eal/linux/eal/eal.c b/lib/librte_eal/linux/eal/eal.c index 2e5499f..9705243 100644 --- a/lib/librte_eal/linux/eal/eal.c +++ b/lib/librte_eal/linux/eal/eal.c @@ -1070,12 +1070,14 @@ rte_eal_init(int argc, char **argv) /* Workaround for KNI which requires physical address to work */ if (iova_mode == RTE_IOVA_VA && rte_eal_check_module("rte_kni") == 1) { +#if KERNEL_VERSION(4, 4, 0) > LINUX_VERSION_CODE if (phys_addrs) { iova_mode = RTE_IOVA_PA; - RTE_LOG(WARNING, EAL, "Forcing IOVA as 'PA' because KNI module is loaded\n"); + RTE_LOG(WARNING, EAL, "Forcing IOVA as 'PA' because KNI module does not support VA\n"); } else { RTE_LOG(DEBUG, EAL, "KNI can not work since physical addresses are unavailable\n"); } +#endif } #endif rte_eal_get_configuration()->iova_mode = iova_mode; diff --git a/lib/librte_kni/rte_kni.c b/lib/librte_kni/rte_kni.c index 2cb653e..959088e 100644 --- a/lib/librte_kni/rte_kni.c +++ b/lib/librte_kni/rte_kni.c @@ -98,11 +98,6 @@ static volatile int kni_fd = -1; int rte_kni_init(unsigned int max_kni_ifaces __rte_unused) { - if (rte_eal_iova_mode() != RTE_IOVA_PA) { - RTE_LOG(ERR, KNI, "KNI requires IOVA as PA\n"); - return -1; - } - /* Check FD and open */ if (kni_fd < 0) { kni_fd = open("/dev/" KNI_DEVICE, O_RDWR);