From patchwork Thu Jun 8 11:05:05 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Santosh Shukla X-Patchwork-Id: 25190 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id BFC3A37B1; Thu, 8 Jun 2017 13:06:27 +0200 (CEST) Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-bl2nam02on0067.outbound.protection.outlook.com [104.47.38.67]) by dpdk.org (Postfix) with ESMTP id 4C2082BA1 for ; Thu, 8 Jun 2017 13:06:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=8yIEu/+esdMoOGDUQ0HaH28qfmv7xqlffmGOLDnk0ng=; b=Y1VhVXEKp0wV5tNx+7WO6YK5btOI8/HT/JLnARIeROz0eGLrim/r/6bP6W67N3WSpj6jmIaf0ONwjfdSsoQM2cl9ejZCECgUVk1ZOtFP0LV4jmlyg6Ft3n9u7Ah8AX8slkn3ER8S3vgF41T/CT7n0I53zY8IGkFD9Y+5uIJrgUE= Authentication-Results: monjalon.net; dkim=none (message not signed) header.d=none;monjalon.net; dmarc=none action=none header.from=caviumnetworks.com; Received: from localhost.localdomain (14.140.2.178) by DM5PR07MB3099.namprd07.prod.outlook.com (10.172.85.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1143.10; Thu, 8 Jun 2017 11:06:07 +0000 From: Santosh Shukla To: thomas@monjalon.net, bruce.richardson@intel.com, dev@dpdk.org Cc: jerin.jacob@caviumnetworks.com, hemant.agrawal@nxp.com, shreyansh.jain@nxp.com, gaetan.rivet@6wind.com, Santosh Shukla Date: Thu, 8 Jun 2017 16:35:05 +0530 Message-Id: <20170608110513.22548-3-santosh.shukla@caviumnetworks.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170608110513.22548-1-santosh.shukla@caviumnetworks.com> References: <20170608110513.22548-1-santosh.shukla@caviumnetworks.com> MIME-Version: 1.0 X-Originating-IP: [14.140.2.178] X-ClientProxiedBy: BMXPR01CA0025.INDPRD01.PROD.OUTLOOK.COM (10.174.214.11) To DM5PR07MB3099.namprd07.prod.outlook.com (10.172.85.9) X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM5PR07MB3099: X-MS-Office365-Filtering-Correlation-Id: 7ed3bfc4-3728-40d8-72db-08d4ae5e5e98 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201703131423075)(201703031133081); SRVR:DM5PR07MB3099; X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3099; 3:x3e2ujO6diS6OpHhiDsXBrbuv0DRgmq4Bq192Pigv2y2Gf1pV6VeJRZaR8ebi93nqpZzAJrq9yYBVCqQS/vOLxhDgaXu2HZtXPMDK0cjMjL4kxg64wdth3QZ3LCdyZSxUMfJwqzWLu6nsDPx/1lQkVt4D46MK670fbmpjRGO1Y6imNchMHh/wyWKqcdDPetrguBPIrFPnAaZpfiyXdOoe00zkotvtBwCfvOtRVkEqlffKEFa58uUvI8B1onOtjbnRfm60EFeiSSBi9irDnlrVpFQPTjEpjmLhWDpN/ogYvQBl4zWSB/o7E+orW1e1wkh85rtxrMp1AwyZMunB3Dp6w==; 25:Jd0pbw2Lityfm5b1NyA3F5kKwxBVyc1m+Pb5a09/Ph+WyFWB5yqaeobyrAQi4WWXkoQlPDQlGo0LkvnykHSN6mH7pHFWgLboflHYENmn65qlTuR0Niy/VteiMlWfexq49AvPWFcP+SzTa6g6MAvgmcb0BmQX6RjDnJqYAxVYCKcIIfQ+hLtS1UJrQKZ12bckdOAacdOSpHAr0kYMMdTaN2JL3ZnZKq8Z+dxPlXbMM55SiI3VZqw3pwEtHRSJT2ruafY/8bj6pVEaziy2LyNewKn0zM988kSuc/Kmn6a7NLmb6sAXvlkgSWwikNka18TCncYJ+cHSIDFd3EmTdEhsyARcyYwCWCXUBA91Zo3O7di9X/qTaHDAiS6Ez/RoPEkpcDyRkCKTDqSEAlmQYkFFhU57J/xo2xxVCIwJFghk5ZrQ2gWfXQx/wdXog4oXsQButP6mszUprNDugLDUoeryyuq0RlMuiLJVdsz+uosV2L0= X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3099; 31:4zzQ/Gzh2rCQGEdYYKsfP0bf2//C9+Of9x8VojuZji6+5ldmhFIzTYyfcCzQtdIe7eLZQd9UwR3Zj+ldMZieoN4XNMT1z9o6mxq5EN5Et2PHyZ+k0aFdZc9ZeAc+IIxAz6w1IDVy1SuMjkoP/rncOCXGmhLMwT8UD5zffwuS62aY7YApeDrRL0uasBryaM9dOThkQS2ygoVolNPXuIT6INPlwXejrHXfp3V5vHe2UCY=; 20:ojGsWc+Jt6qmjLcbpHTEpj0SAwxH2fnczA6mY4tfxr1SgoynlTxMDCPuq3PVc5C65KpI5fm1XaBonFGFOiufyEWSeiYct+9x+/ACc9AcstMZUncmTFlVWSXtLRkYIVrmnNrtHYOQysmSgBsi2IVN+PdlafteBMF5fNfWTotNf2HZZJkI9HDZrEOqj8TG8Gh0/VwERk4oqj804wZsCESUqQkI3jUkXlBHtRzuqBwUb/w/WKLq+i37em1kXO4kSTThB+ditz8BJuE2eZ+6THCvoE4CdXYZWu3vEZbRrvrHBzx2DHIMDUXRpmxFwrLxFnJxWqZ1uOrFyZbDhK1ViaCnzJi48utu1H0HwyNsavw/FUDDD0Fup69+6v6SwkkpU1i8WLmXmylXsXmuxn34MSj4iNwHdJavV0913Mi7W0IOWYOg7Eyxr7ecmBQzSTK1RuCO9/Vk5DA9aPPvLyo7VYl9ZfqobLtiDqXtTN6WjC5WoGviSQYGN6Xx94Rhm/+1d04BamWR0Z9PBsNk0ZZwiMLUX0lR9OA7jPELrRnMitBYJraW60Xf9Ta73z0fZ1YmxlU86hqnJVcBRGgwgAh6xSyrwZLlCi8pRAerDvmXVLpTm4E= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(93006095)(3002001)(100000703101)(100105400095)(10201501046)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123555025)(20161123558100)(20161123564025)(20161123560025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:DM5PR07MB3099; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:DM5PR07MB3099; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM5PR07MB3099; 4:c2ng0Cedc9YVGUHBW5gsBcrGxpIgyvoSMJnA/IRMZI?= 8CfiVi836kkUnEVfaRHw9L0dlYIr0X9yVe+WVpl09ywRrB1J9nNlIbzHmO0mEfxIcgCI1vA3SbaWnNmvqPlUwxkFpxX9U8SVaSzk6mMIs9oEWzch5MYH+Gr0kYP4mCoC7UnIUBAJ+2EFAh14GYxxADHs8+M0JD+76RyUFVDj98jT70Km45PM3UHfnzifxVj4wq1GqfTNjCdfMQHCeDRnOObSZyRLXQCkPjd/nUoKh3Q4gQm5OWhfG1/QuXVx4OQnz84uF8YTJATZG7gs10VfkhvUlCan4yzd7vWC4WBkojwsL8ZnEQE/V5g1z4EQYlSgEufmdZq0ABJR/I52b/gZ63uTMMrsyoci9QRegJgg/SL3KaFUCv9G9m771uY4GEBD8jTjfXuPl0U7B+TMU3DVc3ATyY3lcXV1e7JKxFbzTrpZVNn/ci3sWVndqS4t30ZEyOyBfjZBZ0ezP1v0hX4Ud5ZUVLsSaE+/Qa9jGnVS7GhH716tvUzYiSSp4o2AF4X7nsPNoYNrJgNh7eITwKbH/q99JxSzFwRDgf7Ivbvfd8M363c3ieo7pc18ntfiKFCjMeqxpQQymeilqxU4w5qKJJ9MFHLjCFnk0ypKFvIGmWj2b8v542joYb90Hb3HLj2WF4eIZqiEMXfewtE/4q79HP+Qd22ak4m+DvmlhAPrAjsrd9SUPoOywr4q8Z7/kai++j7x+XhwII7eYFjrgo3CNWnGsbr0Jacg6vgVgMNpXVz4iPrQd5IjZCr/+c0iMAhzRPkkvt+kdJDZOg3NFh47FhErkWyKHc4MQpYEMW6dVWlwu5Iqy5HkHyNqxtdWKTH/QhUu6b9Gq6C1scEIMo6/8UjNsN67D3Ta/ZaaOXxSNOXOsbiIqHpq8TmUnE8BKsuM5BQRi74W0EXqEAb1GlW+ySUHyxFacWnX8CzTDFd3+qdFwjFthglvFC9Ta+336VPvAthwEEnEw8BqAq5lKE0ND1vddI2dJnEf1VfECAjNS8ZvtISDLfb9uROfKJiCf2wPGmAoodOWElYaJt3FOuE4utrNKjROKheCbRkwLjrpv457U3fct/OZu9T2rQ9msm23g= X-Forefront-PRVS: 0332AACBC3 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6069001)(6009001)(39450400003)(39850400002)(39410400002)(39400400002)(39840400002)(81166006)(8676002)(2950100002)(72206003)(6512007)(42882006)(8656002)(5660300001)(110136004)(1076002)(38730400002)(53936002)(3846002)(7736002)(5009440100003)(25786009)(5003940100001)(305945005)(48376002)(50226002)(6506006)(478600001)(42186005)(36756003)(5890100001)(50986999)(6486002)(66066001)(76176999)(189998001)(33646002)(107886003)(47776003)(4326008)(2906002)(110426004); DIR:OUT; SFP:1101; SCL:1; SRVR:DM5PR07MB3099; H:localhost.localdomain; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM5PR07MB3099; 23:/nwn4gQU9UYCrqO9qsqEPHKUDvwxY/ya4u7eQHn5n?= DEYKnRv5qQhOGfi/bx1ML9bSAxJI5RZYGBpMkr6sCqtUy1aBDL9DPSdKlK+EWea/zxAJLs+dFtmH/MEfiOfj7PdGEaFw6EtHBluYJ9xyBgwJ1kEaRJ/bUpPf4sIeg6d3rZqSLyQLuMCYI7pqmcKhSPsD5MfHe83nXDYsPZiu0UXYlqgLYIedPtZ5pT0SVljWsFFe30hkgJsedQC4H91+izgl8DK1VAacZo/N5SXv1yuCY1qbm3tXQxvSmOIYOwXy16HakYq5ByQzoU9oBQtU5xVWWYDwOz6lB8PTgqQVIdikAOO9g3o74JOlJx92weDRDsuJE8G4mKcGM5+GIsCexHjfppLOzXlkQonvQ4J3IPs389d+aT167iWsC8QtIBokDn9VOaWg+DrvUcBJOmFHc4ViB80TBdZzdM5TIv5+uYwD2Qs82qKbnSqJwOlGZmj3GuqBLbfpTvp01WdJZEMpDVm/Vv2xzNsTxLtBkAJQw/uLzrMiJaNypcD3Ud4bNDvUmPQPEmrH4/XRjeh4XxzYOBBHV4HHPdTmUNYCkOEKd3Wuh3dcSjhBRqhRubs/fdobbiKkpM/F/0YU0RBxZKOMtrzZSW9/OM7PNcbB+7Mzd4phXTTgh0hsxHup45+N83im9DOjK8SitdeP7kCqJhIXpE6XkgqshzHSJW/86N2+B/59beAzad17gdTGJ6PZLyw945SzptUMleKyK4adW0reFuLuOmBOiZty/Ygc7WyZ1QQzUv4omW0Dj+aMDqT9BFAHLfW5xXAAptpBl51LD9yOM/WbM3IaXcp9FdpOXRwtFE+AH+v+Qz2iCZ57SjdtvWljSRtujeNHjNddLl9OirCP/tTlPy4xY4HSRtJiWn0HdYfNRPYJnoKmLjmCe7+BGqYzG7qVhfzwYW/8XPIQfdtr4F4gZSlQBLaCOnKQp//aE8Nf0tNp6+2INVzvROZIFsaWBlRHko9pVt9cVeiEYYLgm+jB+e6H3omxPLUB/szBOQiA1jT28rv5xGhphvFfqU79iZ1wMbZUXiPptdE8f2xyWFMrfgUKUQQoUu26h7i04qJ4jdvLOtJLv2Iq3Ig1WXhiAs= X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3099; 6:+w7pvQX+Fdj/xy//AIrIFoHi6dwMJn4VimgRHq0Bn5oiYPYPC3z+gGBuP0pwgVvY9dJwvjAsKbX0E2aPsUKy/Xgyw7B24Mba1R0H41WippfYaQXtzXTPqmZvcEGFSljbAFuHaOer0BKOgVwNa3bw+pOTNp9a7nj53a7Cc+e61L1nsXuIBUPWa2nUuSVZACajyTEqiYU7Vxp/jHwKJdzA7oPK8QiFDRELRM7NsE5SCvFBhwMBhzSl6IbO6C5GMSC1m9QYGgzYlDO5EBeU9zq0zMfJyCgAENC3tf2G2hBJd6lA4aJyYXAL6ltqbx58cawdmN6TRNBQoptoqTjyQWFOPkxW0QJo9KgputxA6p+SLj1pqQ2Lg0LhW3mmDUP1w1cvTEyHYisn3AtlRj9r/xK3+5WiyPc1AeKnLm+N5AokNKPTNgL3oBL8k8iX5CNLjpDaLUO2UpmFnMcdfDJQjmIXF6YQtHN3SwzTMTb45AcUDhj/3IgytbN4WMLqbm70rkD3mZ+JD3L96R49p/Jxban4jA== X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3099; 5:DIX2KH8YMYrml7TBnl4fT6slD58ipufDoEyRwkukBmCi2zZkgrdikPzX5IUE5f+WLn6tqu76ybxlHpYMb/h+qw87/gijE/I52dn1vEEgs4YPzmdsLDlGatzscVC0tEa8Ic0RK16jtKsCiZGXKAhqu1O0XTRS6gErae4LvpSYLGpkHA5Fp22TSIhJ/SFO5NBJxzieQQ2zgdnOluqmswadtglpPPRiQ53WY96AqjqY8BFYGrI8EtaUOghKvk0BSLgvBxH9fadekwac5oDlJUI48deXDjLtWDwdmVKwcymm7WMW4TS3sAKs2f/oNgA0cMWI3bBkPpiqLy9xWX4ErMBZr0Ook3TtIIS3F9QMRICO+9gIC4/DR2MX5WNHFKcnbQUlMHfdt7Hm8tXja8kfJdRsQBP0vekNj4B3qouBj4Kroe8QVih5m5vrryRPRxpqcdRGzK7mwjJL+n2VPXFEzwZEuYv5qc+D2/rnLiqXnoEPaNWH/H65y6WGbTJr/VgNTKfu; 24:nfRowsUXmKyRicbmEe7nW28ShibB9WYISvJVrW1MTsM2zOPR7908vGHkpVJR3zLSJ/knv4dMErX0R9834Dcd4pGedOYPocjKChjIsPezN1M= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DM5PR07MB3099; 7:GV1SckekzS/CB1/4UH+oA2IjPrmWTvyqc1kgc7QkLWhWHygUK/Qm6V842eCgpRosKj5bAgTPImwMOHY+v9d7XX25RPviJs9dWwtZG/eXtYWZtQQYOs0WWcXx+FYTcBI+pVr6JmscpGC1+tgha3BYmMSnpd4nm1/DeXGlUYMiC4r5oU3GyVaGMmgPeElydQlHpsEI4G/CJVyealcGmfYpCjVgnz06tOD1RE/SQSW9OECO7D3DQQZo2JXXNk8hIh4CPsfKzmeyVIZ6tyHJVDW6kk245xGGahts5zs075IggMfJQrzBnZ+pOLaYazPxjE1pLjP6+Vud+KXjeJNLbuhtqQ== X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jun 2017 11:06:07.5165 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR07MB3099 Subject: [dpdk-dev] [PATCH 02/10] linuxapp/eal_pci: get iommu class X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Get iommu class of PCI device on the bus and returns preferred iova mapping mode for that bus. IOVA mapping scheme for linuxapp case: - uio/uio_generic/vfio_noiommu --> default i.e.. (RTE_IOVA_PA) - vfio --> RTE_IOVA_VA. - In case of no device attached to any driver, return RTE_IOVA_DC. Signed-off-by: Santosh Shukla Signed-off-by: Jerin Jacob --- lib/librte_eal/linuxapp/eal/eal_pci.c | 38 +++++++++++++++++++++++++ lib/librte_eal/linuxapp/eal/eal_vfio.c | 23 +++++++++++++++ lib/librte_eal/linuxapp/eal/eal_vfio.h | 4 +++ lib/librte_eal/linuxapp/eal/rte_eal_version.map | 7 +++++ 4 files changed, 72 insertions(+) diff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxapp/eal/eal_pci.c index 595622b21..2772e883e 100644 --- a/lib/librte_eal/linuxapp/eal/eal_pci.c +++ b/lib/librte_eal/linuxapp/eal/eal_pci.c @@ -45,6 +45,7 @@ #include "eal_filesystem.h" #include "eal_private.h" #include "eal_pci_init.h" +#include "eal_vfio.h" /** * @file @@ -488,6 +489,43 @@ rte_pci_scan(void) return -1; } +/* + * Get iommu class of PCI devices on the bus. + * Check that those devices are attached to iommu driver. + * If attached then return iova_va or iova_pa mode, else + * return with dont_care(_DC). + */ +enum rte_iova_mode +rte_pci_get_iommu_class(void) +{ + struct rte_pci_device *dev = NULL; + int ret = RTE_IOVA_DC; + + TAILQ_FOREACH(dev, &rte_pci_bus.device_list, next) { + + if (dev->kdrv == RTE_KDRV_UNKNOWN || + dev->kdrv == RTE_KDRV_NONE) + continue; + + if (dev->kdrv != RTE_KDRV_VFIO) { + ret = RTE_IOVA_PA; + return ret; + } + + ret = RTE_IOVA_VA; + } + + /* In case of iova_va, check for vfio_noiommu mode */ + if (ret == RTE_IOVA_VA) { +#ifdef VFIO_PRESENT + if (vfio_noiommu_is_enabled() == 1) +#endif + ret = RTE_IOVA_PA; + } + + return ret; +} + /* Read PCI config space. */ int rte_pci_read_config(const struct rte_pci_device *device, void *buf, size_t len, off_t offset) diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.c b/lib/librte_eal/linuxapp/eal/eal_vfio.c index 946df7e31..04914406f 100644 --- a/lib/librte_eal/linuxapp/eal/eal_vfio.c +++ b/lib/librte_eal/linuxapp/eal/eal_vfio.c @@ -816,4 +816,27 @@ vfio_noiommu_dma_map(int __rte_unused vfio_container_fd) return 0; } +int +vfio_noiommu_is_enabled(void) +{ +#if LINUX_VERSION_CODE < KERNEL_VERSION(4, 2, 0) + return -1; +#else + int fd, ret, cnt __rte_unused; + char c; + + ret = -1; + fd = open(VFIO_NOIOMMU_MODE, O_RDONLY); + if (fd < 0) + return -1; + + cnt = read(fd, &c, 1); + if (c == 'Y') + ret = 1; + + close(fd); + return ret; +#endif +} + #endif diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.h b/lib/librte_eal/linuxapp/eal/eal_vfio.h index 5ff63e5d7..26ea8e119 100644 --- a/lib/librte_eal/linuxapp/eal/eal_vfio.h +++ b/lib/librte_eal/linuxapp/eal/eal_vfio.h @@ -150,6 +150,8 @@ struct vfio_config { #define VFIO_NOIOMMU_GROUP_FMT "/dev/vfio/noiommu-%u" #define VFIO_GET_REGION_ADDR(x) ((uint64_t) x << 40ULL) #define VFIO_GET_REGION_IDX(x) (x >> 40) +#define VFIO_NOIOMMU_MODE \ + "/sys/module/vfio/parameters/enable_unsafe_noiommu_mode" /* DMA mapping function prototype. * Takes VFIO container fd as a parameter. @@ -210,6 +212,8 @@ int pci_vfio_is_enabled(void); int vfio_mp_sync_setup(void); +int vfio_noiommu_is_enabled(void); + #define SOCKET_REQ_CONTAINER 0x100 #define SOCKET_REQ_GROUP 0x200 #define SOCKET_CLR_GROUP 0x300 diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map index 670bab3a5..2cea7c272 100644 --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map @@ -198,3 +198,10 @@ DPDK_17.05 { vfio_get_group_no; } DPDK_17.02; + +DPDK_17.08 { + global: + + rte_pci_get_iommu_class; + +} DPDK_17.05;