Patch Detail
get:
Show a patch.
patch:
Update a patch.
put:
Update a patch.
GET /api/patches/69316/?format=api
{ "id": 69316, "url": "http://patchwork.dpdk.org/api/patches/69316/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/patch/20200426015514.39489-3-haiyue.wang@intel.com/", "project": { "id": 1, "url": "http://patchwork.dpdk.org/api/projects/1/?format=api", "name": "DPDK", "link_name": "dpdk", "list_id": "dev.dpdk.org", "list_email": "dev@dpdk.org", "web_url": "http://core.dpdk.org", "scm_url": "git://dpdk.org/dpdk", "webscm_url": "http://git.dpdk.org/dpdk", "list_archive_url": "https://inbox.dpdk.org/dev", "list_archive_url_format": "https://inbox.dpdk.org/dev/{}", "commit_url_format": "" }, "msgid": "<20200426015514.39489-3-haiyue.wang@intel.com>", "list_archive_url": "https://inbox.dpdk.org/dev/20200426015514.39489-3-haiyue.wang@intel.com", "date": "2020-04-26T01:55:14", "name": "[v10,2/2] eal: support for VFIO-PCI VF token", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": true, "hash": "a0aea76f1282ab5a616eca3b5a3131887ca96ab7", "submitter": { "id": 1044, "url": "http://patchwork.dpdk.org/api/people/1044/?format=api", "name": "Wang, Haiyue", "email": "haiyue.wang@intel.com" }, "delegate": { "id": 24651, "url": "http://patchwork.dpdk.org/api/users/24651/?format=api", "username": "dmarchand", "first_name": "David", "last_name": "Marchand", "email": "david.marchand@redhat.com" }, "mbox": "http://patchwork.dpdk.org/project/dpdk/patch/20200426015514.39489-3-haiyue.wang@intel.com/mbox/", "series": [ { "id": 9637, "url": "http://patchwork.dpdk.org/api/series/9637/?format=api", "web_url": "http://patchwork.dpdk.org/project/dpdk/list/?series=9637", "date": "2020-04-26T01:55:12", "name": "support for VFIO-PCI VF token interface", "version": 10, "mbox": "http://patchwork.dpdk.org/series/9637/mbox/" } ], "comments": "http://patchwork.dpdk.org/api/patches/69316/comments/", "check": "warning", "checks": "http://patchwork.dpdk.org/api/patches/69316/checks/", "tags": {}, "related": [], "headers": { "Return-Path": "<dev-bounces@dpdk.org>", "X-Original-To": "patchwork@inbox.dpdk.org", "Delivered-To": "patchwork@inbox.dpdk.org", "Received": [ "from dpdk.org (dpdk.org [92.243.14.124])\n\tby inbox.dpdk.org (Postfix) with ESMTP id 03BF1A00C2;\n\tSun, 26 Apr 2020 04:01:30 +0200 (CEST)", "from [92.243.14.124] (localhost [127.0.0.1])\n\tby dpdk.org (Postfix) with ESMTP id B36B71C0B2;\n\tSun, 26 Apr 2020 04:01:21 +0200 (CEST)", "from mga17.intel.com (mga17.intel.com [192.55.52.151])\n by dpdk.org (Postfix) with ESMTP id 12BBF1C066\n for <dev@dpdk.org>; Sun, 26 Apr 2020 04:01:16 +0200 (CEST)", "from fmsmga003.fm.intel.com ([10.253.24.29])\n by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;\n 25 Apr 2020 19:01:16 -0700", "from npg-dpdk-haiyue-3.sh.intel.com ([10.67.119.46])\n by FMSMGA003.fm.intel.com with ESMTP; 25 Apr 2020 19:01:14 -0700" ], "IronPort-SDR": [ "\n U1lL1XTcejKUuMHWGvAHxDCy/2MdfphUMsv+rLdB7JzXj/uiEjeO6urLq2NEdAHsQ+DKkxbyNH\n PRJj/hbvhQgw==", "\n 5AMM8q05xsrG04u0Jgr04woXtiTZnw2R0Y7Wlo38M1KFm6CuoF9JHm+ecU/hOA6v5fL7dY/rET\n GuwNawFGe5AA==" ], "X-Amp-Result": "SKIPPED(no attachment in message)", "X-Amp-File-Uploaded": "False", "X-ExtLoop1": "1", "X-IronPort-AV": "E=Sophos;i=\"5.73,318,1583222400\"; d=\"scan'208\";a=\"301980182\"", "From": "Haiyue Wang <haiyue.wang@intel.com>", "To": "dev@dpdk.org, anatoly.burakov@intel.com, thomas@monjalon.net,\n vattunuru@marvell.com, jerinj@marvell.com, alex.williamson@redhat.com,\n david.marchand@redhat.com", "Cc": "Haiyue Wang <haiyue.wang@intel.com>", "Date": "Sun, 26 Apr 2020 09:55:14 +0800", "Message-Id": "<20200426015514.39489-3-haiyue.wang@intel.com>", "X-Mailer": "git-send-email 2.26.2", "In-Reply-To": "<20200426015514.39489-1-haiyue.wang@intel.com>", "References": "<20200305043311.17065-1-vattunuru@marvell.com>\n <20200426015514.39489-1-haiyue.wang@intel.com>", "MIME-Version": "1.0", "Content-Type": "text/plain; charset=UTF-8", "Content-Transfer-Encoding": "8bit", "Subject": "[dpdk-dev] [PATCH v10 2/2] eal: support for VFIO-PCI VF token", "X-BeenThere": "dev@dpdk.org", "X-Mailman-Version": "2.1.15", "Precedence": "list", "List-Id": "DPDK patches and discussions <dev.dpdk.org>", "List-Unsubscribe": "<https://mails.dpdk.org/options/dev>,\n <mailto:dev-request@dpdk.org?subject=unsubscribe>", "List-Archive": "<http://mails.dpdk.org/archives/dev/>", "List-Post": "<mailto:dev@dpdk.org>", "List-Help": "<mailto:dev-request@dpdk.org?subject=help>", "List-Subscribe": "<https://mails.dpdk.org/listinfo/dev>,\n <mailto:dev-request@dpdk.org?subject=subscribe>", "Errors-To": "dev-bounces@dpdk.org", "Sender": "\"dev\" <dev-bounces@dpdk.org>" }, "content": "The kernel module vfio-pci introduces the VF token to enable SR-IOV\nsupport since 5.7.\n\nThe VF token can be set by a vfio-pci based PF driver and must be known\nby the vfio-pci based VF driver in order to gain access to the device.\n\nSigned-off-by: Haiyue Wang <haiyue.wang@intel.com>\nAcked-by: Vamsi Attunuru <vattunuru@marvell.com>\nTested-by: Vamsi Attunuru <vattunuru@marvell.com>\n---\n doc/guides/linux_gsg/linux_drivers.rst | 41 +++++++++++++-\n doc/guides/rel_notes/release_20_05.rst | 6 +++\n drivers/bus/pci/linux/pci_vfio.c | 74 +++++++++++++++++++++++++-\n lib/librte_eal/freebsd/eal.c | 3 +-\n lib/librte_eal/include/rte_vfio.h | 26 ++++++++-\n lib/librte_eal/linux/eal_vfio.c | 20 +++++--\n lib/librte_eal/rte_eal_version.map | 8 ++-\n 7 files changed, 169 insertions(+), 9 deletions(-)", "diff": "diff --git a/doc/guides/linux_gsg/linux_drivers.rst b/doc/guides/linux_gsg/linux_drivers.rst\nindex 238f3e900..b42fd708b 100644\n--- a/doc/guides/linux_gsg/linux_drivers.rst\n+++ b/doc/guides/linux_gsg/linux_drivers.rst\n@@ -72,11 +72,50 @@ Note that in order to use VFIO, your kernel must support it.\n VFIO kernel modules have been included in the Linux kernel since version 3.6.0 and are usually present by default,\n however please consult your distributions documentation to make sure that is the case.\n \n+The ``vfio-pci`` module since Linux version 5.7 supports the creation of virtual\n+functions. After the PF is bound to vfio-pci module, the user can create the VFs\n+by sysfs interface, and these VFs are bound to vfio-pci module automatically.\n+\n+When the PF is bound to vfio-pci, it has initial VF token generated by random. For\n+security reason, this token is write only, the user can't read it from the kernel\n+directly. For accessing the VF, the user needs to start the PF with token parameter\n+to setup a VF token (uuid format), then the VF can be accessed with this new known\n+VF token.\n+\n+Also if the DPDK application running on the PF device exits, the user wants to start\n+the PF with another different VF token value, it has no issue if no application like\n+DPDK or KVM runs on VFs. Otherwise, the PF will fail to start until all VFs are free\n+to use, after that, the user can select a new VF token to start the PF device.\n+\n+DPDK will use the keyword ``vf_token`` as the device argument to pass the VF token\n+value to PF and its related VFs, the PMD should not use it, and this argument will\n+be pruned from the device argument list, so the PMD can parse its own valid device\n+arguments successfully.\n+\n+.. code-block:: console\n+\n+ 1. Generate the VF token by uuid command\n+ 14d63f20-8445-11ea-8900-1f9ce7d5650d\n+\n+ 2. sudo modprobe vfio-pci enable_sriov=1\n+\n+ 2. ./usertools/dpdk-devbind.py -b vfio-pci 0000:86:00.0\n+\n+ 3. echo 2 > /sys/bus/pci/devices/0000:86:00.0/sriov_numvfs\n+\n+ 4. Start the PF:\n+ ./x86_64-native-linux-gcc/app/testpmd -l 22-25 -n 4 \\\n+ -w 86:00.0,vf_token=14d63f20-8445-11ea-8900-1f9ce7d5650d --file-prefix=pf -- -i\n+\n+ 5. Start the VF:\n+ ./x86_64-native-linux-gcc/app/testpmd -l 26-29 -n 4 \\\n+ -w 86:02.0,vf_token=14d63f20-8445-11ea-8900-1f9ce7d5650d --file-prefix=vf0 -- -i\n+\n Also, to use VFIO, both kernel and BIOS must support and be configured to use IO virtualization (such as Intel® VT-d).\n \n .. note::\n \n- ``vfio-pci`` module doesn't support the creation of virtual functions.\n+ ``vfio-pci`` module doesn't support the creation of virtual functions before Linux version 5.7.\n \n For proper operation of VFIO when running DPDK applications as a non-privileged user, correct permissions should also be set up.\n This can be done by using the DPDK setup script (called dpdk-setup.sh and located in the usertools directory).\ndiff --git a/doc/guides/rel_notes/release_20_05.rst b/doc/guides/rel_notes/release_20_05.rst\nindex b124c3f28..722c61e67 100644\n--- a/doc/guides/rel_notes/release_20_05.rst\n+++ b/doc/guides/rel_notes/release_20_05.rst\n@@ -93,6 +93,12 @@ New Features\n * Added new query: ``rte_flow_get_aged_flows`` to get the aged-out flows\n contexts from the port.\n \n+* **Added the support for vfio-pci new VF token interface.**\n+\n+ Since Linux version 5.7, vfio-pci supports a shared VF token (UUID) to represent\n+ the trust between SR-IOV PF and the created VFs. Update the method to gain access\n+ to the PF and VFs devices by appending the VF token parameter.\n+\n * **Updated Amazon ena driver.**\n \n Updated ena PMD with new features and improvements, including:\ndiff --git a/drivers/bus/pci/linux/pci_vfio.c b/drivers/bus/pci/linux/pci_vfio.c\nindex 64cd84a68..efb64e2ba 100644\n--- a/drivers/bus/pci/linux/pci_vfio.c\n+++ b/drivers/bus/pci/linux/pci_vfio.c\n@@ -11,6 +11,7 @@\n #include <sys/mman.h>\n #include <stdbool.h>\n \n+#include <rte_devargs.h>\n #include <rte_log.h>\n #include <rte_pci.h>\n #include <rte_bus_pci.h>\n@@ -644,12 +645,72 @@ pci_vfio_msix_is_mappable(int vfio_dev_fd, int msix_region)\n \treturn ret;\n }\n \n+static int\n+vfio_pci_vf_token_arg(struct rte_devargs *devargs, rte_uuid_t uuid)\n+{\n+#define VF_TOKEN_ARG \"vf_token=\"\n+\tchar c, *p, *vf_token;\n+\n+\tmemset(uuid, 0, sizeof(rte_uuid_t));\n+\n+\tif (devargs == NULL)\n+\t\treturn 0;\n+\n+\tp = strstr(devargs->args, VF_TOKEN_ARG);\n+\tif (!p)\n+\t\treturn 0;\n+\n+\tvf_token = p + strlen(VF_TOKEN_ARG);\n+\tif (strlen(vf_token) < (RTE_UUID_STRLEN - 1)) {\n+\t\tRTE_LOG(ERR, EAL, \"The VF token length is too short\\n\");\n+\t\treturn -1;\n+\t}\n+\n+\tc = vf_token[RTE_UUID_STRLEN - 1];\n+\tif (c != '\\0' && c != ',') {\n+\t\tRTE_LOG(ERR, EAL,\n+\t\t\t\"The VF token ends with a invalid character : %c\\n\", c);\n+\t\treturn -1;\n+\t}\n+\n+\tvf_token[RTE_UUID_STRLEN - 1] = '\\0';\n+\tif (rte_uuid_parse(vf_token, uuid)) {\n+\t\tRTE_LOG(ERR, EAL,\n+\t\t\t\"The VF token is invalid : %s\\n\", vf_token);\n+\t\tvf_token[RTE_UUID_STRLEN - 1] = c;\n+\t\treturn -1;\n+\t}\n+\n+\tRTE_LOG(DEBUG, EAL,\n+\t\t\"The VF token is found : %s\\n\", vf_token);\n+\n+\tvf_token[RTE_UUID_STRLEN - 1] = c;\n+\n+\t/* This VF token will be treated as a invalid device argument if the\n+\t * PMD calls the rte_devargs parse API with its own valid argument list,\n+\t * so it needs to purge this vfio-pci specific argument.\n+\t */\n+\tif (c != '\\0') {\n+\t\t/* 1. Handle the case : 'vf_token=uuid,arg1=val1' */\n+\t\tmemmove(p, vf_token + RTE_UUID_STRLEN,\n+\t\t\tstrlen(vf_token + RTE_UUID_STRLEN) + 1);\n+\t} else {\n+\t\t/* 2. Handle the case : 'arg1=val1,vf_token=uuid' */\n+\t\tif (p != devargs->args)\n+\t\t\tp--;\n+\n+\t\t*p = '\\0';\n+\t}\n+\n+\treturn 0;\n+}\n \n static int\n pci_vfio_map_resource_primary(struct rte_pci_device *dev)\n {\n \tstruct vfio_device_info device_info = { .argsz = sizeof(device_info) };\n \tchar pci_addr[PATH_MAX] = {0};\n+\trte_uuid_t vf_token;\n \tint vfio_dev_fd;\n \tstruct rte_pci_addr *loc = &dev->addr;\n \tint i, ret;\n@@ -668,8 +729,12 @@ pci_vfio_map_resource_primary(struct rte_pci_device *dev)\n \tsnprintf(pci_addr, sizeof(pci_addr), PCI_PRI_FMT,\n \t\t\tloc->domain, loc->bus, loc->devid, loc->function);\n \n+\tret = vfio_pci_vf_token_arg(dev->device.devargs, vf_token);\n+\tif (ret)\n+\t\treturn ret;\n+\n \tret = rte_vfio_setup_device(rte_pci_get_sysfs_path(), pci_addr,\n-\t\t\t\t\t&vfio_dev_fd, &device_info);\n+\t\t\t\t\t&vfio_dev_fd, &device_info, vf_token);\n \tif (ret)\n \t\treturn ret;\n \n@@ -798,6 +863,7 @@ pci_vfio_map_resource_secondary(struct rte_pci_device *dev)\n {\n \tstruct vfio_device_info device_info = { .argsz = sizeof(device_info) };\n \tchar pci_addr[PATH_MAX] = {0};\n+\trte_uuid_t vf_token;\n \tint vfio_dev_fd;\n \tstruct rte_pci_addr *loc = &dev->addr;\n \tint i, ret;\n@@ -830,8 +896,12 @@ pci_vfio_map_resource_secondary(struct rte_pci_device *dev)\n \t\treturn -1;\n \t}\n \n+\tret = vfio_pci_vf_token_arg(dev->device.devargs, vf_token);\n+\tif (ret)\n+\t\treturn ret;\n+\n \tret = rte_vfio_setup_device(rte_pci_get_sysfs_path(), pci_addr,\n-\t\t\t\t\t&vfio_dev_fd, &device_info);\n+\t\t\t\t\t&vfio_dev_fd, &device_info, vf_token);\n \tif (ret)\n \t\treturn ret;\n \ndiff --git a/lib/librte_eal/freebsd/eal.c b/lib/librte_eal/freebsd/eal.c\nindex 540b7d38c..e42d3e12e 100644\n--- a/lib/librte_eal/freebsd/eal.c\n+++ b/lib/librte_eal/freebsd/eal.c\n@@ -1005,7 +1005,8 @@ rte_eal_vfio_intr_mode(void)\n int rte_vfio_setup_device(__rte_unused const char *sysfs_base,\n \t\t __rte_unused const char *dev_addr,\n \t\t __rte_unused int *vfio_dev_fd,\n-\t\t __rte_unused struct vfio_device_info *device_info)\n+\t\t __rte_unused struct vfio_device_info *device_info,\n+\t\t __rte_unused rte_uuid_t vf_token)\n {\n \treturn -1;\n }\ndiff --git a/lib/librte_eal/include/rte_vfio.h b/lib/librte_eal/include/rte_vfio.h\nindex 20ed8c45a..430f98770 100644\n--- a/lib/librte_eal/include/rte_vfio.h\n+++ b/lib/librte_eal/include/rte_vfio.h\n@@ -16,6 +16,9 @@ extern \"C\" {\n \n #include <stdint.h>\n \n+#include <rte_compat.h>\n+#include <rte_uuid.h>\n+\n /*\n * determine if VFIO is present on the system\n */\n@@ -102,13 +105,34 @@ struct vfio_device_info;\n * @param device_info\n * Device information.\n *\n+ * @param vf_token\n+ * Before linux 5.7, the PF bound to vfio-pci doesn't support SR-IOV to\n+ * create VFs for security reason. Now the VF token is introduced to work\n+ * as some degree of trust or collaboration between PF and VFs.\n+ *\n+ * A). as VF device, if the PF is a vfio device and it is bound to the\n+ * vfio-pci driver, the user needs to provide a VF token to access the\n+ * device, in the form of appending a vf_token to the device name, for\n+ * example:\n+ * \"-w 04:10.0,vf_token=bd8d9d2b-5a5f-4f5a-a211-f591514ba1f3\"\n+ *\n+ * B). as PF device, When presented with a PF which has VFs in use, the\n+ * user must also provide the current VF token to prove collaboration with\n+ * existing VF users. If VFs are not in use, the VF token provided for the\n+ * PF device will act to set the VF token.\n+ *\n+ * The vf_token can be zero uuid, which will be ignored to pass into the\n+ * vfio-pci module.\n+ *\n * @return\n * 0 on success.\n * <0 on failure.\n * >1 if the device cannot be managed this way.\n */\n+__rte_internal\n int rte_vfio_setup_device(const char *sysfs_base, const char *dev_addr,\n-\t\tint *vfio_dev_fd, struct vfio_device_info *device_info);\n+\t\tint *vfio_dev_fd, struct vfio_device_info *device_info,\n+\t\trte_uuid_t vf_token);\n \n /**\n * Release a device mapped to a VFIO-managed I/O MMU group.\ndiff --git a/lib/librte_eal/linux/eal_vfio.c b/lib/librte_eal/linux/eal_vfio.c\nindex d26e1649a..e8d7cbda5 100644\n--- a/lib/librte_eal/linux/eal_vfio.c\n+++ b/lib/librte_eal/linux/eal_vfio.c\n@@ -702,7 +702,8 @@ rte_vfio_clear_group(int vfio_group_fd)\n \n int\n rte_vfio_setup_device(const char *sysfs_base, const char *dev_addr,\n-\t\tint *vfio_dev_fd, struct vfio_device_info *device_info)\n+\t\tint *vfio_dev_fd, struct vfio_device_info *device_info,\n+\t\trte_uuid_t vf_token)\n {\n \tstruct vfio_group_status group_status = {\n \t\t\t.argsz = sizeof(group_status)\n@@ -712,6 +713,7 @@ rte_vfio_setup_device(const char *sysfs_base, const char *dev_addr,\n \tint vfio_container_fd;\n \tint vfio_group_fd;\n \tint iommu_group_num;\n+\tchar dev[PATH_MAX];\n \tint i, ret;\n \n \t/* get group number */\n@@ -895,8 +897,19 @@ rte_vfio_setup_device(const char *sysfs_base, const char *dev_addr,\n \t\t\t\tt->type_id, t->name);\n \t}\n \n+\tif (!rte_uuid_is_null(vf_token)) {\n+\t\tchar vf_token_str[RTE_UUID_STRLEN];\n+\n+\t\trte_uuid_unparse(vf_token, vf_token_str, sizeof(vf_token_str));\n+\t\tsnprintf(dev, sizeof(dev),\n+\t\t\t \"%s vf_token=%s\", dev_addr, vf_token_str);\n+\t} else {\n+\t\tsnprintf(dev, sizeof(dev),\n+\t\t\t \"%s\", dev_addr);\n+\t}\n+\n \t/* get a file descriptor for the device */\n-\t*vfio_dev_fd = ioctl(vfio_group_fd, VFIO_GROUP_GET_DEVICE_FD, dev_addr);\n+\t*vfio_dev_fd = ioctl(vfio_group_fd, VFIO_GROUP_GET_DEVICE_FD, dev);\n \tif (*vfio_dev_fd < 0) {\n \t\t/* if we cannot get a device fd, this implies a problem with\n \t\t * the VFIO group or the container not having IOMMU configured.\n@@ -2083,7 +2096,8 @@ int\n rte_vfio_setup_device(__rte_unused const char *sysfs_base,\n \t\t__rte_unused const char *dev_addr,\n \t\t__rte_unused int *vfio_dev_fd,\n-\t\t__rte_unused struct vfio_device_info *device_info)\n+\t\t__rte_unused struct vfio_device_info *device_info,\n+\t\t__rte_unused rte_uuid_t vf_token)\n {\n \treturn -1;\n }\ndiff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map\nindex 6088e7f6c..5bcdccae6 100644\n--- a/lib/librte_eal/rte_eal_version.map\n+++ b/lib/librte_eal/rte_eal_version.map\n@@ -213,7 +213,6 @@ DPDK_20.0 {\n \trte_vfio_is_enabled;\n \trte_vfio_noiommu_is_enabled;\n \trte_vfio_release_device;\n-\trte_vfio_setup_device;\n \trte_vlog;\n \trte_zmalloc;\n \trte_zmalloc_socket;\n@@ -388,3 +387,10 @@ EXPERIMENTAL {\n \trte_trace_regexp;\n \trte_trace_save;\n };\n+\n+INTERNAL {\n+\tglobal:\n+\n+\t# added in 20.05\n+\trte_vfio_setup_device;\n+};\n", "prefixes": [ "v10", "2/2" ] }