From patchwork Thu Sep 14 12:36:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 131416 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5921842597; Thu, 14 Sep 2023 14:36:49 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 244F440693; Thu, 14 Sep 2023 14:36:38 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 1FEDF4069F for ; Thu, 14 Sep 2023 14:36:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694694994; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=W+DPt7GcYNNdupYDddAk6RuyPSHvJQTiyiJAWNXt9lg=; b=Q04y5YKBj6AI9AIQ3R4h81/U33HJkzW4bzkePEzsrZtfrkkoo2r3GC2EMdNKc14fL+IjAz IaLVKLT9jyuby+m7K7DpBONiO4XCE6p8mha4ChO8Z/yCoLw6RGOcfJiNVNraIIWladF6Am Q97VMC8GFm7L37UJ9ralD40OKX1CJy0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-33-ET0E_fTEP7mjU565CpEerg-1; Thu, 14 Sep 2023 08:36:31 -0400 X-MC-Unique: ET0E_fTEP7mjU565CpEerg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EC53D101FAA1; Thu, 14 Sep 2023 12:36:30 +0000 (UTC) Received: from dmarchan.redhat.com (unknown [10.45.225.25]) by smtp.corp.redhat.com (Postfix) with ESMTP id A225D21B2413; Thu, 14 Sep 2023 12:36:29 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: thomas@monjalon.net, ferruh.yigit@amd.com, chenbo.xia@intel.com, nipun.gupta@amd.com, bruce.richardson@intel.com, Anatoly Burakov Subject: [PATCH v3 03/15] bus/pci: rework MSIX discovery with VFIO Date: Thu, 14 Sep 2023 14:36:02 +0200 Message-ID: <20230914123615.1705654-4-david.marchand@redhat.com> In-Reply-To: <20230914123615.1705654-1-david.marchand@redhat.com> References: <20230803075038.307012-1-david.marchand@redhat.com> <20230914123615.1705654-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This is a preparatory step before using new helpers for finding PCI capabilities. In the code querying PCI capabilities for checking MSIX availability, replace direct calls to VFIO fd with the existing helpers for reading PCI configuration space: this requires setting VFIO fd in the PCI device object than was done before this change and removes the need to pass around this vfio_dev_fd variable. Signed-off-by: David Marchand Reviewed-by: Chenbo Xia --- drivers/bus/pci/linux/pci_vfio.c | 27 ++++++++++----------------- 1 file changed, 10 insertions(+), 17 deletions(-) diff --git a/drivers/bus/pci/linux/pci_vfio.c b/drivers/bus/pci/linux/pci_vfio.c index 8fa7fa458f..958f8b3b52 100644 --- a/drivers/bus/pci/linux/pci_vfio.c +++ b/drivers/bus/pci/linux/pci_vfio.c @@ -107,23 +107,16 @@ pci_vfio_write_config(const struct rte_pci_device *dev, /* get PCI BAR number where MSI-X interrupts are */ static int -pci_vfio_get_msix_bar(const struct rte_pci_device *dev, int fd, +pci_vfio_get_msix_bar(const struct rte_pci_device *dev, struct pci_msix_table *msix_table) { int ret; uint32_t reg; uint16_t flags; uint8_t cap_id, cap_offset; - uint64_t size, offset; - - if (pci_vfio_get_region(dev, VFIO_PCI_CONFIG_REGION_INDEX, - &size, &offset) != 0) { - RTE_LOG(ERR, EAL, "Cannot get offset of CONFIG region.\n"); - return -1; - } /* read PCI capability pointer from config space */ - ret = pread64(fd, ®, sizeof(reg), offset + PCI_CAPABILITY_LIST); + ret = rte_pci_read_config(dev, ®, sizeof(reg), PCI_CAPABILITY_LIST); if (ret != sizeof(reg)) { RTE_LOG(ERR, EAL, "Cannot read capability pointer from PCI config space!\n"); @@ -136,7 +129,7 @@ pci_vfio_get_msix_bar(const struct rte_pci_device *dev, int fd, while (cap_offset) { /* read PCI capability ID */ - ret = pread64(fd, ®, sizeof(reg), offset + cap_offset); + ret = rte_pci_read_config(dev, ®, sizeof(reg), cap_offset); if (ret != sizeof(reg)) { RTE_LOG(ERR, EAL, "Cannot read capability ID from PCI config space!\n"); @@ -148,7 +141,7 @@ pci_vfio_get_msix_bar(const struct rte_pci_device *dev, int fd, /* if we haven't reached MSI-X, check next capability */ if (cap_id != PCI_CAP_ID_MSIX) { - ret = pread64(fd, ®, sizeof(reg), offset + cap_offset); + ret = rte_pci_read_config(dev, ®, sizeof(reg), cap_offset); if (ret != sizeof(reg)) { RTE_LOG(ERR, EAL, "Cannot read capability pointer from PCI config space!\n"); @@ -163,14 +156,14 @@ pci_vfio_get_msix_bar(const struct rte_pci_device *dev, int fd, /* else, read table offset */ else { /* table offset resides in the next 4 bytes */ - ret = pread64(fd, ®, sizeof(reg), offset + cap_offset + 4); + ret = rte_pci_read_config(dev, ®, sizeof(reg), cap_offset + 4); if (ret != sizeof(reg)) { RTE_LOG(ERR, EAL, "Cannot read table offset from PCI config space!\n"); return -1; } - ret = pread64(fd, &flags, sizeof(flags), offset + cap_offset + 2); + ret = rte_pci_read_config(dev, &flags, sizeof(flags), cap_offset + 2); if (ret != sizeof(flags)) { RTE_LOG(ERR, EAL, "Cannot read table flags from PCI config space!\n"); @@ -306,9 +299,6 @@ pci_vfio_setup_interrupts(struct rte_pci_device *dev, int vfio_dev_fd) if (rte_intr_fd_set(dev->intr_handle, fd)) return -1; - if (rte_intr_dev_fd_set(dev->intr_handle, vfio_dev_fd)) - return -1; - switch (i) { case VFIO_PCI_MSIX_IRQ_INDEX: intr_mode = RTE_INTR_MODE_MSIX; @@ -838,6 +828,9 @@ pci_vfio_map_resource_primary(struct rte_pci_device *dev) if (ret) return ret; + if (rte_intr_dev_fd_set(dev->intr_handle, vfio_dev_fd)) + goto err_vfio_dev_fd; + /* allocate vfio_res and get region info */ vfio_res = rte_zmalloc("VFIO_RES", sizeof(*vfio_res), 0); if (vfio_res == NULL) { @@ -869,7 +862,7 @@ pci_vfio_map_resource_primary(struct rte_pci_device *dev) /* get MSI-X BAR, if any (we have to know where it is because we can't * easily mmap it when using VFIO) */ - ret = pci_vfio_get_msix_bar(dev, vfio_dev_fd, &vfio_res->msix_table); + ret = pci_vfio_get_msix_bar(dev, &vfio_res->msix_table); if (ret < 0) { RTE_LOG(ERR, EAL, "%s cannot get MSI-X BAR number!\n", pci_addr);