From patchwork Thu Sep 29 12:05:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Olivier Matz X-Patchwork-Id: 117136 X-Patchwork-Delegate: david.marchand@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2ED7DA00C4; Thu, 29 Sep 2022 14:05:25 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 21C2541143; Thu, 29 Sep 2022 14:05:25 +0200 (CEST) Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) by mails.dpdk.org (Postfix) with ESMTP id ED14640395 for ; Thu, 29 Sep 2022 14:05:23 +0200 (CEST) Received: by mail-wr1-f42.google.com with SMTP id s14so1886874wro.0 for ; Thu, 29 Sep 2022 05:05:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind.com; s=google; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date; bh=an/F51QeKLEVtvdGauOVZ95z+Adl+E1/pYMRoAwBhE4=; b=d5Zk9WI+k+SVS3tW9TkecBl2DBd8nQ2mvVgR/5QaQFljIJSEVSLTLHf+ksmSGvnNlI UEYvkArdAnQVCyUIlIxUoSRGVp6oaXKO1yLApVg0vfELq2BZbxj+dl2rXr6puiuSEb// oBWlmuFy6DX70+R3PE+tdCA5zdR4dIjdSy3V+Gn9lZIsPsg4ylUHFnlPMiKIBbpbu/aF dmEKX/+bhebKO3n6CiqFmYTlF64gUt3Ctet7IrcM4uUkswk0XU+rSko0CBvuvCyz8DXV 0z/dxSzXNnFHNXNEQeIGfhSIHZoU+g6vQOABkUcXwwnycNKRMsmU9DVHNPCYvKl+KUVJ L7Xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date; bh=an/F51QeKLEVtvdGauOVZ95z+Adl+E1/pYMRoAwBhE4=; b=WwscmME5L+YgHVjJ46mwBOS5a3ZsxCImN8VYKdCOSbfpU5928h8mjEKQff9Ot3YEET p4ld8tU+pIbM+tzYTICf/N9hoC9oLhiJuhhhUwgIw10s8yrBRHLfykHwnSLlAU4Lg1X9 /oCKF35w7pYS0B68OFTzCEInYQ9Q63IqKMy/lRte9/5FV2BIL2Pd+D+pGPi44jytUFsm bHMfvDeyVr37vPJv8w9hLBcH3tbJfVnfqiTp+suHHWkmQapJ4jyZmIY6Hja3/GIpwfWp nAMsPZURi8J2I3Dog7QgV8/j43Gc3Pno69IIO5XwH3DI0N2NPLYBMSYOHMDso+2iTpSW 2/fg== X-Gm-Message-State: ACrzQf3Rht9bDhhDA1F2Mn9nBl2Jl4O2fCvLsOvZ+zqHfVVXdAYqCJeM y4TnATiJF4ZhlS/e1uIgcEt6Vn5g80uHMQ8u X-Google-Smtp-Source: AMsMyM524Zkt5Gd8WvQdwxOpQwk0Kl0Cny5Yf4fvcc4b1MfjrMGdi91enxCzrD5v48Fr62Wyh3s3Lg== X-Received: by 2002:a5d:52d0:0:b0:21e:4923:fa09 with SMTP id r16-20020a5d52d0000000b0021e4923fa09mr2080802wrv.244.1664453123678; Thu, 29 Sep 2022 05:05:23 -0700 (PDT) Received: from gojira.dev.6wind.com ([185.13.181.2]) by smtp.gmail.com with ESMTPSA id r4-20020adfe684000000b0022ae0965a8asm6494725wrm.24.2022.09.29.05.05.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Sep 2022 05:05:22 -0700 (PDT) From: Olivier Matz To: dev@dpdk.org Cc: Ray Kinsella , Parav Pandit , Xueming Li , Hemant Agrawal , Sachin Saxena , Stephen Hemminger , Long Li Subject: [PATCH] drivers/bus: set device NUMA node to unknown by default Date: Thu, 29 Sep 2022 14:05:12 +0200 Message-Id: <20220929120512.480-1-olivier.matz@6wind.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The dev->device.numa_node field is set by each bus driver for every device it manages to indicate on which NUMA node this device lies. When this information is unknown, the assigned value is not consistent across the bus drivers. Set the default value to SOCKET_ID_ANY (-1) by all bus drivers when the NUMA information is unavailable. Signed-off-by: Olivier Matz --- doc/guides/rel_notes/deprecation.rst | 7 ------- doc/guides/rel_notes/release_22_11.rst | 6 ++++++ drivers/bus/auxiliary/auxiliary_common.c | 8 ++------ drivers/bus/auxiliary/linux/auxiliary.c | 13 +++++-------- drivers/bus/dpaa/dpaa_bus.c | 1 + drivers/bus/fslmc/fslmc_bus.c | 1 + drivers/bus/pci/bsd/pci.c | 2 +- drivers/bus/pci/linux/pci.c | 16 ++++++---------- drivers/bus/pci/pci_common.c | 8 ++------ drivers/bus/pci/windows/pci.c | 1 - drivers/bus/vmbus/linux/vmbus_bus.c | 1 - drivers/bus/vmbus/vmbus_common.c | 8 ++------ 12 files changed, 26 insertions(+), 46 deletions(-) diff --git a/doc/guides/rel_notes/deprecation.rst b/doc/guides/rel_notes/deprecation.rst index be231ff8d2..75645f0a9a 100644 --- a/doc/guides/rel_notes/deprecation.rst +++ b/doc/guides/rel_notes/deprecation.rst @@ -36,13 +36,6 @@ Deprecation Notices ``__atomic_thread_fence`` must be used for patches that need to be merged in 20.08 onwards. This change will not introduce any performance degradation. -* bus: The ``dev->device.numa_node`` field is set by each bus driver for - every device it manages to indicate on which NUMA node this device lies. - When this information is unknown, the assigned value is not consistent - across the bus drivers. - In DPDK 22.11, the default value will be set to -1 by all bus drivers - when the NUMA information is unavailable. - * kni: The KNI kernel module and library are not recommended for use by new applications - other technologies such as virtio-user are recommended instead. Following the DPDK technical board diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst index 0231959874..5989e4a04f 100644 --- a/doc/guides/rel_notes/release_22_11.rst +++ b/doc/guides/rel_notes/release_22_11.rst @@ -266,6 +266,12 @@ ABI Changes * eventdev: Added ``weight`` and ``affinity`` fields to ``rte_event_queue_conf`` structure. +* bus: Changed the device numa node to -1 when NUMA information is unavailable. + The ``dev->device.numa_node`` field is set by each bus driver for + every device it manages to indicate on which NUMA node this device lies. + When this information is unknown, the assigned value was not consistent + across the bus drivers. + Known Issues ------------ diff --git a/drivers/bus/auxiliary/auxiliary_common.c b/drivers/bus/auxiliary/auxiliary_common.c index 259ff152c4..6bb1fe7c96 100644 --- a/drivers/bus/auxiliary/auxiliary_common.c +++ b/drivers/bus/auxiliary/auxiliary_common.c @@ -105,12 +105,8 @@ rte_auxiliary_probe_one_driver(struct rte_auxiliary_driver *drv, return -1; } - if (dev->device.numa_node < 0) { - if (rte_socket_count() > 1) - AUXILIARY_LOG(INFO, "Device %s is not NUMA-aware, defaulting socket to 0", - dev->name); - dev->device.numa_node = 0; - } + if (dev->device.numa_node < 0 && rte_socket_count() > 1) + RTE_LOG(INFO, EAL, "Device %s is not NUMA-aware\n", dev->name); iova_mode = rte_eal_iova_mode(); if ((drv->drv_flags & RTE_AUXILIARY_DRV_NEED_IOVA_AS_VA) > 0 && diff --git a/drivers/bus/auxiliary/linux/auxiliary.c b/drivers/bus/auxiliary/linux/auxiliary.c index d4c564cd78..02fc9285dc 100644 --- a/drivers/bus/auxiliary/linux/auxiliary.c +++ b/drivers/bus/auxiliary/linux/auxiliary.c @@ -40,14 +40,11 @@ auxiliary_scan_one(const char *dirname, const char *name) /* Get NUMA node, default to 0 if not present */ snprintf(filename, sizeof(filename), "%s/%s/numa_node", dirname, name); - if (access(filename, F_OK) != -1) { - if (eal_parse_sysfs_value(filename, &tmp) == 0) - dev->device.numa_node = tmp; - else - dev->device.numa_node = -1; - } else { - dev->device.numa_node = 0; - } + if (access(filename, F_OK) == 0 && + eal_parse_sysfs_value(filename, &tmp) == 0) + dev->device.numa_node = tmp; + else + dev->device.numa_node = SOCKET_ID_ANY; auxiliary_on_scan(dev); diff --git a/drivers/bus/dpaa/dpaa_bus.c b/drivers/bus/dpaa/dpaa_bus.c index 682427ba2c..447b222a76 100644 --- a/drivers/bus/dpaa/dpaa_bus.c +++ b/drivers/bus/dpaa/dpaa_bus.c @@ -179,6 +179,7 @@ dpaa_create_device_list(void) } dev->device.bus = &rte_dpaa_bus.bus; + dev->device.numa_node = SOCKET_ID_ANY; /* Allocate interrupt handle instance */ dev->intr_handle = diff --git a/drivers/bus/fslmc/fslmc_bus.c b/drivers/bus/fslmc/fslmc_bus.c index 8503004e3d..57bfb5111a 100644 --- a/drivers/bus/fslmc/fslmc_bus.c +++ b/drivers/bus/fslmc/fslmc_bus.c @@ -156,6 +156,7 @@ scan_one_fslmc_device(char *dev_name) } dev->device.bus = &rte_fslmc_bus.bus; + dev->device.numa_node = SOCKET_ID_ANY; /* Allocate interrupt instance */ dev->intr_handle = diff --git a/drivers/bus/pci/bsd/pci.c b/drivers/bus/pci/bsd/pci.c index 844d145fed..7459d15c7e 100644 --- a/drivers/bus/pci/bsd/pci.c +++ b/drivers/bus/pci/bsd/pci.c @@ -246,7 +246,7 @@ pci_scan_one(int dev_pci_fd, struct pci_conf *conf) dev->max_vfs = 0; /* FreeBSD has no NUMA support (yet) */ - dev->device.numa_node = 0; + dev->device.numa_node = SOCKET_ID_ANY; pci_common_set(dev); diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c index c8703d52f3..ade17079fc 100644 --- a/drivers/bus/pci/linux/pci.c +++ b/drivers/bus/pci/linux/pci.c @@ -283,17 +283,13 @@ pci_scan_one(const char *dirname, const struct rte_pci_addr *addr) } /* get numa node, default to 0 if not present */ - snprintf(filename, sizeof(filename), "%s/numa_node", - dirname); + snprintf(filename, sizeof(filename), "%s/numa_node", dirname); - if (access(filename, F_OK) != -1) { - if (eal_parse_sysfs_value(filename, &tmp) == 0) - dev->device.numa_node = tmp; - else - dev->device.numa_node = -1; - } else { - dev->device.numa_node = 0; - } + if (access(filename, F_OK) == 0 && + eal_parse_sysfs_value(filename, &tmp) == 0) + dev->device.numa_node = tmp; + else + dev->device.numa_node = SOCKET_ID_ANY; pci_common_set(dev); diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c index 5ea72bcf23..a59c5b4286 100644 --- a/drivers/bus/pci/pci_common.c +++ b/drivers/bus/pci/pci_common.c @@ -235,12 +235,8 @@ rte_pci_probe_one_driver(struct rte_pci_driver *dr, return 1; } - if (dev->device.numa_node < 0) { - if (rte_socket_count() > 1) - RTE_LOG(INFO, EAL, "Device %s is not NUMA-aware, defaulting socket to 0\n", - dev->name); - dev->device.numa_node = 0; - } + if (dev->device.numa_node < 0 && rte_socket_count() > 1) + RTE_LOG(INFO, EAL, "Device %s is not NUMA-aware\n", dev->name); already_probed = rte_dev_is_probed(&dev->device); if (already_probed && !(dr->drv_flags & RTE_PCI_DRV_PROBE_AGAIN)) { diff --git a/drivers/bus/pci/windows/pci.c b/drivers/bus/pci/windows/pci.c index 3f7a8b9432..5cf05ce1a0 100644 --- a/drivers/bus/pci/windows/pci.c +++ b/drivers/bus/pci/windows/pci.c @@ -249,7 +249,6 @@ get_device_resource_info(HDEVINFO dev_info, DWORD error = GetLastError(); if (error == ERROR_NOT_FOUND) { /* On older CPUs, NUMA is not bound to PCIe locality. */ - dev->device.numa_node = 0; return ERROR_SUCCESS; } RTE_LOG_WIN32_ERR("SetupDiGetDevicePropertyW" diff --git a/drivers/bus/vmbus/linux/vmbus_bus.c b/drivers/bus/vmbus/linux/vmbus_bus.c index f502783f7a..01d8111b85 100644 --- a/drivers/bus/vmbus/linux/vmbus_bus.c +++ b/drivers/bus/vmbus/linux/vmbus_bus.c @@ -293,7 +293,6 @@ vmbus_scan_one(const char *name) goto error; dev->device.numa_node = tmp; } else { - /* if no NUMA support, set default to 0 */ dev->device.numa_node = SOCKET_ID_ANY; } diff --git a/drivers/bus/vmbus/vmbus_common.c b/drivers/bus/vmbus/vmbus_common.c index 03b39c82b7..8d32d66504 100644 --- a/drivers/bus/vmbus/vmbus_common.c +++ b/drivers/bus/vmbus/vmbus_common.c @@ -111,12 +111,8 @@ vmbus_probe_one_driver(struct rte_vmbus_driver *dr, /* reference driver structure */ dev->driver = dr; - if (dev->device.numa_node < 0) { - if (rte_socket_count() > 1) - VMBUS_LOG(INFO, "Device %s is not NUMA-aware, defaulting socket to 0", - guid); - dev->device.numa_node = 0; - } + if (dev->device.numa_node < 0 && rte_socket_count() > 1) + VMBUS_LOG(INFO, "Device %s is not NUMA-aware", guid); /* call the driver probe() function */ VMBUS_LOG(INFO, " probe driver: %s", dr->driver.name);