From patchwork Fri Nov 24 10:09:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Ehrhardt X-Patchwork-Id: 134605 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6538A433B7; Fri, 24 Nov 2023 11:09:12 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 39273402B3; Fri, 24 Nov 2023 11:09:12 +0100 (CET) Received: from smtp-relay-internal-0.canonical.com (smtp-relay-internal-0.canonical.com [185.125.188.122]) by mails.dpdk.org (Postfix) with ESMTP id E52C240283 for ; Fri, 24 Nov 2023 11:09:10 +0100 (CET) Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id A0126402D4 for ; Fri, 24 Nov 2023 10:09:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1700820550; bh=L8u+0eBXDRe0Q7KiOzA8vc6ifYXDHSd1EyXRzF+EFwE=; h=From:To:Cc:Subject:Date:Message-Id:Reply-To:MIME-Version; b=SeLxvl4d0k7SK4lD+U96zzSNU5/CbQYOrDhq8W2JfDXEpXCI40Uja7UIGtc8uRfGH t7DDaDs1/PeXlRs9X1ROHAO9FAwPsmq/mgu8hJJJhpNHtlBBojgkCzxVIoRx5+YBCs ZmlketEhRkxwGHNYuBhNZXOAkP5Ke+tsqrkpHcZfpD4g3K+uHMUNJJBzmOubarpBjO zCEkI7HhuDCjnPdauFFUCGiYJtVWkyYWSN6jJzVc6uZcsOCTcnhtlEmz3syF/BRuxf quOafTPW8eO2SRkR8+Sag+ldzxsY72LIJmmH6XSLJMGBqR4DkTfZF6Ar18B70V11fa kNtAO45NzbzWA== Received: by mail-ej1-f72.google.com with SMTP id a640c23a62f3a-a01991968f5so121660666b.2 for ; Fri, 24 Nov 2023 02:09:10 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700820549; x=1701425349; h=content-transfer-encoding:mime-version:reply-to:message-id:date :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=L8u+0eBXDRe0Q7KiOzA8vc6ifYXDHSd1EyXRzF+EFwE=; b=h3SB8lCz2pxVI3/ENXnLD+TBINQIfZ3bkSmVtCzZZXEIS+xqcSFqA2rISkB4NWq9i9 1mob8S81TLwMFmUEYj6ncjgQMGjbASWwUhpyrpDSZlaHDkln8ocjKM8Yl+YsF4uGJFkh RYsgpO0tSoVW3nR1M68d4KuYvfqgUe2uev3Fg5xaPWAomptLqGjV9YPQmLaZvwS25oec ifUhwNPmMp4kTnlXSYaZV6X9ZFvc2gsMBgdTxDXQyXONkeNBfToujnr1daLSB1XYu6rH +IMB2PzyGBzh+krDjTqqRsk7a4DHz3BTdgHjhC7roOY3rIisJX6eSTSZvL3ryqUVphjS zcFg== X-Gm-Message-State: AOJu0YxdAwIOHQjY1IWSmV+Phe970t3IoEcxWuv42Vq3eRrxaBDPv30x cSUzc1ioovkGfkoBus1szbxGjIIljXDr9wDmPRA98p63dMsAlBdLgDFBP3u1m/5Wtzfc/XRm320 7g8JeUbHE/Io6EBXOupijLkqH4ozs X-Received: by 2002:a17:906:d210:b0:a08:17de:6409 with SMTP id w16-20020a170906d21000b00a0817de6409mr1622005ejz.7.1700820549675; Fri, 24 Nov 2023 02:09:09 -0800 (PST) X-Google-Smtp-Source: AGHT+IFAvdqIYXg4dxT6u0UjeQKI5nsaL5VTV9UcLJz0IO3/WtL5oz6goll0CJs7fxS3lIZ8eZyY0Q== X-Received: by 2002:a17:906:d210:b0:a08:17de:6409 with SMTP id w16-20020a170906d21000b00a0817de6409mr1621984ejz.7.1700820549268; Fri, 24 Nov 2023 02:09:09 -0800 (PST) Received: from Keschdeichel.fritz.box ([2a02:6d40:39cc:3f00:4db5:969c:3195:7ad9]) by smtp.gmail.com with ESMTPSA id w21-20020a170906481500b00977cad140a8sm1872156ejq.218.2023.11.24.02.09.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Nov 2023 02:09:08 -0800 (PST) From: christian.ehrhardt@canonical.com To: dev Cc: Luca Boccassi , David Wilder , Christian Ehrhardt Subject: [PATCH v2] eal/linux: force iova-mode va without pa available Date: Fri, 24 Nov 2023 11:09:04 +0100 Message-Id: <20231124100904.388453-1-christian.ehrhardt@canonical.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: --no-chain-reply-to@dpdk.org Errors-To: dev-bounces@dpdk.org From: David Wilder When using --no-huge option physical address are not guaranteed to be persistent. This change effectively makes "--no-huge" the same as "--no-huge --iova-mode=va". When --no-huge is used (or any other condition making physical addresses unavailable) setting --iova-mode=pa will have no effect. Signed-off-by: Christian Ehrhardt --- doc/guides/prog_guide/env_abstraction_layer.rst | 9 ++++++--- lib/eal/linux/eal.c | 16 ++++++++++------ 2 files changed, 16 insertions(+), 9 deletions(-) diff --git a/doc/guides/prog_guide/env_abstraction_layer.rst b/doc/guides/prog_guide/env_abstraction_layer.rst index 6debf54efb..20c7355e0f 100644 --- a/doc/guides/prog_guide/env_abstraction_layer.rst +++ b/doc/guides/prog_guide/env_abstraction_layer.rst @@ -559,9 +559,12 @@ IOVA Mode is selected by considering what the current usable Devices on the system require and/or support. On FreeBSD, RTE_IOVA_PA is always the default. On Linux, the IOVA mode is -detected based on a 2-step heuristic detailed below. +detected based on a heuristic detailed below. -For the first step, EAL asks each bus its requirement in terms of IOVA mode +For the first step, if no Physical Addresses are available RTE_IOVA_VA is +selected. + +Then EAL asks each bus its requirement in terms of IOVA mode and decides on a preferred IOVA mode. - if all buses report RTE_IOVA_PA, then the preferred IOVA mode is RTE_IOVA_PA, @@ -575,7 +578,7 @@ and decides on a preferred IOVA mode. If the buses have expressed no preference on which IOVA mode to pick, then a default is selected using the following logic: -- if physical addresses are not available, RTE_IOVA_VA mode is used +- if enable_iova_as_pa was not set at build RTE_IOVA_VA mode is used - if /sys/kernel/iommu_groups is not empty, RTE_IOVA_VA mode is used - otherwise, RTE_IOVA_PA mode is used diff --git a/lib/eal/linux/eal.c b/lib/eal/linux/eal.c index 57da058cec..2f1fce3c54 100644 --- a/lib/eal/linux/eal.c +++ b/lib/eal/linux/eal.c @@ -1067,6 +1067,16 @@ rte_eal_init(int argc, char **argv) phys_addrs = rte_eal_using_phys_addrs() != 0; + if (!phys_addrs) { + /* if we have no access to physical addresses, pick IOVA as VA mode. */ + if (internal_conf->iova_mode == RTE_IOVA_PA) + RTE_LOG(WARNING, EAL, "WARNING: --iova-mode=pa, but Physical addresses are unavailable, selecting IOVA as VA mode.\n"); + else + RTE_LOG(DEBUG, EAL, "Physical addresses are unavailable, selecting IOVA as VA mode.\n"); + internal_conf->iova_mode = RTE_IOVA_VA; + rte_eal_get_configuration()->iova_mode = internal_conf->iova_mode; + } + /* if no EAL option "--iova-mode=", use bus IOVA scheme */ if (internal_conf->iova_mode == RTE_IOVA_DC) { /* autodetect the IOVA mapping mode */ @@ -1078,12 +1088,6 @@ rte_eal_init(int argc, char **argv) if (!RTE_IOVA_IN_MBUF) { iova_mode = RTE_IOVA_VA; RTE_LOG(DEBUG, EAL, "IOVA as VA mode is forced by build option.\n"); - } else if (!phys_addrs) { - /* if we have no access to physical addresses, - * pick IOVA as VA mode. - */ - iova_mode = RTE_IOVA_VA; - RTE_LOG(DEBUG, EAL, "Physical addresses are unavailable, selecting IOVA as VA mode.\n"); } else if (is_iommu_enabled()) { /* we have an IOMMU, pick IOVA as VA mode */ iova_mode = RTE_IOVA_VA;