From patchwork Tue Aug 17 13:44:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 97001 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 92F5EA0548; Tue, 17 Aug 2021 15:46:50 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8C461411FD; Tue, 17 Aug 2021 15:45:37 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2065.outbound.protection.outlook.com [40.107.236.65]) by mails.dpdk.org (Postfix) with ESMTP id 11078411FD for ; Tue, 17 Aug 2021 15:45:36 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LclanSLVL663JP128D/o9eKO9U3+ba9h6jsqz+yqkWHnTB0lUurZObOWuOa6sJo+IKTfK9cH2b28KPPyd6Y9WOaRSy3b+IqHTcDB68fFySySmOu+GFt+k9JfR10Dl8fw1UE5eOPxh3dtJLbPCjJRmH+xw5tSJC2F4Y/odhQ5MxG9n8lXjHTYiWyHollXaQxlVpXnYmcgcbUPKBwS/AwiZcBSyNbCG8HLnA25mHFfjkFK99b480uF7/8BrZhcy/0x/8zR3RznStnnINaRg+3Li6Z8l6recSX9cq3RV84IYB20aLAFxDbWjNQXgGPxBkbgG+cRxBn71QyXmBR+CPj9TQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LusB84Gbn+n0O0fsM07rjREfcejrLsEJMUgXr2I7OYg=; b=hXWxGXYCTyfvwM7XJBM8SF/QNCNFX0l+vt4Y3kVPByqsgiYdl9LQ361JhLIBdrZWFCYpoLkGhlWHjSI6gJ9+Al3y42gQ+4tvhnAq/epilPtO3ebkUmRMHjZsAJBYyr3WK2R7RSOjOb2ODXT6UW8ohTbtINHLulqfLEQ0xU0QjrG0tvWi1iXyb5RG/PKD0Tcos0hzioTM7gWrweUDF2fjFaXXMQafv9YJVW2DxtyzFbDITkvV9xVzulgV5RJjQkKlYPHs9tDbZikjx/5fiiGnBf0C0wFUlyzfsKoxWAXTJqxca/LVXImz1bxbXsMrZtxVEdaCetkZUYfyuNTvi/JvBA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.35) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LusB84Gbn+n0O0fsM07rjREfcejrLsEJMUgXr2I7OYg=; b=VxLTd6NO2pCGQu16svtKlm0n2bK8anrK2rjfQ7WiiEQm8dBL03Yp61joPjstEMu0s8bdLtjji70RN91Kw+EfdmUP6xtDP5o4RoAbzbispnjlyR7wHRfy64RC4HYHD7vvqJz6QazVD08eLF0zidPfLPcZBGFHJBDTuaO/rpoAdvpw9fh+EiYtrf6fp4FxkMCFJmk8l4KEN3XVK9PBQSPbvxRPu9kqtxIcikk4v2vd/gH5+0t/rIOA5NrMDlLH/1NWgk3sLhlDntuvjQtvhwOh8/dadCtWUIIw5FX9E/2gutUbM8mfLCPi12vnN61tiV0DgYNXvuoTMXRxsm/N6ap2fw== Received: from BN9PR03CA0254.namprd03.prod.outlook.com (2603:10b6:408:ff::19) by DM5PR12MB1706.namprd12.prod.outlook.com (2603:10b6:3:10f::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.19; Tue, 17 Aug 2021 13:45:34 +0000 Received: from BN8NAM11FT005.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ff:cafe::7f) by BN9PR03CA0254.outlook.office365.com (2603:10b6:408:ff::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.17 via Frontend Transport; Tue, 17 Aug 2021 13:45:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.35) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.35 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.35; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.35) by BN8NAM11FT005.mail.protection.outlook.com (10.13.176.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4415.16 via Frontend Transport; Tue, 17 Aug 2021 13:45:34 +0000 Received: from DRHQMAIL107.nvidia.com (10.27.9.16) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 17 Aug 2021 13:45:33 +0000 Received: from nvidia.com (172.20.187.6) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 17 Aug 2021 13:45:31 +0000 From: Michael Baum To: CC: Matan Azrad , Raslan Darawsheh , Viacheslav Ovsiienko Date: Tue, 17 Aug 2021 16:44:31 +0300 Message-ID: <20210817134441.1966618-12-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210817134441.1966618-1-michaelba@nvidia.com> References: <20210817134441.1966618-1-michaelba@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To DRHQMAIL107.nvidia.com (10.27.9.16) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 42837e17-219f-4d54-093a-08d96185499e X-MS-TrafficTypeDiagnostic: DM5PR12MB1706: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:473; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CXFPDLimDGfTbkDEuusyhSl0ka8AKn0TCKwGDijC7e3d2M4UsGJvUdlj7b21YtiG7R9YWPeIA776BiiDOF1TsRxpN1v0iVCqO/ySKX8/roDxHPa93rcuT6D6kQrBrtcJQfzbG4ZDZq2PjFH+wxNjnqEoJN+vLI4+ha+0c8c/U/FixUAbhQl9qnWeRoq2wNMEqZGKuVEPt00JeLhOzgKrRKvoEnyx3UQ/hK1vFrwTbAxMaGH4V85G07pNEyy3L8aIbnXm5Mjq99soNgTbpsk+BCYoEXIrVKcufGvopeH/vS+Jbo3QSa9PYJRjsjRynmpysTMPxG0maHX7ktfGuJl8mofl/edrvWmrGgDw9xRH7scDgFRm1Tg1Wjel5B8fuKLOENNzkevchEGi4RzcIJT2eFE3D6XLZbcA6GUvMNppu49rgbxLkrbOmctNMrttp7wejoPLy9a3TEtUZ5cjLq8NY4FeYiaATO+hz1nn0x3X0GvF+gUP3ZIoP2G5G3jhSOdXpYAv+a9l/rcJ5fsdLL612iUtYEM4qmU5QmMUzdrXNM7q5VBADJrwFSgUoi3929v6UpYvv9AF4uRUtayoDm7rJdEvuAzOVEMVd//v9/reCbgJOQ30rBwehaJniUlueODQVRsbNtfwDJOrC44po6BZwVzze5supGw/qhVHovpT7B9IEQQCTdbkqJeNVBWebwBhdK/61u0Yv7KU+C7MO6XPdw== X-Forefront-Antispam-Report: CIP:216.228.112.35; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid02.nvidia.com; CAT:NONE; SFS:(4636009)(39860400002)(136003)(396003)(346002)(376002)(36840700001)(46966006)(86362001)(26005)(7696005)(186003)(16526019)(2616005)(8676002)(8936002)(478600001)(36860700001)(82740400003)(47076005)(316002)(82310400003)(36756003)(356005)(6286002)(1076003)(7636003)(83380400001)(54906003)(107886003)(70206006)(336012)(4326008)(5660300002)(426003)(70586007)(2906002)(6916009)(55016002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Aug 2021 13:45:34.1615 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 42837e17-219f-4d54-093a-08d96185499e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.35]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT005.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1706 Subject: [dpdk-dev] [RFC 11/21] net/mlx5: move NUMA node field to context device X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Remove numa node field from sh structure, and use instead in context device structure. Signed-off-by: Michael Baum --- drivers/net/mlx5/mlx5.c | 3 +-- drivers/net/mlx5/mlx5.h | 1 - drivers/net/mlx5/mlx5_devx.c | 11 ++++++----- drivers/net/mlx5/mlx5_txpp.c | 10 +++++----- 4 files changed, 12 insertions(+), 13 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index f5f325d35a..b695f2f6d3 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -1142,7 +1142,6 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn, goto exit; } sh->devx = config->devx; - sh->numa_node = dev_ctx->numa_node; if (spawn->bond_info) sh->bond = *spawn->bond_info; pthread_mutex_init(&sh->txpp.mutex, NULL); @@ -1207,7 +1206,7 @@ mlx5_alloc_shared_dev_ctx(const struct mlx5_dev_spawn_data *spawn, */ err = mlx5_mr_btree_init(&sh->share_cache.cache, MLX5_MR_BTREE_CACHE_N * 2, - sh->numa_node); + dev_ctx->numa_node); if (err) { err = rte_errno; goto error; diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 1e52b9ac9a..f6d8e1d817 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1145,7 +1145,6 @@ struct mlx5_dev_ctx_shared { char ibdev_name[MLX5_FS_NAME_MAX]; /* SYSFS dev name. */ char ibdev_path[MLX5_FS_PATH_MAX]; /* SYSFS dev path for secondary */ struct mlx5_dev_attr device_attr; /* Device properties. */ - int numa_node; /* Numa node of backing physical device. */ LIST_ENTRY(mlx5_dev_ctx_shared) mem_event_cb; /**< Called by memory event callback. */ struct mlx5_mr_share_cache share_cache; diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index 3cafd46837..787c771167 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -366,7 +366,7 @@ mlx5_rxq_create_devx_cq_resources(struct rte_eth_dev *dev, uint16_t idx) log_cqe_n = log2above(cqe_n); /* Create CQ using DevX API. */ ret = mlx5_devx_cq_create(sh->dev_ctx->ctx, &rxq_ctrl->obj->cq_obj, - log_cqe_n, &cq_attr, sh->numa_node); + log_cqe_n, &cq_attr, sh->dev_ctx->numa_node); if (ret) return ret; cq_obj = &rxq_ctrl->obj->cq_obj; @@ -981,6 +981,7 @@ mlx5_txq_create_devx_sq_resources(struct rte_eth_dev *dev, uint16_t idx, uint16_t log_desc_n) { struct mlx5_priv *priv = dev->data->dev_private; + struct mlx5_dev_ctx *dev_ctx = priv->sh->dev_ctx; struct mlx5_txq_data *txq_data = (*priv->txqs)[idx]; struct mlx5_txq_ctrl *txq_ctrl = container_of(txq_data, struct mlx5_txq_ctrl, txq); @@ -994,15 +995,15 @@ mlx5_txq_create_devx_sq_resources(struct rte_eth_dev *dev, uint16_t idx, .tis_lst_sz = 1, .tis_num = priv->sh->tis->id, .wq_attr = (struct mlx5_devx_wq_attr){ - .pd = priv->sh->dev_ctx->pdn, + .pd = dev_ctx->pdn, .uar_page = mlx5_os_get_devx_uar_page_id(priv->sh->tx_uar), }, .ts_format = mlx5_ts_format_conv(priv->sh->sq_ts_format), }; /* Create Send Queue object with DevX. */ - return mlx5_devx_sq_create(priv->sh->dev_ctx->ctx, &txq_obj->sq_obj, - log_desc_n, &sq_attr, priv->sh->numa_node); + return mlx5_devx_sq_create(dev_ctx->ctx, &txq_obj->sq_obj, log_desc_n, + &sq_attr, dev_ctx->numa_node); } #endif @@ -1059,7 +1060,7 @@ mlx5_txq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx) } /* Create completion queue object with DevX. */ ret = mlx5_devx_cq_create(sh->dev_ctx->ctx, &txq_obj->cq_obj, - log_desc_n, &cq_attr, sh->numa_node); + log_desc_n, &cq_attr, sh->dev_ctx->numa_node); if (ret) { DRV_LOG(ERR, "Port %u Tx queue %u CQ creation failure.", dev->data->port_id, idx); diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c index ff1c3d204c..b49a47bd77 100644 --- a/drivers/net/mlx5/mlx5_txpp.c +++ b/drivers/net/mlx5/mlx5_txpp.c @@ -247,7 +247,7 @@ mlx5_txpp_create_rearm_queue(struct mlx5_dev_ctx_shared *sh) /* Create completion queue object for Rearm Queue. */ ret = mlx5_devx_cq_create(sh->dev_ctx->ctx, &wq->cq_obj, log2above(MLX5_TXPP_REARM_CQ_SIZE), &cq_attr, - sh->numa_node); + sh->dev_ctx->numa_node); if (ret) { DRV_LOG(ERR, "Failed to create CQ for Rearm Queue."); return ret; @@ -261,7 +261,7 @@ mlx5_txpp_create_rearm_queue(struct mlx5_dev_ctx_shared *sh) /* There should be no WQE leftovers in the cyclic queue. */ ret = mlx5_devx_sq_create(sh->dev_ctx->ctx, &wq->sq_obj, log2above(MLX5_TXPP_REARM_SQ_SIZE), &sq_attr, - sh->numa_node); + sh->dev_ctx->numa_node); if (ret) { rte_errno = errno; DRV_LOG(ERR, "Failed to create SQ for Rearm Queue."); @@ -401,7 +401,7 @@ mlx5_txpp_create_clock_queue(struct mlx5_dev_ctx_shared *sh) sh->txpp.tsa = mlx5_malloc(MLX5_MEM_RTE | MLX5_MEM_ZERO, MLX5_TXPP_REARM_SQ_SIZE * sizeof(struct mlx5_txpp_ts), - 0, sh->numa_node); + 0, sh->dev_ctx->numa_node); if (!sh->txpp.tsa) { DRV_LOG(ERR, "Failed to allocate memory for CQ stats."); return -ENOMEM; @@ -411,7 +411,7 @@ mlx5_txpp_create_clock_queue(struct mlx5_dev_ctx_shared *sh) /* Create completion queue object for Clock Queue. */ ret = mlx5_devx_cq_create(sh->dev_ctx->ctx, &wq->cq_obj, log2above(MLX5_TXPP_CLKQ_SIZE), &cq_attr, - sh->numa_node); + sh->dev_ctx->numa_node); if (ret) { DRV_LOG(ERR, "Failed to create CQ for Clock Queue."); goto error; @@ -448,7 +448,7 @@ mlx5_txpp_create_clock_queue(struct mlx5_dev_ctx_shared *sh) sq_attr.ts_format = mlx5_ts_format_conv(sh->sq_ts_format); ret = mlx5_devx_sq_create(sh->dev_ctx->ctx, &wq->sq_obj, log2above(wq->sq_size), - &sq_attr, sh->numa_node); + &sq_attr, sh->dev_ctx->numa_node); if (ret) { rte_errno = errno; DRV_LOG(ERR, "Failed to create SQ for Clock Queue.");