From patchwork Fri Oct 22 09:11:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xueming Li X-Patchwork-Id: 102632 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id EDC9BA0C43; Fri, 22 Oct 2021 11:12:16 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7FFB34118E; Fri, 22 Oct 2021 11:12:16 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2051.outbound.protection.outlook.com [40.107.236.51]) by mails.dpdk.org (Postfix) with ESMTP id 06B334118E for ; Fri, 22 Oct 2021 11:12:15 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TRvCIJnYrTX+FZvuugwOAPoDtHVEkwdSD2Q8BXKPDaZnh9oHZpF4rnI0fwkfmTGpNUbyDL8hMhkH29bFSsvbcjs4qqLK9ll043bTuxbL3i8qc+nMCFJhDGJtbgdmF27mOWukZQ3wOLpIFxsTfiMwNR0ye0oHDR0JNQO3Dcb9kgMRmMlPcwGCaLJCfDrRaoJN8JEncCr2ZsDJaQuFZVjUvEyWptWOq10Pj9/TGnBADuPdMNIgH85WrtfZaM2GZAgL0yx5XIrifUhlT8kgjvpv6j4iU4g4F+V9FmcRf+OefQtmjniMHA4Ve89oik0i7qUfrxjOq4ntRBTsFLpYQ6qFJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SXcCCSoNajRa2vVa08vb69qKfOEaBGltTXpa5jFfFGI=; b=PU2+Whku2DQxEPwIFXwSdc7fcvWEapa6cYncGT3KW1EBx1EeHp6HWlshYBgnCRmryqLBcWnfM62T85XgWxApTPCgPGfaBrBe08dXVWTWb9n4h/QX8qYS2PyxcZdXhNcUeXXxYWc3ePDW1yrtX07gUiqnK64T6itGXHbH0dUXPq/I4ZNbYndzclFEvCh/8brYF7TA2/qEFiekpVBBEInUWpCaW1RSED3dIShGqJ8W4Jjgt2vnP+F8KgU4+YMBdaruTQP7K1ltOc6xTun7mdosIeF5zjH/kudWUW2hTIgxmBFBVDgn/dw7iVNOzjjo+KLIUBghJFnB2rOfnj+HYj85+Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SXcCCSoNajRa2vVa08vb69qKfOEaBGltTXpa5jFfFGI=; b=oqVqs4ayYdKFmYT50/KJri/WXX0su3jRTeXut3Z4kykMo0a1XMQ+4Dl4lfSszjIFQPKM7/gIE7Gs8ZBwB3nvjahLeV/jiTqSIatCVUSgUpKHGxMzPBU1nD4ukl8C9kOly1LX/snr2NpZXldUrwiFeBr8gwAqhZXloIprp7Y/4pV+Xar8W4XY1UxKUpldHhCZbIybm/H61K4LTkG+PkStDd9Szbot9TVtilwfX/UYFvTxZ7iDrvevoAzv+CYAE+QO8bGL16lWm3sZrBwqWGu3NwtTHwNWor+rBSk4oC9aTryHb7hIQXQGJG6ZZTcivnUJgjSOmizsPD0YKk+RH+yGkQ== Received: from MW4PR03CA0280.namprd03.prod.outlook.com (2603:10b6:303:b5::15) by DM6PR12MB3738.namprd12.prod.outlook.com (2603:10b6:5:1c7::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.18; Fri, 22 Oct 2021 09:12:13 +0000 Received: from CO1NAM11FT038.eop-nam11.prod.protection.outlook.com (2603:10b6:303:b5:cafe::37) by MW4PR03CA0280.outlook.office365.com (2603:10b6:303:b5::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4628.16 via Frontend Transport; Fri, 22 Oct 2021 09:12:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by CO1NAM11FT038.mail.protection.outlook.com (10.13.174.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4628.16 via Frontend Transport; Fri, 22 Oct 2021 09:12:12 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 22 Oct 2021 09:12:07 +0000 From: Xueming Li To: CC: , Viacheslav Ovsiienko , Lior Margalit , Matan Azrad Date: Fri, 22 Oct 2021 17:11:36 +0800 Message-ID: <20211022091142.51397-3-xuemingl@nvidia.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211022091142.51397-1-xuemingl@nvidia.com> References: <20210927083256.337450-1-xuemingl@nvidia.com> <20211022091142.51397-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a38dae8e-6089-4a8d-7858-08d9953c08b5 X-MS-TrafficTypeDiagnostic: DM6PR12MB3738: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2399; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9Nt225jIANA2A616Ws/V7YTJvIiLpLkYJ9fYEZEU5IVefgqBtzYxLFAHlvxvCp5iu4ROxC6H3ooiZDAghrgmr+2oUSgJE5KJ7S0Ua5Je6WmlLku7XW788eVG8KLo5D4lk/xyP7duH3lpCa6rBJ7b3jiO7YWq60NLM7bcVtxIN5tyrAcb2jq8HNwbk0rubO3AWnWLiRH1vZ6I2RBs8dmOap+SfIFqEivy3J0PTRunCI2gEBO3D6OG4Jex6VkVztpNKvCHmz6hNFz4hIRKmEpusuUsenvzdXmlIXFuFIiv20eosRyzsKnfB7GhywDojIkmfnOJSa72IbdHxCcz6djm827xmFRjq6h4cjfcdOjWcJbYLf4pWPKdAXAAHbeOiuu+u4JkZrwawWV1uLJ/dacOkOwWuCyep67YxlwzUWZSdaiJVx81KgJjhEOCYvpBdpn0Be0uYKag6/KcrXf0RqCzPWz2GG+YXnOtdirj2xD1sWzGZ8pz523zZr0+CGNhfBxWOr/KKfSX5M1hZOvmnCm4j6hJrHu4o1WKdn5NiB2BwUGqalBZIro19olZ8MT4QG9IAu/ogKboAyF6hSJqGYgYzxQ8vSEzOfB3oNq/68P3EJiRFz9z8dSxWhb5YArd6T5MOYApQo72JVy+r433ljwNgvyFAJzAUMbIsJIK2+R1qqVGz0Ak1k58B4xSwcxhAQ1mk60TzTzp3lYCqSdblTLTVzJB5tlYOj/P5W/nrIRtJ3M= X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(36840700001)(46966006)(55016002)(47076005)(16526019)(70586007)(54906003)(2616005)(4326008)(70206006)(1076003)(7636003)(2906002)(6916009)(6286002)(8676002)(83380400001)(426003)(508600001)(26005)(107886003)(336012)(36756003)(86362001)(6666004)(356005)(8936002)(316002)(36860700001)(186003)(82310400003)(5660300002)(7696005)(309714004); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Oct 2021 09:12:12.3784 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a38dae8e-6089-4a8d-7858-08d9953c08b5 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT038.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3738 Subject: [dpdk-dev] [PATCH v4 2/8] net/mlx5: use netlink when IB port greater than 255 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" IB spec doesn't allow 255 ports on a single HCA, port number of 256 was cast to u8 value 0 which invalid to ibv_query_port() This patch invokes Netlink api to query port state when port number greater than 255. Signed-off-by: Xueming Li Acked-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_os.c | 46 +++++++++++++++++++++----------- 1 file changed, 31 insertions(+), 15 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 54e0ba9f3a9..101ef943f42 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -905,7 +905,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, { const struct mlx5_switch_info *switch_info = &spawn->info; struct mlx5_dev_ctx_shared *sh = NULL; - struct ibv_port_attr port_attr; + struct ibv_port_attr port_attr = { .state = IBV_PORT_NOP }; struct mlx5dv_context dv_attr = { .comp_mask = 0 }; struct rte_eth_dev *eth_dev = NULL; struct mlx5_priv *priv = NULL; @@ -924,6 +924,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, int own_domain_id = 0; uint16_t port_id; struct mlx5_port_info vport_info = { .query_flags = 0 }; + int nl_rdma = -1; int i; /* Determine if this port representor is supposed to be spawned. */ @@ -1121,20 +1122,35 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, " old OFED/rdma-core version or firmware configuration"); #endif config->mpls_en = mpls_en; + nl_rdma = mlx5_nl_init(NETLINK_RDMA); /* Check port status. */ - err = mlx5_glue->query_port(sh->cdev->ctx, spawn->phys_port, - &port_attr); - if (err) { - DRV_LOG(ERR, "port query failed: %s", strerror(err)); - goto error; - } - if (port_attr.link_layer != IBV_LINK_LAYER_ETHERNET) { - DRV_LOG(ERR, "port is not configured in Ethernet mode"); - err = EINVAL; - goto error; + if (spawn->phys_port <= UINT8_MAX) { + /* Legacy Verbs api only support u8 port number. */ + err = mlx5_glue->query_port(sh->cdev->ctx, spawn->phys_port, + &port_attr); + if (err) { + DRV_LOG(ERR, "port query failed: %s", strerror(err)); + goto error; + } + if (port_attr.link_layer != IBV_LINK_LAYER_ETHERNET) { + DRV_LOG(ERR, "port is not configured in Ethernet mode"); + err = EINVAL; + goto error; + } + } else if (nl_rdma >= 0) { + /* IB doesn't allow more than 255 ports, must be Ethernet. */ + err = mlx5_nl_port_state(nl_rdma, spawn->phys_dev_name, + spawn->phys_port); + if (err < 0) { + DRV_LOG(INFO, "Failed to get netlink port state: %s", + strerror(rte_errno)); + err = -rte_errno; + goto error; + } + port_attr.state = (enum ibv_port_state)err; } if (port_attr.state != IBV_PORT_ACTIVE) - DRV_LOG(DEBUG, "port is not active: \"%s\" (%d)", + DRV_LOG(INFO, "port is not active: \"%s\" (%d)", mlx5_glue->port_state_str(port_attr.state), port_attr.state); /* Allocate private eth device data. */ @@ -1151,7 +1167,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, priv->pci_dev = spawn->pci_dev; priv->mtu = RTE_ETHER_MTU; /* Some internal functions rely on Netlink sockets, open them now. */ - priv->nl_socket_rdma = mlx5_nl_init(NETLINK_RDMA); + priv->nl_socket_rdma = nl_rdma; priv->nl_socket_route = mlx5_nl_init(NETLINK_ROUTE); priv->representor = !!switch_info->representor; priv->master = !!switch_info->master; @@ -1844,8 +1860,6 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, mlx5_os_free_shared_dr(priv); if (priv->nl_socket_route >= 0) close(priv->nl_socket_route); - if (priv->nl_socket_rdma >= 0) - close(priv->nl_socket_rdma); if (priv->vmwa_context) mlx5_vlan_vmwa_exit(priv->vmwa_context); if (eth_dev && priv->drop_queue.hrxq) @@ -1869,6 +1883,8 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, } if (sh) mlx5_free_shared_dev_ctx(sh); + if (nl_rdma >= 0) + close(nl_rdma); MLX5_ASSERT(err > 0); rte_errno = err; return NULL;