From patchwork Tue Oct 26 08:48:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rongwei Liu X-Patchwork-Id: 102879 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D3B1EA0C47; Tue, 26 Oct 2021 10:48:53 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1B86B426DC; Tue, 26 Oct 2021 10:48:52 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2051.outbound.protection.outlook.com [40.107.94.51]) by mails.dpdk.org (Postfix) with ESMTP id A0D16426D5 for ; Tue, 26 Oct 2021 10:48:50 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YaRjvShRouM0of8SbLRStxmxraTHhMRLTH3JQlpLdeINCEsyQ4MkYIKscLizt2rY2E/SfodnOv8Azwyl1YXyyHm15Sr7nuLNietSA7QIaeDE7GfFMDwoiCpNn1oToQRffM1ffiF4e4Qrn9kh+vvGK5rUMuGOG7u0mmhH631J1g2EoCqsX7oV18+SxP4ABRO0uRKSZrwtYCfBYYkb2G5frOHwi+zVs5/deN6UtXRT2Fx6lwn4o+jshvNGBbQhh7e32RCB+REaEDZU5x4flhlOXokBLz6d3d1QzzZ+dkobtN+ecKCAo9yMUzsNWlBCTa0Aiz7IOM03CGerMPRrqq84rQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Fi/NWbBf8hYYrL56EPDGouPpO1xLXl4GQxC4cAH8Mbk=; b=OIcXrtx7H7lDnv3d7o8t/+pzR7E5Z+sLlKuB7Yr/q1MSJMk1k3OdAu65LTskEqnd/gmMZsGjOsok4XAcDFO5ix/+Loq2b/4K8Me4UqQW0vNXekCK+n88IY4Vnqsva25qqaW4L3mx2V4+0N3Wff7toz3khl52v1e1F69JjDqKGflRHop2vpCo3JWRwmo3jwnrJ9qeUMJenXGr/HcPzOX4ciloayT67sQUmYNnps1qr6jXjKxI57tTdXLmeyqv+GDdn9mto4ErmyZCSmYqoAqo8uTeB1c/t8kMYGpTqYPs/XmBBYCTGQDNRjiX7tP7OqWbgaR8vNqJUkOK9piS6grvNA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Fi/NWbBf8hYYrL56EPDGouPpO1xLXl4GQxC4cAH8Mbk=; b=qJ14c4YKn2k9Z4cmB+QB74jx+Izq4Et5sQ039dTRmdQ4tYX74zHIUHqv83n1mP6Gbq7n8jAxmgwY5JdrRl/GBpccvjMIWJyLuutJJ5teYfsHc3ez3UYi6H1T1SCkiblAeDhh9vtBi7A0q1q5RNU2V+SPObHWmvVTV+y315tgRo48H8TwiAjDLv09PBADzfZ92Ja2JDim0LYbgEU6TIXCZkuXHs+jZLPyIvP5jfY7v2v0kXAr+ciazr4HjQ7HoG95bz0+PU/h7UaFkr2SCib5BdcH2Z25hDgvB6zOjI/VmB0wMQhP16tmLsoo56hCYVE3MySnaJBoSrUVJ9S/GAfJFQ== Received: from DM6PR18CA0011.namprd18.prod.outlook.com (2603:10b6:5:15b::24) by DM5PR12MB2406.namprd12.prod.outlook.com (2603:10b6:4:b7::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4628.16; Tue, 26 Oct 2021 08:48:48 +0000 Received: from DM6NAM11FT050.eop-nam11.prod.protection.outlook.com (2603:10b6:5:15b:cafe::b4) by DM6PR18CA0011.outlook.office365.com (2603:10b6:5:15b::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4649.13 via Frontend Transport; Tue, 26 Oct 2021 08:48:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT050.mail.protection.outlook.com (10.13.173.111) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4628.16 via Frontend Transport; Tue, 26 Oct 2021 08:48:48 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 26 Oct 2021 08:48:45 +0000 From: Rongwei Liu To: , , , CC: , Date: Tue, 26 Oct 2021 11:48:29 +0300 Message-ID: <20211026084830.440951-2-rongweil@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211026084830.440951-1-rongweil@nvidia.com> References: <20211026084830.440951-1-rongweil@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4dc542a9-ea7c-4f56-eece-08d9985d6d64 X-MS-TrafficTypeDiagnostic: DM5PR12MB2406: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:595; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4E7237/Yp9JKg6kPr/Hw5js+j+ANTnKkuUvKKEgZlQRlvTUrIWsaN1G76uQrVy92mkkjGjn25Rd4gEn6Nxp9kTFCOwBBGfBW71KdSi3BGXJogxCXG/ltKYivORaoGOVnje/AEI0OFUuL9tXwfC/HHWgPNjpXAJjIAzeUsBFwv4V92iCx+TLK5dRrJclLdeT2PkkCFX+CSpF7fDp9nBdkTNvuNBGpoa7bYO4gL/1Ntsy/Kmd1D4s7AjpKC+H+lEeByIAhpChVQxaCYG+3pFg6q82nZGS8ibidywF8LMfO2el9eXmI25O6630a5waEZ1BlkE5Sq/lBAwv2jwy5TgRsVSeb77JHdHHPKCBR9czprxMzhAMCkuyTdoGaGNR4xe5FU3WTCY93P4U5TJVGfmoSyG6x8S3lIgE9cbuJjLDXTEVtTN8pYV0bTJYIs1Cw4cAfu5iMy55xtmyn/sCDkFqfuu3MUy3Nf6ObMsgesWnw0IJX3Hp2weGe0XjMSgQUUrqEQXff8O8nxdmhfDGAFU1VuXuNdFWfipFN9gtWYbbRvpAi1OCjSzgBcW/OlhTuM9Rr7t1KE0rvumMbYDmwr/At9RoYb1bIvYswgitdcMd1qKmtxlxoZ3IUBG0WOJqaK4g+HbKoYSZNpqiOmAzch7kPf2jrrI/oVw/YkPCpWHdaFGgH0tFSr1C+g0Dx3VHDW97p2khuGrn+CNomHdN367O5MA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(36860700001)(4326008)(8676002)(356005)(70586007)(47076005)(26005)(7696005)(5660300002)(186003)(16526019)(86362001)(82310400003)(70206006)(6666004)(54906003)(83380400001)(107886003)(55016002)(36756003)(2906002)(8936002)(336012)(426003)(110136005)(316002)(2616005)(7636003)(508600001)(6286002)(1076003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Oct 2021 08:48:48.3118 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4dc542a9-ea7c-4f56-eece-08d9985d6d64 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT050.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB2406 Subject: [dpdk-dev] [PATCH v5 1/2] common/mlx5: support pcie device guid query X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" sysfs entry "phys_switch_id" holds each PCIe device' guid. The devices which reside in the same physical NIC should have the same guid. Signed-off-by: Rongwei Liu Acked-by: Viacheslav Ovsiienko --- drivers/common/mlx5/linux/mlx5_common_os.c | 40 ++++++++++++++++++++++ drivers/common/mlx5/linux/mlx5_common_os.h | 19 ++++++++++ 2 files changed, 59 insertions(+) diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index 8db3fe790a..b516564b79 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -2,6 +2,7 @@ * Copyright 2020 Mellanox Technologies, Ltd */ +#include #include #include #include @@ -704,3 +705,42 @@ mlx5_os_open_device(struct mlx5_common_device *cdev, uint32_t classes) DRV_LOG(ERR, "Failed to open IB device \"%s\".", ibv->name); return -rte_errno; } +int +mlx5_get_device_guid(const struct rte_pci_addr *dev, uint8_t *guid, size_t len) +{ + char tmp[512]; + char cur_ifname[IF_NAMESIZE + 1]; + FILE *id_file; + DIR *dir; + struct dirent *ptr; + int ret; + + if (guid == NULL || len < sizeof(u_int64_t) + 1) + return -1; + memset(guid, 0, len); + snprintf(tmp, sizeof(tmp), "/sys/bus/pci/devices/%04x:%02x:%02x.%x/net", + dev->domain, dev->bus, dev->devid, dev->function); + dir = opendir(tmp); + if (dir == NULL) + return -1; + /* Traverse to identify PF interface */ + do { + ptr = readdir(dir); + if (ptr == NULL || ptr->d_type != DT_DIR) { + closedir(dir); + return -1; + } + } while (strchr(ptr->d_name, '.') || strchr(ptr->d_name, '_') || + strchr(ptr->d_name, 'v')); + snprintf(cur_ifname, sizeof(cur_ifname), "%s", ptr->d_name); + closedir(dir); + snprintf(tmp + strlen(tmp), sizeof(tmp) - strlen(tmp), + "/%s/phys_switch_id", cur_ifname); + /* Older OFED like 5.3 doesn't support read */ + id_file = fopen(tmp, "r"); + if (!id_file) + return 0; + ret = fscanf(id_file, "%16s", guid); + fclose(id_file); + return ret; +} diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h b/drivers/common/mlx5/linux/mlx5_common_os.h index c2957f91ec..83066e752d 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.h +++ b/drivers/common/mlx5/linux/mlx5_common_os.h @@ -284,4 +284,23 @@ mlx5_os_free(void *addr) void mlx5_set_context_attr(struct rte_device *dev, struct ibv_context *ctx); +/** + * This is used to query system_image_guid as describing in PRM. + * + * @param dev[in] + * Pointer to a device instance as PCIe id. + * @param guid[out] + * Pointer to the buffer to hold device guid. + * Guid is uint64_t and corresponding to 17 bytes string. + * @param len[in] + * Guid buffer length, 17 bytes at least. + * + * @return + * -1 if internal failure. + * 0 if OFED doesn't support. + * >0 if success. + */ +int +mlx5_get_device_guid(const struct rte_pci_addr *dev, uint8_t *guid, size_t len); + #endif /* RTE_PMD_MLX5_COMMON_OS_H_ */ From patchwork Tue Oct 26 08:48:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rongwei Liu X-Patchwork-Id: 102880 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6579AA0C47; Tue, 26 Oct 2021 10:48:59 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 18B50426E2; Tue, 26 Oct 2021 10:48:53 +0200 (CEST) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2069.outbound.protection.outlook.com [40.107.102.69]) by mails.dpdk.org (Postfix) with ESMTP id 5520E426D8 for ; Tue, 26 Oct 2021 10:48:51 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bKG0qSBLiB5KzYisZoKgsJiORKMndNInGf6N8uN4iZuikOVSFsnHZ1Soee7L97ZtQqKqWoVPjHIMjkE+B2AXwA8ZsGmApmUXM42SZHxXlNkZ1i3ObEoxZVyRY03UtsTDBzHBzWLwGVAnWZjWUSfHLMrOHLBwz5cxfhXNSVYRO/tfV/o28V7GRiNW4AeSiX3ZEA8LX1C9u7Kyz88ASQsmFMj2pcXpHCzoGatsRF032v1l+ipi3wUuNRHNWPLeQmKfduUZD/SIW1oWpjJQBuDur9kXCoJ5Ka674yXo7Nd5Rp6JxGWeqZNsGveMfI1HamCca+sZ2i+Q8HT0IrYWZuAeYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=afTLafmd1l4sskYwJfcMVoaLFpCmt0wEPxsnwtIF3a8=; b=gzC5sR1Rc8L2jYZ6hY+HLnk63uYKXwotbKOmfiamxd3FP7FzAkzUK2ZXH5XSL8IafohhlAYOB5mDLvi+h7aRBCBoSH5CX1nW7EEwnyhPn+7YVFu+Ft1bOk1txvHmWsNSucqilXEaB1PsHuGBWQqhILbsC6XDDDm8TH6bp7TLmTl2XkgyonrGsVySAVJpd30iJC0vhJ+7DNdIHoSWSXlc2vETMNf5bLmFDIjnglm0f8ohcH3/0NEl0rkWRycuutUz0X+8ALoKgv9FQL55cpTblcgUuOj2rQKQP6zQTd9ULEQqILXIm7JpUJ+J4leBhthTnbIwasvLiwa8eEJUDmzMlA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=afTLafmd1l4sskYwJfcMVoaLFpCmt0wEPxsnwtIF3a8=; b=CNMVk64T4BkjW1f2S06uwpqkxhICx3frIHs+1VuRTzImW2j/taMDlCOqNUl7FUJj5MYouKQv3mDE+aLPlBRlPoo+p+l86w7ynNaI7FOTXvfmnoNyvjc2/W1demt1eoMX4I+pbQT5+E5Haty9CnT/cbzrHezYhaAuHboUl9uiEtjBp7tv0i8E4yKNKjErVJkqQWMbTmajSv68ow700uB6MnfwHraonfgRHsBfpcD5S3A6CDKe91yeGSA416dFKKprx9XGhNgo9CDd7YS3rgbcGJ5yteEswIo7a2KQi2GEeRNbiqXB1/XZetKCrbjzWvg3pwaBqWsf3mv5FwNhVQhsiQ== Received: from DM6PR18CA0033.namprd18.prod.outlook.com (2603:10b6:5:15b::46) by BY5PR12MB5511.namprd12.prod.outlook.com (2603:10b6:a03:1d8::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4628.18; Tue, 26 Oct 2021 08:48:49 +0000 Received: from DM6NAM11FT050.eop-nam11.prod.protection.outlook.com (2603:10b6:5:15b:cafe::20) by DM6PR18CA0033.outlook.office365.com (2603:10b6:5:15b::46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4628.18 via Frontend Transport; Tue, 26 Oct 2021 08:48:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; dpdk.org; dkim=none (message not signed) header.d=none;dpdk.org; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT050.mail.protection.outlook.com (10.13.173.111) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4628.16 via Frontend Transport; Tue, 26 Oct 2021 08:48:49 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Tue, 26 Oct 2021 08:48:47 +0000 From: Rongwei Liu To: , , , CC: , Date: Tue, 26 Oct 2021 11:48:30 +0300 Message-ID: <20211026084830.440951-3-rongweil@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20211026084830.440951-1-rongweil@nvidia.com> References: <20211026084830.440951-1-rongweil@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL105.nvidia.com (172.20.187.12) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9cc179e1-a51b-44c3-ad7f-08d9985d6e1e X-MS-TrafficTypeDiagnostic: BY5PR12MB5511: X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1265; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hyr3xdeFgIg1dZWRfHkaFr+FwC0P7ffuEnxHXpVhiJwNVbfJ6prSWqohsxfg2XuCepf0zgJY0fffOEOklLynfJZ0CppRRoholJvVvSEA5m1QXlSNn4atE9JaCXxPD1PIGD62grS5EMxkTakJBk7pV/n35cJci5uZO4FS/atErEO6pl1xnF3IjSt5aN/ZdYs5vnca78RWhkx8WE4u/AnX6SepeX6VDv+XldxxcNAAMaOGF25ghUKJTZKtZgCDcVx4eFsZdbNVqkzbCnS/6M62NOiiapctf4JcvRNN+6Dy8w0GjU8GKAviVSkX0EH/wo7VRpJ7F09APSKNcZDWT99QOBrUw7xvLfFHkdmBhMN/TqtRTnRkg/8AE1oTQWbOZj2HYMpLI1mS+CYPoHnPJxI5Y6dLJa0GIM6ShuRfb/51Bb8OiPaRViS0Zszf9rolintmmcXNxh8TN6eqDMd2XTZE6/f0RZul5j86pxS/KshP7is3Lu3LdrelAdd3SK80+mRrqlJ27vgUY939x5JdWa+5j/uDrisfMI0Tf7NNzPMmIOrEpwQo/qcruIXCwrac/ZpuwvBe26v2ACMDlsZQDCfSEZHV571nV8AncuIPoAQEf8+FWKvhLCwzHrzqvNf5HjuWoYeLO5AVQ1GvjqM0aKe/9xZCTIS2lZKJv+ZxWXIde+MXTvBlWFEZNfC4mVU+tklSvP/tGTEv7o0D5YE256eHrA== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(1076003)(83380400001)(70586007)(55016002)(4326008)(186003)(26005)(36860700001)(16526019)(2616005)(8936002)(70206006)(86362001)(426003)(356005)(2906002)(6666004)(47076005)(107886003)(336012)(316002)(508600001)(6286002)(36906005)(36756003)(7636003)(7696005)(5660300002)(8676002)(82310400003)(110136005)(54906003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Oct 2021 08:48:49.4851 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9cc179e1-a51b-44c3-ad7f-08d9985d6e1e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT050.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB5511 Subject: [dpdk-dev] [PATCH v5 2/2] net/mlx5: support socket direct mode bonding X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In socket direct mode, it's possible to bind any two (maybe four in future) PCIe devices with IDs like xxxx:xx:xx.x and yyyy:yy:yy.y. Bonding member interfaces are unnecessary to have the same PCIe domain/bus/device ID anymore, Kernel driver uses "system_image_guid" to identify if devices can be bound together or not. Sysfs "phys_switch_id" is used to get "system_image_guid" of each network interface. OFED 5.4+ is required to support "phys_switch_id". Signed-off-by: Rongwei Liu Acked-by: Viacheslav Ovsiienko --- doc/guides/nics/mlx5.rst | 4 +++ doc/guides/rel_notes/release_21_11.rst | 4 +++ drivers/net/mlx5/linux/mlx5_os.c | 43 ++++++++++++++++++++------ 3 files changed, 42 insertions(+), 9 deletions(-) diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst index 47709d93b3..45f44c97d7 100644 --- a/doc/guides/nics/mlx5.rst +++ b/doc/guides/nics/mlx5.rst @@ -468,6 +468,10 @@ Limitations - TXQ affinity subjects to HW hash once enabled. +- Bonding under socket direct mode + + - Needs OFED 5.4+. + Statistics ---------- diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index 1ccac87b73..2f46b27709 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -217,6 +217,10 @@ New Features * Added PDCP short MAC-I support. * Added raw vector datapath API support. +* **Updated Mellanox mlx5 driver.** + + * Added socket direct mode bonding support. + * **Updated NXP dpaa2_sec crypto PMD.** * Added PDCP short MAC-I support. diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/mlx5_os.c index 72bbb665cf..3deae861d5 100644 --- a/drivers/net/mlx5/linux/mlx5_os.c +++ b/drivers/net/mlx5/linux/mlx5_os.c @@ -1898,6 +1898,8 @@ mlx5_device_bond_pci_match(const char *ibdev_name, FILE *bond_file = NULL, *file; int pf = -1; int ret; + uint8_t cur_guid[32] = {0}; + uint8_t guid[32] = {0}; /* * Try to get master device name. If something goes wrong suppose @@ -1911,6 +1913,8 @@ mlx5_device_bond_pci_match(const char *ibdev_name, np = mlx5_nl_portnum(nl_rdma, ibdev_name); if (!np) return -1; + if (mlx5_get_device_guid(pci_dev, cur_guid, sizeof(cur_guid)) < 0) + return -1; /* * The master device might not be on the predefined port(not on port * index 1, it is not guaranteed), we have to scan all Infiniband @@ -1938,6 +1942,7 @@ mlx5_device_bond_pci_match(const char *ibdev_name, char tmp_str[IF_NAMESIZE + 32]; struct rte_pci_addr pci_addr; struct mlx5_switch_info info; + int ret; /* Process slave interface names in the loop. */ snprintf(tmp_str, sizeof(tmp_str), @@ -1969,15 +1974,6 @@ mlx5_device_bond_pci_match(const char *ibdev_name, tmp_str); break; } - /* Match PCI address, allows BDF0+pfx or BDFx+pfx. */ - if (pci_dev->domain == pci_addr.domain && - pci_dev->bus == pci_addr.bus && - pci_dev->devid == pci_addr.devid && - ((pci_dev->function == 0 && - pci_dev->function + owner == pci_addr.function) || - (pci_dev->function == owner && - pci_addr.function == owner))) - pf = info.port_name; /* Get ifindex. */ snprintf(tmp_str, sizeof(tmp_str), "/sys/class/net/%s/ifindex", ifname); @@ -1994,6 +1990,30 @@ mlx5_device_bond_pci_match(const char *ibdev_name, bond_info->ports[info.port_name].pci_addr = pci_addr; bond_info->ports[info.port_name].ifindex = ifindex; bond_info->n_port++; + /* + * Under socket direct mode, bonding will use + * system_image_guid as identification. + * After OFED 5.4, guid is readable (ret >= 0) under sysfs. + * All bonding members should have the same guid even if driver + * is using PCIe BDF. + */ + ret = mlx5_get_device_guid(&pci_addr, guid, sizeof(guid)); + if (ret < 0) + break; + else if (ret > 0) { + if (!memcmp(guid, cur_guid, sizeof(guid)) && + owner == info.port_name && + (owner != 0 || (owner == 0 && + !rte_pci_addr_cmp(pci_dev, &pci_addr)))) + pf = info.port_name; + } else if (pci_dev->domain == pci_addr.domain && + pci_dev->bus == pci_addr.bus && + pci_dev->devid == pci_addr.devid && + ((pci_dev->function == 0 && + pci_dev->function + owner == pci_addr.function) || + (pci_dev->function == owner && + pci_addr.function == owner))) + pf = info.port_name; } if (pf >= 0) { /* Get bond interface info */ @@ -2006,6 +2026,11 @@ mlx5_device_bond_pci_match(const char *ibdev_name, DRV_LOG(INFO, "PF device %u, bond device %u(%s)", ifindex, bond_info->ifindex, bond_info->ifname); } + if (owner == 0 && pf != 0) { + DRV_LOG(INFO, "PCIe instance %04x:%02x:%02x.%x isn't bonding owner", + pci_dev->domain, pci_dev->bus, pci_dev->devid, + pci_dev->function); + } return pf; }