From patchwork Fri Jun 9 15:28:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 128476 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A9A2442C6F; Fri, 9 Jun 2023 17:29:31 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 63C4E43031; Fri, 9 Jun 2023 17:29:22 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2076.outbound.protection.outlook.com [40.107.223.76]) by mails.dpdk.org (Postfix) with ESMTP id 2690E4302C for ; Fri, 9 Jun 2023 17:29:19 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Gcnas3DFdKeQazTzk5EH97ZfLmr49xf49LG4+tsZi72MPiZ1H6IiEHELWj2/N1f9/HVRb/n954Tc2+84xUH8rOKJDNW76wCZIvB7KobbbxAr3r7JMgixj3CzXb6EHJsrVbGItz6Mj7Oq1JoKjV693Ja72P1BMeASLnfPxs+lmwaw9uRuOAw5d5GKMkNw+MN4EcA7H+xWrCP81buPaWuauJaBJYOlUnezr5+2330ecUyG9ZoBqtdf6z+14A9/PtO6iVbVdSzF+5JF3WOSa3W+DsJqxm9lx4ND01wL4WPKqvImGl2MDFM59hemgN1l+TxiVcnmyMyUNVyLyJB8Gx/GoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=/VTtSNpxNHnhfoGbJbqzG8tId7O/7MsL9IN1QZUjPaE=; b=NXaTAjGFHYyeM/NemE649YkJh40EeGJQNwOkyZhK28l0pgrRgpcF1fNlwiZqdOAtsp7m/KxJn9m/aa4QiZzk4HUaV4x5A16OYXLSYvaYWw1VrIJpdt237aIF7E7mMh5EIISZ/Vmb0VopHsxKY59VdjbL6G/Scsn+t2qF6jAKalUkmNuW8L7VlyvgoYy68BDaUZ/KDH+R8cxu7j85HdyOWJmkzHmwFk0abKo5Cql4tjY78EUdI6uuBpKlA/BPKq2e21TxJd1BbYdxOS+EqKiE3RCktqIXdOj9ZfXwe1JU3+JcNYhMnuSSUOKymlUypAi2Aj8gWjs/GiG+nFfUmhUB+g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/VTtSNpxNHnhfoGbJbqzG8tId7O/7MsL9IN1QZUjPaE=; b=cUylR8VLwo3kxzU2NFlfXAwp7won2DUtxWh8Et5QE5PGXbyL+J08a8TLT31Nlq2rk49R9LONR/JojJ1weAP7opTZcMsTeBkZwT6I/pSo/PsLbPIqpNnepQOb4VW/ViiwZSt2vBoYGruzeTJsDIKe2RfdTMAkTU6ucKlrzxBIdE2uFiU9wNaz7Bng6h0mSUp9quaZSjxsYkw+fn4mUUTEMJQGHReWV/Ct0ISfPRoLH/ATVrDymxDGlvSoVBgR7XKZQnruiaCLXGasH8F+qzwI7JVd938iUuR7/Hw7MURFf1RMo9MYYilGlEzWzmfYuUTo+cCnNbpKQpYSvqnqix5Bng== Received: from DS7PR06CA0023.namprd06.prod.outlook.com (2603:10b6:8:2a::14) by BY5PR12MB4872.namprd12.prod.outlook.com (2603:10b6:a03:1c4::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33; Fri, 9 Jun 2023 15:29:15 +0000 Received: from DM6NAM11FT014.eop-nam11.prod.protection.outlook.com (2603:10b6:8:2a:cafe::2c) by DS7PR06CA0023.outlook.office365.com (2603:10b6:8:2a::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.26 via Frontend Transport; Fri, 9 Jun 2023 15:29:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DM6NAM11FT014.mail.protection.outlook.com (10.13.173.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.29 via Frontend Transport; Fri, 9 Jun 2023 15:29:14 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Fri, 9 Jun 2023 08:29:06 -0700 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Fri, 9 Jun 2023 08:29:05 -0700 From: Viacheslav Ovsiienko To: Subject: [PATCH 1/5] app/testpmd: add trace dump command Date: Fri, 9 Jun 2023 18:28:43 +0300 Message-ID: <20230609152847.32496-2-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20230609152847.32496-1-viacheslavo@nvidia.com> References: <20230420100803.494-1-viacheslavo@nvidia.com> <20230609152847.32496-1-viacheslavo@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT014:EE_|BY5PR12MB4872:EE_ X-MS-Office365-Filtering-Correlation-Id: 6c11b210-2f33-4dfa-df69-08db68fe486b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xgC7vfXww09OMirxq6+vZxY2a2s8zR40X/Whs0QkdxPYAzK5kfttpWASto9SfxlAZgy8DGQyrlmki1tE959tiJSG5o9xwEnQSE5Y0R5AhPtWLhsTGxGfqjwvpW4m7xJU9BEL7+U2Tn4tBV5bkiVsnUxiARhOI99KGPFePBQrN8utcoBiLT80j+Chxdez3cH9cyPbKVmSPzHtjINsLSGKDIjzprF3nqJZnW/uQI5O90dbfrcuJr63WhVWPajmmCkuTuTfcnq8elsdXqiCNhU6sgNkEwQxVntrhwzlW5NR8opohd2mShCVYMNQrZgm1GOSdBUxEMaaV4N1dbkggT/wozODSp6jQIDJB377820hOgTgjCH+wuCIjKfGi0QcFH4J64w1Tk8cHgaoGRdN+A62my4lbRqrsD5aOuN9Q62zH/QeKINNAYGFzW7tAZ20Euspj/6o0xKJ6S5vTXG2He+FzaK7Wqqa2x7PYrTbaJD5ZQwJFDrkEEMV9RhuMR6aXl5t5+uyW8N4FhzLw7uCT4xcP9bzmSWl+uCUpwsEgJJdiGj5y/QwXViMe6hFbt0V2sc7nX+HsaDJSzcFFjp44uWVjYRvUKCjiagR0pMKTGoiOntp+ORLWYijXvt1tfuypPaY4KvKrSM1OKLBrubDBav6cuIXwzYFeinbyiaQAJz6aqriabngYYahL/Wynz+w9mswKyJ0WbvAymHLr3vqjoBuOJQ+IZizzaViqeEJbjkjK/8mk3+hXZgiNfmrUGwW3ng7 X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(346002)(376002)(39860400002)(136003)(396003)(451199021)(40470700004)(46966006)(36840700001)(478600001)(7696005)(6666004)(82310400005)(36860700001)(2616005)(47076005)(336012)(426003)(86362001)(36756003)(83380400001)(7636003)(55016003)(356005)(40480700001)(186003)(82740400003)(16526019)(1076003)(40460700003)(26005)(6286002)(8936002)(8676002)(5660300002)(41300700001)(2906002)(6916009)(316002)(70206006)(70586007); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2023 15:29:14.7505 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6c11b210-2f33-4dfa-df69-08db68fe486b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT014.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4872 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The "dump_trace" CLI command is added to trigger saving the trace dumps to the trace directory. Signed-off-by: Viacheslav Ovsiienko --- app/test-pmd/cmdline.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 7b20bef4e9..be9e3a9ed6 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -39,6 +39,7 @@ #include #endif #include +#include #include #include @@ -8367,6 +8368,8 @@ static void cmd_dump_parsed(void *parsed_result, rte_lcore_dump(stdout); else if (!strcmp(res->dump, "dump_log_types")) rte_log_dump(stdout); + else if (!strcmp(res->dump, "dump_trace")) + rte_trace_save(); } static cmdline_parse_token_string_t cmd_dump_dump = @@ -8379,7 +8382,8 @@ static cmdline_parse_token_string_t cmd_dump_dump = "dump_mempool#" "dump_devargs#" "dump_lcores#" - "dump_log_types"); + "dump_log_types#" + "dump_trace"); static cmdline_parse_inst_t cmd_dump = { .f = cmd_dump_parsed, /* function to call */ From patchwork Fri Jun 9 15:28:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 128475 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C7BFD42C6F; Fri, 9 Jun 2023 17:29:24 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5B7184302E; Fri, 9 Jun 2023 17:29:20 +0200 (CEST) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by mails.dpdk.org (Postfix) with ESMTP id BFB8242D8F for ; Fri, 9 Jun 2023 17:29:17 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=THL5Vbuxt0N7To7yjdbDPOHRn19yX6pjCbjcYZye0+KuMlKGXJfCIUiN/QnGDgyxxwmVsWoOjWWYOO4WYPIHbSsJdZ2MRMM/t3iGcfLOcuj59qkG1zbmsJYSRz6oFdbllnzAgUupS6JfKsQVAfTgQ+NmQo7bM91H5jhF70NuN8CWbT+Go31OzNPVSx88I7E89T8wsQihfFyTCRzrpT6PHXTLB0Ol1JhQYvyjZVzPAK8sEqhxkqZ5JGjJFi9s4VjzttdePF+DJREEjgOeU09jNEfOEODKKukACih2MzppICfSbpUNvb4dtIMZKp3XfZ+vPqVzKD41+dLP2aGqaChmIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KkiRht1cQyEH0p3tHocnTu0n32/D32DfJu4k0Rs4m+s=; b=U3iM2SC62d4EPekZGIsYjqwsOvH5+fZjVvwPe064EEUs1Vk+ielLqBHaOykfRIgXeiudk1V/FBNNW3frbwXgbguWic11iDQFhn8UOfuKPsDEc7VRYKwW7sjva6S0X5YgFH2xAVo//Haz9IHJGgT4rxUa0T+1kvVPCQppUsjoICyEJC6zKiXCblQ9q0jJWKX/BBW/Hj1U+5ixiRVlexSvj4Byn8ZVVOEcHqZpCTX/sMEGrBg6kK4QvjL1xvWEgXKFOR7bAUDqV4SHdf7sW8HET7mtaN5syNEDRL6/kW6j6amlzp0+GbBNSvj4GiXZICq7T2oiOVVeKtu4Biwh02Vq4w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KkiRht1cQyEH0p3tHocnTu0n32/D32DfJu4k0Rs4m+s=; b=PX/YDmZfXFpLzVyIVcP5M3LhpuPkRnP0wObJ2BJ9gHa3dKt+SSCAPbOK6/sWi2a9x8N8yFORsYVIIIBoPVRgU8sXkYMLL1tWSvmyr/WytPQ59mt6YD7GPqkAcbkrhoyQJsKAfKvHczRx6gta9RZfMYcAOWQgD+4gQroaqGfg/iyifHaEZEzVxBfSx6sQU8pTYuzJkRpMBZWodwNX0nK2nynb4n1CE+vy4GemGpLIDv+O1uAIIK2AMfluJ58BpilTlz+gX5OE0cborwlAsqtpfVgR6y8Ob+4IeGRXD1BUkQDf9dZxvuzFlGu1C7Ro2S99OnqNMxPl4CtPI5RWYHi3cg== Received: from DS7PR06CA0023.namprd06.prod.outlook.com (2603:10b6:8:2a::14) by PH7PR12MB5618.namprd12.prod.outlook.com (2603:10b6:510:134::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33; Fri, 9 Jun 2023 15:29:16 +0000 Received: from DM6NAM11FT014.eop-nam11.prod.protection.outlook.com (2603:10b6:8:2a:cafe::2c) by DS7PR06CA0023.outlook.office365.com (2603:10b6:8:2a::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.26 via Frontend Transport; Fri, 9 Jun 2023 15:29:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DM6NAM11FT014.mail.protection.outlook.com (10.13.173.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.29 via Frontend Transport; Fri, 9 Jun 2023 15:29:15 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Fri, 9 Jun 2023 08:29:07 -0700 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Fri, 9 Jun 2023 08:29:06 -0700 From: Viacheslav Ovsiienko To: Subject: [PATCH 2/5] common/mlx5: introduce tracepoints for mlx5 drivers Date: Fri, 9 Jun 2023 18:28:44 +0300 Message-ID: <20230609152847.32496-3-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20230609152847.32496-1-viacheslavo@nvidia.com> References: <20230420100803.494-1-viacheslavo@nvidia.com> <20230609152847.32496-1-viacheslavo@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT014:EE_|PH7PR12MB5618:EE_ X-MS-Office365-Filtering-Correlation-Id: 1b51e0a2-568b-4ff3-380f-08db68fe4909 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sz9/raCDj9vlEwyswjb/YTsn0aMmV/jyDfBENUpTsFGIzA5E9WYQWNNk4gEn73xUcppe7CYk4ctM8bi+Dzf0FfUo0IU4w1qpTxNZTIlZ1/2Gf8zKZ2ygL4aPsySH8xUGch64Ss5idHY2gLMkJEpli2Iu3z8yzaW+SiUSnLK2Dst/5YCorgeaaaCDu52U+XZnj6ZcsdineDQBMMvnC8PDTuAh0EWEpGPVXBl68AGnl2zoxuTO85grFDkMXT+4+5y53eYkIy+3lXL4JnF52VSXK8PCQWYPEp1aeFUyyVPv7w9SyFoLTiARhEJlin+PsaK8+V1ocWxAwNBgQRIuqr5XU+qEhcBlYTl2bJqkJ54/7WObCqC9XfgFWL00CFSbc/HRCKHedV0qcseSbhimK4RkLiGBuSf+WmibQUWmdZXDIY/lEkAg3+JJG+ijbit3pGsG03FPGPD5hK3zgzq0eRg+LCWlO7EpmM0PqSar1b1t8p4Kjh2kkFKSwq7AJgKh2AUyGL7jTNEbiAEhe7a6CTjEiF84TK45JR5nvvZLuGfWR8RUstp86ytVn/Ixk3FoLsUgxnSy0RROVaT4H+ftVbG2SYdb60TxnebenKSYDuyQZkfICFgWAfKoDxnQBAyFMyMW3VcOwBdPjcUxvottqXxjnbQwn0iSTUOtw1byo0bpmSS7nDCJ6ArWtKNdCNxUTcuTG9mcYiimyEKcHnhXw+ksJSNb4g+5VEHsJehVDOFpiuk= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(346002)(396003)(39860400002)(376002)(136003)(451199021)(40470700004)(46966006)(36840700001)(47076005)(336012)(426003)(40480700001)(478600001)(82740400003)(55016003)(8936002)(41300700001)(8676002)(356005)(316002)(70586007)(70206006)(7636003)(40460700003)(86362001)(6916009)(5660300002)(36756003)(6666004)(7696005)(2906002)(36860700001)(82310400005)(6286002)(26005)(186003)(16526019)(1076003)(2616005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2023 15:29:15.7661 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1b51e0a2-568b-4ff3-380f-08db68fe4909 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT014.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5618 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org There is an intention to engage DPDK tracing capabilities for mlx5 PMDs monitoring and profiling in various modes. The patch introduces tracepoints for the Tx datapath in the ethernet device driver. Signed-off-by: Viacheslav Ovsiienko --- drivers/common/mlx5/meson.build | 1 + drivers/common/mlx5/mlx5_trace.c | 25 +++++++++++ drivers/common/mlx5/mlx5_trace.h | 72 ++++++++++++++++++++++++++++++++ drivers/common/mlx5/version.map | 8 ++++ 4 files changed, 106 insertions(+) create mode 100644 drivers/common/mlx5/mlx5_trace.c create mode 100644 drivers/common/mlx5/mlx5_trace.h diff --git a/drivers/common/mlx5/meson.build b/drivers/common/mlx5/meson.build index 1eefc02f06..28bfbfa324 100644 --- a/drivers/common/mlx5/meson.build +++ b/drivers/common/mlx5/meson.build @@ -19,6 +19,7 @@ sources += files( 'mlx5_common_mp.c', 'mlx5_common_mr.c', 'mlx5_malloc.c', + 'mlx5_trace.c', 'mlx5_common_pci.c', 'mlx5_common_devx.c', 'mlx5_common_utils.c', diff --git a/drivers/common/mlx5/mlx5_trace.c b/drivers/common/mlx5/mlx5_trace.c new file mode 100644 index 0000000000..b9f14413ad --- /dev/null +++ b/drivers/common/mlx5/mlx5_trace.c @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2022 NVIDIA Corporation & Affiliates + */ + +#include +#include + +RTE_TRACE_POINT_REGISTER(rte_pmd_mlx5_trace_tx_entry, + pmd.net.mlx5.tx.entry) + +RTE_TRACE_POINT_REGISTER(rte_pmd_mlx5_trace_tx_exit, + pmd.net.mlx5.tx.exit) + +RTE_TRACE_POINT_REGISTER(rte_pmd_mlx5_trace_tx_wqe, + pmd.net.mlx5.tx.wqe) + +RTE_TRACE_POINT_REGISTER(rte_pmd_mlx5_trace_tx_wait, + pmd.net.mlx5.tx.wait) + +RTE_TRACE_POINT_REGISTER(rte_pmd_mlx5_trace_tx_push, + pmd.net.mlx5.tx.push) + +RTE_TRACE_POINT_REGISTER(rte_pmd_mlx5_trace_tx_complete, + pmd.net.mlx5.tx.complete) + diff --git a/drivers/common/mlx5/mlx5_trace.h b/drivers/common/mlx5/mlx5_trace.h new file mode 100644 index 0000000000..57512e654f --- /dev/null +++ b/drivers/common/mlx5/mlx5_trace.h @@ -0,0 +1,72 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2022 NVIDIA Corporation & Affiliates + */ + +#ifndef RTE_PMD_MLX5_TRACE_H_ +#define RTE_PMD_MLX5_TRACE_H_ + +/** + * @file + * + * API for mlx5 PMD trace support + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include + +RTE_TRACE_POINT_FP( + rte_pmd_mlx5_trace_tx_entry, + RTE_TRACE_POINT_ARGS(uint16_t port_id, uint16_t queue_id), + rte_trace_point_emit_u16(port_id); + rte_trace_point_emit_u16(queue_id); +) + +RTE_TRACE_POINT_FP( + rte_pmd_mlx5_trace_tx_exit, + RTE_TRACE_POINT_ARGS(uint16_t nb_sent, uint16_t nb_req), + rte_trace_point_emit_u16(nb_sent); + rte_trace_point_emit_u16(nb_req); +) + +RTE_TRACE_POINT_FP( + rte_pmd_mlx5_trace_tx_wqe, + RTE_TRACE_POINT_ARGS(uint32_t opcode), + rte_trace_point_emit_u32(opcode); +) + +RTE_TRACE_POINT_FP( + rte_pmd_mlx5_trace_tx_wait, + RTE_TRACE_POINT_ARGS(uint64_t ts), + rte_trace_point_emit_u64(ts); +) + + +RTE_TRACE_POINT_FP( + rte_pmd_mlx5_trace_tx_push, + RTE_TRACE_POINT_ARGS(const struct rte_mbuf *mbuf, uint16_t wqe_id), + rte_trace_point_emit_ptr(mbuf); + rte_trace_point_emit_u32(mbuf->pkt_len); + rte_trace_point_emit_u16(mbuf->nb_segs); + rte_trace_point_emit_u16(wqe_id); +) + +RTE_TRACE_POINT_FP( + rte_pmd_mlx5_trace_tx_complete, + RTE_TRACE_POINT_ARGS(uint16_t port_id, uint16_t queue_id, + uint16_t wqe_id, uint64_t ts), + rte_trace_point_emit_u16(port_id); + rte_trace_point_emit_u16(queue_id); + rte_trace_point_emit_u64(ts); + rte_trace_point_emit_u16(wqe_id); +) + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PMD_MLX5_TRACE_H_ */ diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index e05e1aa8c5..d0ec8571e6 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -158,5 +158,13 @@ INTERNAL { mlx5_os_interrupt_handler_create; # WINDOWS_NO_EXPORT mlx5_os_interrupt_handler_destroy; # WINDOWS_NO_EXPORT + + __rte_pmd_mlx5_trace_tx_entry; + __rte_pmd_mlx5_trace_tx_exit; + __rte_pmd_mlx5_trace_tx_wqe; + __rte_pmd_mlx5_trace_tx_wait; + __rte_pmd_mlx5_trace_tx_push; + __rte_pmd_mlx5_trace_tx_complete; + local: *; }; From patchwork Fri Jun 9 15:28:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 128477 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C4A6E42C6F; Fri, 9 Jun 2023 17:29:37 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 765B243035; Fri, 9 Jun 2023 17:29:24 +0200 (CEST) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2052.outbound.protection.outlook.com [40.107.243.52]) by mails.dpdk.org (Postfix) with ESMTP id 049B343026 for ; Fri, 9 Jun 2023 17:29:22 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=e5OT3BFyF/GKjQ5S58GktwHEBzDqSMVZ8A0wRQ88NL6F2VEewyKjqP/RI7huvZRW3dwi+yIQou6t31HJDyjeAAa0d73RiZy2r0jKJ4oUelsia0asMaaVPyq+rj6ZXkJF0pKdf2S9VL7IAqC4wyYvnvgBotNRjbUcJPLQxv7XKMu+rhecyjdAel6LGjUb1NcNw9aiJQvtmqIVpl9dDCCi7945QCKx2Byl4DKZ3PycrFYAK3gacUHGSpPdvfB3oRy9J7be+PIeRl4Ci0Y6XzaBnO1RGA973EHATpWsUBH92DgBSjQmxvd3LVwWB+dWZgvgOZWjuS2m3+eUj4LU3U3W7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CTMbpJCZgFOGxkQemJU1R0Xaa5rsz6XaY8JH3nRkNPY=; b=Fq3OAIsYHNFSOiYS7mfxmN6GTcTEt68iNPwm8CX6y2FN2mMEieRfpapWvacq2Jdef7LSXn6rrrzjGSJi7TxszHKoHEQTuHd1PAnwZYltk9VSv/l51hUqqJpxYH5QSlnrPblyc8qGIN50UERNT/PIOvCOqN6vsVkKatATs0Q1KwXvmtsPueOg52iCglnulGGwGq1gDMsvIiAFBTRRbGmx4OeLy2IWcegNPpMW39M2J7hDghNGM1i80/iwkssodRtlvzemccETcKji1zmiC94JGwXyS3TGcXRFhXTmCVADvbChfo9ACqFQV9dxkV1FVznme00CtkNILkzG9Mfrk/rg7Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CTMbpJCZgFOGxkQemJU1R0Xaa5rsz6XaY8JH3nRkNPY=; b=jPGrN6kgsbIi2NNgDXOaZfCVHkBrLRkanpCwpjW1//eanvVicigowy4C6iHtWWsrUEPe59u5sQR2/l8I4dhYuuWslYCgY/MAnoHlrZQlxZEV2nUivnM0l+Dh/F3Gr986QHNeeb2YDxj6r6GYKWUxImjNMww9J0q4XFJO4JP60bwfSj4phyob+tTM98xHzS3T3eXdlsEOXKQSj8KERMxYSb1H3FbgItL0Rle0suLuNYy+c4OIEy5iCOYYpT2EjaF8aRuFu4Plo+6bkhnGJIB3du6ReF6XszOaRc/tHgNpEwAovvaI8KXKpZN55GKCv7/P4FiAAGrI2Ogi/rM6jVZf4A== Received: from SJ0PR03CA0003.namprd03.prod.outlook.com (2603:10b6:a03:33a::8) by LV3PR12MB9186.namprd12.prod.outlook.com (2603:10b6:408:197::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33; Fri, 9 Jun 2023 15:29:19 +0000 Received: from DM6NAM11FT108.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:33a:cafe::d9) by SJ0PR03CA0003.outlook.office365.com (2603:10b6:a03:33a::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.19 via Frontend Transport; Fri, 9 Jun 2023 15:29:18 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DM6NAM11FT108.mail.protection.outlook.com (10.13.172.95) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.29 via Frontend Transport; Fri, 9 Jun 2023 15:29:18 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Fri, 9 Jun 2023 08:29:09 -0700 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Fri, 9 Jun 2023 08:29:07 -0700 From: Viacheslav Ovsiienko To: Subject: [PATCH 3/5] net/mlx5: add Tx datapath tracing Date: Fri, 9 Jun 2023 18:28:45 +0300 Message-ID: <20230609152847.32496-4-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20230609152847.32496-1-viacheslavo@nvidia.com> References: <20230420100803.494-1-viacheslavo@nvidia.com> <20230609152847.32496-1-viacheslavo@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT108:EE_|LV3PR12MB9186:EE_ X-MS-Office365-Filtering-Correlation-Id: 8949d040-c2b5-430f-4d18-08db68fe4ab1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: y3Z7cniffDPp4et9sEuOTAJhAFanQydzaFTG+f25vgIqCn6sxnSuQ1xTebUL46F7UFxZGQZxKN/yiWGAEGJCxgS+TkaRcDQhyOtMFwZBcW0rdvnkeYDxVB85OjstIAK8xLPovflW+wyQ3rvrh4IU+41z/Xlb003NX7pD2w/D8Co82P9+8gV2t15Ip5MjdoU7H2kzR6QvXwc+w7+0wFIEzS28ZZre7xP7ZB1sV0O+D8C+NgoyaGzZ+fTCahjFkpmVYQI0I0Ks5Ec4HYl86vEdW4Sti5jJmE682vnkcXl9mzzgadguCuGzDoI5co+QHLAj/u5C1xcK+qxFxkWGP26ascdAOMZPH6nd4fVpTvi/71/dnJNPXYG9VmG6sn/cJnBVhdYd4S+FB1m8xvKTDWDQaywlxsjlQnWl6ZGhhYpeyvF7bfZHxWJgNtvurStocmuK/yzDJrIPzlAsU3FvvBQy8M7SXDCXZ1c67+qdWHw2VHgapCUSw/PB4kmwrr75wmRiLJTb+lGtF7San7pCIpvFZ6NV/r4t3FihfbhiKv3fVvqCLDtKRWh548DvFq7g4Z6EWTHc79CpK71+2dr6hq7swY1MiB5vG9JeTOHrR/r9ifYScsK/kZhi1hoAQCBlmVQmR48sNPtrgC1e0iRzEuNmR9XyMajn+1nmjY51zlmyHTJ6sVMbyPltfXJGC91XC2UM2UYGkR78AXFROjAO6+NE4ahWB93uiCb9Rfcg/lFmHLGE5R0I/k7+EtBG2HLdEplt X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(136003)(376002)(396003)(39860400002)(346002)(451199021)(36840700001)(40470700004)(46966006)(36756003)(5660300002)(2906002)(82310400005)(86362001)(40480700001)(55016003)(6666004)(40460700003)(83380400001)(1076003)(6286002)(36860700001)(426003)(7636003)(26005)(16526019)(478600001)(6916009)(316002)(336012)(356005)(2616005)(7696005)(82740400003)(70586007)(70206006)(186003)(47076005)(8936002)(8676002)(41300700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2023 15:29:18.5766 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8949d040-c2b5-430f-4d18-08db68fe4ab1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT108.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9186 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The patch adds tracing capability to Tx datapath. To engage this tracing capability the following steps should be taken: - meson option -Denable_trace_fp=true - meson option -Dc_args='-DALLOW_EXPERIMENTAL_API' - EAL command line parameter --trace=pmd.net.mlx5.tx.* The Tx datapath tracing allows to get information how packets are pushed into hardware descriptors, time stamping for scheduled wait and send completions, etc. To provide the human readable form of trace results the dedicated post-processing script is presumed. Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5_rx.h | 19 ------------------- drivers/net/mlx5/mlx5_rxtx.h | 19 +++++++++++++++++++ drivers/net/mlx5/mlx5_tx.c | 9 +++++++++ drivers/net/mlx5/mlx5_tx.h | 25 +++++++++++++++++++++++-- 4 files changed, 51 insertions(+), 21 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rx.h b/drivers/net/mlx5/mlx5_rx.h index 52c35c83f8..ed912ffb99 100644 --- a/drivers/net/mlx5/mlx5_rx.h +++ b/drivers/net/mlx5/mlx5_rx.h @@ -376,25 +376,6 @@ mlx5_rx_mb2mr(struct mlx5_rxq_data *rxq, struct rte_mbuf *mb) return mlx5_mr_mempool2mr_bh(mr_ctrl, mb->pool, addr); } -/** - * Convert timestamp from HW format to linear counter - * from Packet Pacing Clock Queue CQE timestamp format. - * - * @param sh - * Pointer to the device shared context. Might be needed - * to convert according current device configuration. - * @param ts - * Timestamp from CQE to convert. - * @return - * UTC in nanoseconds - */ -static __rte_always_inline uint64_t -mlx5_txpp_convert_rx_ts(struct mlx5_dev_ctx_shared *sh, uint64_t ts) -{ - RTE_SET_USED(sh); - return (ts & UINT32_MAX) + (ts >> 32) * NS_PER_S; -} - /** * Set timestamp in mbuf dynamic field. * diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index 876aa14ae6..b109d50758 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -43,4 +43,23 @@ int mlx5_queue_state_modify_primary(struct rte_eth_dev *dev, int mlx5_queue_state_modify(struct rte_eth_dev *dev, struct mlx5_mp_arg_queue_state_modify *sm); +/** + * Convert timestamp from HW format to linear counter + * from Packet Pacing Clock Queue CQE timestamp format. + * + * @param sh + * Pointer to the device shared context. Might be needed + * to convert according current device configuration. + * @param ts + * Timestamp from CQE to convert. + * @return + * UTC in nanoseconds + */ +static __rte_always_inline uint64_t +mlx5_txpp_convert_rx_ts(struct mlx5_dev_ctx_shared *sh, uint64_t ts) +{ + RTE_SET_USED(sh); + return (ts & UINT32_MAX) + (ts >> 32) * NS_PER_S; +} + #endif /* RTE_PMD_MLX5_RXTX_H_ */ diff --git a/drivers/net/mlx5/mlx5_tx.c b/drivers/net/mlx5/mlx5_tx.c index 14e1487e59..1fe9521dfc 100644 --- a/drivers/net/mlx5/mlx5_tx.c +++ b/drivers/net/mlx5/mlx5_tx.c @@ -232,6 +232,15 @@ mlx5_tx_handle_completion(struct mlx5_txq_data *__rte_restrict txq, MLX5_ASSERT((txq->fcqs[txq->cq_ci & txq->cqe_m] >> 16) == cqe->wqe_counter); #endif + if (__rte_trace_point_fp_is_enabled()) { + uint64_t ts = rte_be_to_cpu_64(cqe->timestamp); + uint16_t wqe_id = rte_be_to_cpu_16(cqe->wqe_counter); + + if (txq->rt_timestamp) + ts = mlx5_txpp_convert_rx_ts(NULL, ts); + rte_pmd_mlx5_trace_tx_complete(txq->port_id, txq->idx, + wqe_id, ts); + } ring_doorbell = true; ++txq->cq_ci; last_cqe = cqe; diff --git a/drivers/net/mlx5/mlx5_tx.h b/drivers/net/mlx5/mlx5_tx.h index cc8f7e98aa..7f624de58e 100644 --- a/drivers/net/mlx5/mlx5_tx.h +++ b/drivers/net/mlx5/mlx5_tx.h @@ -19,6 +19,8 @@ #include "mlx5.h" #include "mlx5_autoconf.h" +#include "mlx5_trace.h" +#include "mlx5_rxtx.h" /* TX burst subroutines return codes. */ enum mlx5_txcmp_code { @@ -764,6 +766,9 @@ mlx5_tx_cseg_init(struct mlx5_txq_data *__rte_restrict txq, cs->flags = RTE_BE32(MLX5_COMP_ONLY_FIRST_ERR << MLX5_COMP_MODE_OFFSET); cs->misc = RTE_BE32(0); + if (__rte_trace_point_fp_is_enabled() && !loc->pkts_sent) + rte_pmd_mlx5_trace_tx_entry(txq->port_id, txq->idx); + rte_pmd_mlx5_trace_tx_wqe((txq->wqe_ci << 8) | opcode); } /** @@ -1692,6 +1697,7 @@ mlx5_tx_schedule_send(struct mlx5_txq_data *restrict txq, if (txq->wait_on_time) { /* The wait on time capability should be used. */ ts -= sh->txpp.skew; + rte_pmd_mlx5_trace_tx_wait(ts); mlx5_tx_cseg_init(txq, loc, wqe, 1 + sizeof(struct mlx5_wqe_wseg) / MLX5_WSEG_SIZE, @@ -1706,6 +1712,7 @@ mlx5_tx_schedule_send(struct mlx5_txq_data *restrict txq, if (unlikely(wci < 0)) return MLX5_TXCMP_CODE_SINGLE; /* Build the WAIT WQE with specified completion. */ + rte_pmd_mlx5_trace_tx_wait(ts - sh->txpp.skew); mlx5_tx_cseg_init(txq, loc, wqe, 1 + sizeof(struct mlx5_wqe_qseg) / MLX5_WSEG_SIZE, @@ -1810,6 +1817,7 @@ mlx5_tx_packet_multi_tso(struct mlx5_txq_data *__rte_restrict txq, wqe = txq->wqes + (txq->wqe_ci & txq->wqe_m); loc->wqe_last = wqe; mlx5_tx_cseg_init(txq, loc, wqe, 0, MLX5_OPCODE_TSO, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); ds = mlx5_tx_mseg_build(txq, loc, wqe, vlan, inlen, 1, olx); wqe->cseg.sq_ds = rte_cpu_to_be_32(txq->qp_num_8s | ds); txq->wqe_ci += (ds + 3) / 4; @@ -1892,6 +1900,7 @@ mlx5_tx_packet_multi_send(struct mlx5_txq_data *__rte_restrict txq, wqe = txq->wqes + (txq->wqe_ci & txq->wqe_m); loc->wqe_last = wqe; mlx5_tx_cseg_init(txq, loc, wqe, ds, MLX5_OPCODE_SEND, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); mlx5_tx_eseg_none(txq, loc, wqe, olx); dseg = &wqe->dseg[0]; do { @@ -2115,6 +2124,7 @@ mlx5_tx_packet_multi_inline(struct mlx5_txq_data *__rte_restrict txq, wqe = txq->wqes + (txq->wqe_ci & txq->wqe_m); loc->wqe_last = wqe; mlx5_tx_cseg_init(txq, loc, wqe, 0, MLX5_OPCODE_SEND, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); ds = mlx5_tx_mseg_build(txq, loc, wqe, vlan, inlen, 0, olx); wqe->cseg.sq_ds = rte_cpu_to_be_32(txq->qp_num_8s | ds); txq->wqe_ci += (ds + 3) / 4; @@ -2318,8 +2328,8 @@ mlx5_tx_burst_tso(struct mlx5_txq_data *__rte_restrict txq, */ wqe = txq->wqes + (txq->wqe_ci & txq->wqe_m); loc->wqe_last = wqe; - mlx5_tx_cseg_init(txq, loc, wqe, ds, - MLX5_OPCODE_TSO, olx); + mlx5_tx_cseg_init(txq, loc, wqe, ds, MLX5_OPCODE_TSO, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); dseg = mlx5_tx_eseg_data(txq, loc, wqe, vlan, hlen, 1, olx); dptr = rte_pktmbuf_mtod(loc->mbuf, uint8_t *) + hlen - vlan; dlen -= hlen - vlan; @@ -2688,6 +2698,7 @@ mlx5_tx_burst_empw_simple(struct mlx5_txq_data *__rte_restrict txq, /* Update sent data bytes counter. */ slen += dlen; #endif + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); mlx5_tx_dseg_ptr (txq, loc, dseg, rte_pktmbuf_mtod(loc->mbuf, uint8_t *), @@ -2926,6 +2937,7 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *__rte_restrict txq, tlen += sizeof(struct rte_vlan_hdr); if (room < tlen) break; + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); dseg = mlx5_tx_dseg_vlan(txq, loc, dseg, dptr, dlen, olx); #ifdef MLX5_PMD_SOFT_COUNTERS @@ -2935,6 +2947,7 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *__rte_restrict txq, } else { if (room < tlen) break; + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); dseg = mlx5_tx_dseg_empw(txq, loc, dseg, dptr, dlen, olx); } @@ -2980,6 +2993,7 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *__rte_restrict txq, if (MLX5_TXOFF_CONFIG(VLAN)) MLX5_ASSERT(!(loc->mbuf->ol_flags & RTE_MBUF_F_TX_VLAN)); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); mlx5_tx_dseg_ptr(txq, loc, dseg, dptr, dlen, olx); /* We have to store mbuf in elts.*/ txq->elts[txq->elts_head++ & txq->elts_m] = loc->mbuf; @@ -3194,6 +3208,7 @@ mlx5_tx_burst_single_send(struct mlx5_txq_data *__rte_restrict txq, loc->wqe_last = wqe; mlx5_tx_cseg_init(txq, loc, wqe, seg_n, MLX5_OPCODE_SEND, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); mlx5_tx_eseg_data(txq, loc, wqe, vlan, inlen, 0, olx); txq->wqe_ci += wqe_n; @@ -3256,6 +3271,7 @@ mlx5_tx_burst_single_send(struct mlx5_txq_data *__rte_restrict txq, loc->wqe_last = wqe; mlx5_tx_cseg_init(txq, loc, wqe, ds, MLX5_OPCODE_SEND, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); dseg = mlx5_tx_eseg_data(txq, loc, wqe, vlan, txq->inlen_mode, 0, olx); @@ -3297,6 +3313,7 @@ mlx5_tx_burst_single_send(struct mlx5_txq_data *__rte_restrict txq, loc->wqe_last = wqe; mlx5_tx_cseg_init(txq, loc, wqe, 4, MLX5_OPCODE_SEND, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); mlx5_tx_eseg_dmin(txq, loc, wqe, vlan, olx); dptr = rte_pktmbuf_mtod(loc->mbuf, uint8_t *) + MLX5_ESEG_MIN_INLINE_SIZE - vlan; @@ -3338,6 +3355,7 @@ mlx5_tx_burst_single_send(struct mlx5_txq_data *__rte_restrict txq, loc->wqe_last = wqe; mlx5_tx_cseg_init(txq, loc, wqe, 3, MLX5_OPCODE_SEND, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); mlx5_tx_eseg_none(txq, loc, wqe, olx); mlx5_tx_dseg_ptr (txq, loc, &wqe->dseg[0], @@ -3707,6 +3725,9 @@ mlx5_tx_burst_tmpl(struct mlx5_txq_data *__rte_restrict txq, #endif if (MLX5_TXOFF_CONFIG(INLINE) && loc.mbuf_free) __mlx5_tx_free_mbuf(txq, pkts, loc.mbuf_free, olx); + /* Trace productive bursts only. */ + if (__rte_trace_point_fp_is_enabled() && loc.pkts_sent) + rte_pmd_mlx5_trace_tx_exit(loc.pkts_sent, pkts_n); return loc.pkts_sent; } From patchwork Fri Jun 9 15:28:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 128478 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C400A42C6F; Fri, 9 Jun 2023 17:29:44 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A051C4303D; Fri, 9 Jun 2023 17:29:26 +0200 (CEST) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2083.outbound.protection.outlook.com [40.107.94.83]) by mails.dpdk.org (Postfix) with ESMTP id 51D7343033 for ; Fri, 9 Jun 2023 17:29:24 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bA6X+UnlypU5dU2XEz5iFY0XvAQSBrcFX9QTr4SDtHZioCBZEnTDtOs/5CT0OPuXHnxmkNYOpIfPc2iVm78wxv30fYTFgObrcXfmAHYd+vmMA8HGePxJ5KguMzuzOqL4TDZ3qvXYX+NC29AFe+4N727OVTMk6ROCDiJYNYrTNUssYKsc5XbxhefyRNa5sSV/Mo2HSfnQSE2mnwuFXcmgDdt70KIyf33h3ztRyf8Ve9fvez61OAj2eFWVnLXC150Jh1lm53HDq7a2Npd1X4QoxK1KiWdrxUYWoZwURhca9YAUIs7VBZt6AVg8bL5DMDgiZbcmRtPrwg4ec2vhVO4tfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZPZy0912N+8rrr56xuslSXObdPBbtqNskbc/7siw6LA=; b=WrckFeiqql5NGU7AjHVtkgRiNjRhrRBlROrwEbYufpkKKrVNUb0Xen1spkshF7h+6pBgFnHRfTGMheULVZnkoYPesfDhYj8lVzyn38JjTuZRypgZHMbqegbtl53iQZQQyeuenR0fp5iCMZ3AimrwG2E4uAO1087Do30Z0Its0yIDg1iMS5hNhBK+CvJTn1Z/Ttc6jrA7RJhNXnBvJGpeWgJ6tkkjHg/snyh5hiO/MJMy8ylGDcHSpjETvmbnr4hAw8OBof1kw0CU4C1rtlK01rO3pznESjA7Pasv1hlpu6ToxnXuNPehFJ4dlzrX8tQ2Cl3/7BFKOtMyApyGuEjuew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZPZy0912N+8rrr56xuslSXObdPBbtqNskbc/7siw6LA=; b=JkjaeT2Gs04urNCbgfDOC8IpytKA5Ru5jdTfbdHpaZYBjj0N8NMWYQ8RExkITOQW/smg56Bj14c1myVTfTcu+HNkmEX5R55ZS+UnOBLVtx0XR7zXAeA5dzfsLznKeKD67rVi1pucnJRdEqhIQ/SOYHcGtVdSOhHey90Z8GXkNgLfgW9dlFAd6g+3bwkP9H63KRiUVdwfLDQFHDJFq1vqj+H5uKjgk/z/RZEsFyIyw2h8ADfbED7xFTS0Nxwjlh3UPWy7mL4m+5SjuTmfUXzsV7IOgj+gnwmmH0QYigL9tsYxHYblarMqyAXKx/0tvmERhi16JjQpKuh+RdN7t+szIw== Received: from SA0PR12CA0017.namprd12.prod.outlook.com (2603:10b6:806:6f::22) by SJ0PR12MB6878.namprd12.prod.outlook.com (2603:10b6:a03:483::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.32; Fri, 9 Jun 2023 15:29:22 +0000 Received: from SN1PEPF000252A3.namprd05.prod.outlook.com (2603:10b6:806:6f:cafe::ac) by SA0PR12CA0017.outlook.office365.com (2603:10b6:806:6f::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.27 via Frontend Transport; Fri, 9 Jun 2023 15:29:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by SN1PEPF000252A3.mail.protection.outlook.com (10.167.242.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.13 via Frontend Transport; Fri, 9 Jun 2023 15:29:21 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Fri, 9 Jun 2023 08:29:10 -0700 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Fri, 9 Jun 2023 08:29:09 -0700 From: Viacheslav Ovsiienko To: Subject: [PATCH 4/5] net/mlx5: add comprehensive send completion trace Date: Fri, 9 Jun 2023 18:28:46 +0300 Message-ID: <20230609152847.32496-5-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20230609152847.32496-1-viacheslavo@nvidia.com> References: <20230420100803.494-1-viacheslavo@nvidia.com> <20230609152847.32496-1-viacheslavo@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF000252A3:EE_|SJ0PR12MB6878:EE_ X-MS-Office365-Filtering-Correlation-Id: 53cb37dc-1feb-4337-19e5-08db68fe4ca8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Yg//iHHx6GO6kUjtFF7fmC4bXFn196Cz662kKZXLWSB8WDVwoWnda5csIHzF9C0nOsWpVx53AztQo+h5bwvglJnftrnjbD5M7Ph2lD/OSGCLPXCoEgNv4MXcRejqbfIcMG/v+yVNk31WiyX8Wfu4QeA35te3/J/e9zE9JH2QGJ8h1d99O7k0XvYOsHJ1W9y7cF2Enl9MjTafjs5ef1IChWg8e4TwY9k4YIHD96Ctdm82d8Gu8DrSoT0Xm8QCmngINRqE03Stl/dNrSHk9AE5z4xlCCYOgarvzSKrk7URa3W3/WMZyyWY8eIgviN/pWN8mEYPfvIFqOIx+P1v61ybPgkcDljeOvq+us5DMEV2RdZUkoC2f5sQYAVSPbEBUNShPd5U5Jqko3Kg5EpqeHCTQIaScMk8QKuMDDjd7aj9WBxagT5/X9JvJKGwoUspo9SBz0ZYxjUdpgnrxQeajuSVq05/84jqp0GNAYWkeGwzLIbFdR1renoWJPXqsY4R0QPhY0B56dPSP40P6AwGCgKqAnfLg4CUXhtTiZ5TPoQMKxMqhPDrXCrViOstLtNoda50YWHaXFEGshxaTzpVVgVJB4H4OCn/FIJOxQSsqXg41NXDrMN09z64BwHkvLHt4GuLt5s+1zT/YR6vUogq8pfMs6SLTiEs+t5fTKuV/sQw1EBu3vSPk+8JedwulTgk+BUMugyAPfQgJgrJKz2oI32DEtYJuPe2hyxzHSEFhaCPIvsxhJL9s0ahNVQPxSQf954Z X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(346002)(136003)(376002)(396003)(39860400002)(451199021)(46966006)(36840700001)(40470700004)(186003)(26005)(16526019)(1076003)(6286002)(2616005)(426003)(336012)(83380400001)(86362001)(47076005)(82310400005)(6666004)(2906002)(7696005)(36756003)(36860700001)(8676002)(82740400003)(55016003)(40480700001)(6916009)(5660300002)(41300700001)(7636003)(40460700003)(356005)(8936002)(478600001)(70206006)(70586007)(316002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2023 15:29:21.8561 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 53cb37dc-1feb-4337-19e5-08db68fe4ca8 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF000252A3.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6878 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org There is the demand to trace the send completions of every WQE if time scheduling is enabled. The patch extends the size of completion queue and requests completion on every issued WQE in the send queue. As the result hardware provides CQE on each completed WQE and driver is able to fetch completion timestamp for dedicated operation. The add code is under conditional compilation RTE_ENABLE_TRACE_FP flag and does not impact the release code. Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_verbs.c | 8 +++- drivers/net/mlx5/mlx5_devx.c | 8 +++- drivers/net/mlx5/mlx5_tx.h | 63 +++++++++++++++++++++++++++-- 3 files changed, 71 insertions(+), 8 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_verbs.c b/drivers/net/mlx5/linux/mlx5_verbs.c index 7233c2c7fa..b54f3ccd9a 100644 --- a/drivers/net/mlx5/linux/mlx5_verbs.c +++ b/drivers/net/mlx5/linux/mlx5_verbs.c @@ -968,8 +968,12 @@ mlx5_txq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx) rte_errno = EINVAL; return -rte_errno; } - cqe_n = desc / MLX5_TX_COMP_THRESH + - 1 + MLX5_TX_COMP_THRESH_INLINE_DIV; + if (__rte_trace_point_fp_is_enabled() && + txq_data->offloads & RTE_ETH_TX_OFFLOAD_SEND_ON_TIMESTAMP) + cqe_n = UINT16_MAX / 2 - 1; + else + cqe_n = desc / MLX5_TX_COMP_THRESH + + 1 + MLX5_TX_COMP_THRESH_INLINE_DIV; txq_obj->cq = mlx5_glue->create_cq(priv->sh->cdev->ctx, cqe_n, NULL, NULL, 0); if (txq_obj->cq == NULL) { diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index 4369d2557e..5082a7e178 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -1465,8 +1465,12 @@ mlx5_txq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx) MLX5_ASSERT(ppriv); txq_obj->txq_ctrl = txq_ctrl; txq_obj->dev = dev; - cqe_n = (1UL << txq_data->elts_n) / MLX5_TX_COMP_THRESH + - 1 + MLX5_TX_COMP_THRESH_INLINE_DIV; + if (__rte_trace_point_fp_is_enabled() && + txq_data->offloads & RTE_ETH_TX_OFFLOAD_SEND_ON_TIMESTAMP) + cqe_n = UINT16_MAX / 2 - 1; + else + cqe_n = (1UL << txq_data->elts_n) / MLX5_TX_COMP_THRESH + + 1 + MLX5_TX_COMP_THRESH_INLINE_DIV; log_desc_n = log2above(cqe_n); cqe_n = 1UL << log_desc_n; if (cqe_n > UINT16_MAX) { diff --git a/drivers/net/mlx5/mlx5_tx.h b/drivers/net/mlx5/mlx5_tx.h index 7f624de58e..9f29df280f 100644 --- a/drivers/net/mlx5/mlx5_tx.h +++ b/drivers/net/mlx5/mlx5_tx.h @@ -728,6 +728,54 @@ mlx5_tx_request_completion(struct mlx5_txq_data *__rte_restrict txq, } } +/** + * Set completion request flag for all issued WQEs. + * This routine is intended to be used with enabled fast path tracing + * and send scheduling on time to provide the detailed report in trace + * for send completions on every WQE. + * + * @param txq + * Pointer to TX queue structure. + * @param loc + * Pointer to burst routine local context. + * @param olx + * Configured Tx offloads mask. It is fully defined at + * compile time and may be used for optimization. + */ +static __rte_always_inline void +mlx5_tx_request_completion_trace(struct mlx5_txq_data *__rte_restrict txq, + struct mlx5_txq_local *__rte_restrict loc, + unsigned int olx) +{ + uint16_t head = txq->elts_comp; + + while (txq->wqe_comp != txq->wqe_ci) { + volatile struct mlx5_wqe *wqe; + uint32_t wqe_n; + + MLX5_ASSERT(loc->wqe_last); + wqe = txq->wqes + (txq->wqe_comp & txq->wqe_m); + if (wqe == loc->wqe_last) { + head = txq->elts_head; + head += MLX5_TXOFF_CONFIG(INLINE) ? + 0 : loc->pkts_sent - loc->pkts_copy; + txq->elts_comp = head; + } + /* Completion request flag was set on cseg constructing. */ +#ifdef RTE_LIBRTE_MLX5_DEBUG + txq->fcqs[txq->cq_pi++ & txq->cqe_m] = head | + (wqe->cseg.opcode >> 8) << 16; +#else + txq->fcqs[txq->cq_pi++ & txq->cqe_m] = head; +#endif + /* A CQE slot must always be available. */ + MLX5_ASSERT((txq->cq_pi - txq->cq_ci) <= txq->cqe_s); + /* Advance to the next WQE in the queue. */ + wqe_n = rte_be_to_cpu_32(wqe->cseg.sq_ds) & 0x3F; + txq->wqe_comp += RTE_ALIGN(wqe_n, 4) / 4; + } +} + /** * Build the Control Segment with specified opcode: * - MLX5_OPCODE_SEND @@ -754,7 +802,7 @@ mlx5_tx_cseg_init(struct mlx5_txq_data *__rte_restrict txq, struct mlx5_wqe *__rte_restrict wqe, unsigned int ds, unsigned int opcode, - unsigned int olx __rte_unused) + unsigned int olx) { struct mlx5_wqe_cseg *__rte_restrict cs = &wqe->cseg; @@ -763,8 +811,12 @@ mlx5_tx_cseg_init(struct mlx5_txq_data *__rte_restrict txq, opcode = MLX5_OPCODE_TSO | MLX5_OPC_MOD_MPW << 24; cs->opcode = rte_cpu_to_be_32((txq->wqe_ci << 8) | opcode); cs->sq_ds = rte_cpu_to_be_32(txq->qp_num_8s | ds); - cs->flags = RTE_BE32(MLX5_COMP_ONLY_FIRST_ERR << - MLX5_COMP_MODE_OFFSET); + if (MLX5_TXOFF_CONFIG(TXPP) && __rte_trace_point_fp_is_enabled()) + cs->flags = RTE_BE32(MLX5_COMP_ALWAYS << + MLX5_COMP_MODE_OFFSET); + else + cs->flags = RTE_BE32(MLX5_COMP_ONLY_FIRST_ERR << + MLX5_COMP_MODE_OFFSET); cs->misc = RTE_BE32(0); if (__rte_trace_point_fp_is_enabled() && !loc->pkts_sent) rte_pmd_mlx5_trace_tx_entry(txq->port_id, txq->idx); @@ -3662,7 +3714,10 @@ mlx5_tx_burst_tmpl(struct mlx5_txq_data *__rte_restrict txq, if (unlikely(loc.pkts_sent == loc.pkts_loop)) goto burst_exit; /* Request CQE generation if limits are reached. */ - mlx5_tx_request_completion(txq, &loc, olx); + if (MLX5_TXOFF_CONFIG(TXPP) && __rte_trace_point_fp_is_enabled()) + mlx5_tx_request_completion_trace(txq, &loc, olx); + else + mlx5_tx_request_completion(txq, &loc, olx); /* * Ring QP doorbell immediately after WQE building completion * to improve latencies. The pure software related data treatment From patchwork Fri Jun 9 15:28:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 128479 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7233442C6F; Fri, 9 Jun 2023 17:29:54 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0465343039; Fri, 9 Jun 2023 17:29:29 +0200 (CEST) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2040.outbound.protection.outlook.com [40.107.243.40]) by mails.dpdk.org (Postfix) with ESMTP id D968543038 for ; Fri, 9 Jun 2023 17:29:27 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VgOT1HCfDAzhkZQnuQRoIHuDggywiGuvTzJH2qNnY/Bc9qqcxblE2k7gQdsW+sj06Bs8t19C6AY/kec2VKMK2W+VTdopWDau+nmKUJOjimPpUotEaP9euRRN+uveNucgKBPkSgFcqcSCiOFsbHKQwm3KbDTazDi4YVBA69nBERjn9RDuZgek60GnEEfnYvbLHC1QDY6T/X56tVA9Sit+nCzpC+ndkLKCQ1oN8/tb995h2vzfKbtiOpJ9JIXXu3zOMwIqCo8aIeMLnGgEUb3Bm72g+iQqyDNsqBRipngGl02X5z8PHU+AOK90v3ho2KKozM5Li/c0WIcS4WBu9gBTeQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MYiacIuwMsvJ4xeh+TMKQLFoa38kzzHwJ+42WXNm0ZU=; b=c3D7+iwYNHznVj2uUgcdLCZFmRY/6meYM9fj02nCDQeA59eBsxDm/H41uXIHuoD0P+C4w2KTVpaOSGqn9xBFaDgiMOGsgVhQyDt9rYtuiAdPmhfUPDwpTkIF5VETAMmuAetEFPmN4HYeEtNGmgJDCsWDDgDeg/USiqry8fUMst0B7yUEM4BR65Wh9h22VI4yZ/CcAjHtRjlF7/ZoTyBx0khYuKBKCojKzDhofFzZ2G1Qpuwk/0IjOmjSrhpAzvijfii0hLImZIfokAXngUgCbB6BclS2EcrPHXcb11PEXBgcWnIxdDwgEOj27I44yrFdcN1/boTBAqfW29fBlfTj3g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MYiacIuwMsvJ4xeh+TMKQLFoa38kzzHwJ+42WXNm0ZU=; b=BT5oH7Py0m+YDbHEYpG8OZ0oKAsY3jUdSm4i0oMvUWZoz6sTym9/BTUpnQkiw3d41vnmcU9lDvNoNXgufY9ivQWTbE0U4ASGBLyHcDfshP8ywUTpmemyRwoZI8xLvphPJzci5CJnnxKavLux/XBdv3kWAPI2HwVx/3B/WWB38EyefeCd0ciHLoLFtC4cnA10WJYpwK6ner3tsK9gvfj48uyCw2n2S6X2oI36FhKnn1Ve2anf62ckMk1D1aeglZ1N9WqqRVn0J+QtuLDsL5qFANv/dEnQ7aJUP+KKuk4grKZhD6sniqon0A5BqKzuwBkG5spVqHwPUmKPQczlHB0rvw== Received: from DS7PR06CA0021.namprd06.prod.outlook.com (2603:10b6:8:2a::23) by MW4PR12MB6950.namprd12.prod.outlook.com (2603:10b6:303:207::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33; Fri, 9 Jun 2023 15:29:25 +0000 Received: from DM6NAM11FT014.eop-nam11.prod.protection.outlook.com (2603:10b6:8:2a:cafe::5) by DS7PR06CA0021.outlook.office365.com (2603:10b6:8:2a::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.29 via Frontend Transport; Fri, 9 Jun 2023 15:29:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by DM6NAM11FT014.mail.protection.outlook.com (10.13.173.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.29 via Frontend Transport; Fri, 9 Jun 2023 15:29:25 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Fri, 9 Jun 2023 08:29:11 -0700 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Fri, 9 Jun 2023 08:29:10 -0700 From: Viacheslav Ovsiienko To: Subject: [PATCH 5/5] net/mlx5: add Tx datapath trace analyzing script Date: Fri, 9 Jun 2023 18:28:47 +0300 Message-ID: <20230609152847.32496-6-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20230609152847.32496-1-viacheslavo@nvidia.com> References: <20230420100803.494-1-viacheslavo@nvidia.com> <20230609152847.32496-1-viacheslavo@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT014:EE_|MW4PR12MB6950:EE_ X-MS-Office365-Filtering-Correlation-Id: 1911556b-3906-41fb-c663-08db68fe4ee4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wSvuERHoBWVe2LjiQ8u2wH0W6/iDV/74lwfbPHX54Gb5gufdEm6/p9FGSP4dGvZ9/2QIYNIVc983jcqxET5SxQFX+4hsXdSsAHvrXM1Ko2fyHgkbbPRCgG8sFrEkVg7pwnHtz/KqU2WYTlcXeI82c8C2aZDE/+6ltUV4tmzI9hQjFnJi7ecDWWnsJCykqFnDQeMJURE4V2Xdu4L0HraeDwEOS5GmM4157ezoEHqHGNj5oBAX3DtJ3kxZ47zLPx8sQjwW9PfbRxRkxgMMJKaAPlQHXay8vLnQA1HdmhQOWchGHsP0ia5ADFeqSZfChzVTj/SQXb2Fe/KOXRcwqWIaVBFMJCeFPQ2uJUEzjGTCkLuxQ8y2805tDPFPTjGBMO/5OSC4fmHxhlKE2cJ4gdjxKCrZBS7bFMhxFRQDztXiURpkgWJMBbJqgtPRypoyUBPj8K1g891dVri0L2Gd80cJ2REXDrq+tAFpKpNiwKSI6/fL7xOgs/LephwlJtteakOX1CGVzDwIDMOHC4PHlXt0lrDwf32QgWQ/mebDbjKWRHu2qw/+bl14FXorglN84JInyNQbMX647z2JHD8AVMBPpzqvOAvN888WUop46MS4aFYqQUxshDFw7GXe/1fEWbkSYcKlOTmoL1cv09Nm/RMqqBGAHPf/Mhg736vfdcAHqinAyNvr+02WkusUGX6WPerigNwYRxFrg5UnISAmxeiQmyKUH5xjWBjsZInlni5qPgdjaCNd+Hn8EqB5mYEp4vLZ X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(136003)(376002)(396003)(39860400002)(346002)(451199021)(36840700001)(40470700004)(46966006)(36756003)(5660300002)(2906002)(82310400005)(86362001)(40480700001)(55016003)(6666004)(40460700003)(83380400001)(1076003)(6286002)(36860700001)(426003)(7636003)(26005)(16526019)(478600001)(6916009)(316002)(336012)(356005)(2616005)(7696005)(82740400003)(70586007)(70206006)(186003)(47076005)(8936002)(8676002)(41300700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2023 15:29:25.6248 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1911556b-3906-41fb-c663-08db68fe4ee4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT014.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB6950 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The Python script is intended to analyze mlx5 PMD datapath traces and report: - tx_burst routine timings - how packets are pushed to WQEs - how packet sending is completed with timings Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/tools/mlx5_trace.py | 271 +++++++++++++++++++++++++++ 1 file changed, 271 insertions(+) create mode 100755 drivers/net/mlx5/tools/mlx5_trace.py diff --git a/drivers/net/mlx5/tools/mlx5_trace.py b/drivers/net/mlx5/tools/mlx5_trace.py new file mode 100755 index 0000000000..c8fa63a7b9 --- /dev/null +++ b/drivers/net/mlx5/tools/mlx5_trace.py @@ -0,0 +1,271 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2023 NVIDIA Corporation & Affiliates + +''' +Analyzing the mlx5 PMD datapath tracings +''' +import sys +import argparse +import pathlib +import bt2 + +PFX_TX = "pmd.net.mlx5.tx." +PFX_TX_LEN = len(PFX_TX) + +tx_blst = {} # current Tx bursts per CPU +tx_qlst = {} # active Tx queues per port/queue +tx_wlst = {} # wait timestamp list per CPU + +class mlx5_queue(object): + def __init__(self): + self.done_burst = [] # completed bursts + self.wait_burst = [] # waiting for completion + self.pq_id = 0 + + def log(self): + for txb in self.done_burst: + txb.log() + + +class mlx5_mbuf(object): + def __init__(self): + self.wqe = 0 # wqe id + self.ptr = None # first packet mbuf pointer + self.len = 0 # packet data length + self.nseg = 0 # number of segments + + def log(self): + out = " %X: %u" % (self.ptr, self.len) + if self.nseg != 1: + out += " (%d segs)" % self.nseg + print(out) + + +class mlx5_wqe(object): + def __init__(self): + self.mbuf = [] # list of mbufs in WQE + self.wait_ts = 0 # preceding wait/push timestamp + self.comp_ts = 0 # send/recv completion timestamp + self.opcode = 0 + + def log(self): + id = (self.opcode >> 8) & 0xFFFF + op = self.opcode & 0xFF + fl = self.opcode >> 24 + out = " %04X: " % id + if op == 0xF: + out += "WAIT" + elif op == 0x29: + out += "EMPW" + elif op == 0xE: + out += "TSO " + elif op == 0xA: + out += "SEND" + else: + out += "0x%02X" % op + if self.comp_ts != 0: + out += " (%d, %d)" % (self.wait_ts, self.comp_ts - self.wait_ts) + else: + out += " (%d)" % self.wait_ts + print(out) + for mbuf in self.mbuf: + mbuf.log() + + # return 0 if WQE in not completed + def comp(self, wqe_id, ts): + if self.comp_ts != 0: + return 1 + id = (self.opcode >> 8) & 0xFFFF + if id > wqe_id: + id -= wqe_id + if id <= 0x8000: + return 0 + else: + id = wqe_id - id + if id >= 0x8000: + return 0 + self.comp_ts = ts + return 1 + + +class mlx5_burst(object): + def __init__(self): + self.wqes = [] # issued burst WQEs + self.done = 0 # number of sent/recv packets + self.req = 0 # requested number of packets + self.call_ts = 0 # burst routine invocation + self.done_ts = 0 # burst routine done + self.queue = None + + def log(self): + port = self.queue.pq_id >> 16 + queue = self.queue.pq_id & 0xFFFF + if self.req == 0: + print("%u: tx(p=%u, q=%u, %u/%u pkts (incomplete)" % + (self.call_ts, port, queue, self.done, self.req)) + else: + print("%u: tx(p=%u, q=%u, %u/%u pkts in %u" % + (self.call_ts, port, queue, self.done, self.req, + self.done_ts - self.call_ts)) + for wqe in self.wqes: + wqe.log() + + # return 0 if not all of WQEs in burst completed + def comp(self, wqe_id, ts): + wlen = len(self.wqes) + if wlen == 0: + return 0 + for wqe in self.wqes: + if wqe.comp(wqe_id, ts) == 0: + return 0 + return 1 + + +def do_tx_entry(msg): + event = msg.event + cpu_id = event["cpu_id"] + burst = tx_blst.get(cpu_id) + if burst is not None: + # continue existing burst after WAIT + return + # allocate the new burst and append to the queue + burst = mlx5_burst() + burst.call_ts = msg.default_clock_snapshot.ns_from_origin + tx_blst[cpu_id] = burst + pq_id = event["port_id"] << 16 | event["queue_id"] + queue = tx_qlst.get(pq_id) + if queue is None: + # queue does not exist - allocate the new one + queue = mlx5_queue(); + queue.pq_id = pq_id + tx_qlst[pq_id] = queue + burst.queue = queue + queue.wait_burst.append(burst) + + +def do_tx_exit(msg): + event = msg.event + cpu_id = event["cpu_id"] + burst = tx_blst.get(cpu_id) + if burst is None: + return + burst.done_ts = msg.default_clock_snapshot.ns_from_origin + burst.req = event["nb_req"] + burst.done = event["nb_sent"] + tx_blst.pop(cpu_id) + + +def do_tx_wqe(msg): + event = msg.event + cpu_id = event["cpu_id"] + burst = tx_blst.get(cpu_id) + if burst is None: + return + wqe = mlx5_wqe() + wqe.wait_ts = tx_wlst.get(cpu_id) + if wqe.wait_ts is None: + wqe.wait_ts = msg.default_clock_snapshot.ns_from_origin + wqe.opcode = event["opcode"] + burst.wqes.append(wqe) + + +def do_tx_wait(msg): + event = msg.event + cpu_id = event["cpu_id"] + tx_wlst[cpu_id] = event["ts"] + + +def do_tx_push(msg): + event = msg.event + cpu_id = event["cpu_id"] + burst = tx_blst.get(cpu_id) + if burst is None: + return + if not burst.wqes: + return + wqe = burst.wqes[-1] + mbuf = mlx5_mbuf() + mbuf.wqe = event["wqe_id"] + mbuf.ptr = event["mbuf"] + mbuf.len = event["mbuf_pkt_len"] + mbuf.nseg = event["mbuf_nb_segs"] + wqe.mbuf.append(mbuf) + + +def do_tx_complete(msg): + event = msg.event + pq_id = event["port_id"] << 16 | event["queue_id"] + queue = tx_qlst.get(pq_id) + if queue is None: + return + qlen = len(queue.wait_burst) + if qlen == 0: + return + wqe_id = event["wqe_id"] + ts = event["ts"] + rmv = 0 + while rmv < qlen: + burst = queue.wait_burst[rmv] + if burst.comp(wqe_id, ts) == 0: + break + rmv += 1 + # mode completed burst to done list + if rmv != 0: + idx = 0 + while idx < rmv: + queue.done_burst.append(burst) + idx += 1 + del queue.wait_burst[0:rmv] + + +def do_tx(msg): + name = msg.event.name[PFX_TX_LEN:] + if name == "entry": + do_tx_entry(msg) + elif name == "exit": + do_tx_exit(msg) + elif name == "wqe": + do_tx_wqe(msg) + elif name == "wait": + do_tx_wait(msg) + elif name == "push": + do_tx_push(msg) + elif name == "complete": + do_tx_complete(msg) + else: + print("Error: unrecognized Tx event name: %s" % msg.event.name) + sys.exit(1) + + +def do_log(msg_it): + for msg in msg_it: + if type(msg) is not bt2._EventMessageConst: + continue + event = msg.event + if event.name.startswith(PFX_TX): + do_tx(msg) + # Handling of other log event cathegories can be added here + + +def do_print(): + for pq_id in tx_qlst: + queue = tx_qlst.get(pq_id) + queue.log() + + +def main(args): + parser = argparse.ArgumentParser() + parser.add_argument("path", + nargs = 1, + type = str, + help = "input trace folder") + args = parser.parse_args() + + msg_it = bt2.TraceCollectionMessageIterator(args.path) + do_log(msg_it) + do_print() + exit(0) + +if __name__ == "__main__": + main(sys.argv)