From patchwork Tue Jun 13 16:58:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 128638 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4DCF642CA7; Tue, 13 Jun 2023 18:59:48 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CF64E42C24; Tue, 13 Jun 2023 18:59:26 +0200 (CEST) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2043.outbound.protection.outlook.com [40.107.223.43]) by mails.dpdk.org (Postfix) with ESMTP id 69DC942D13 for ; Tue, 13 Jun 2023 18:59:24 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T9EWjGsV4LJAgUbK9Idhv2WvA+rhPOzRuMik8c88RgRog7pTGWg8n+MVzAGx9FeQxoCVUnBxONPboY7TLmuYC/mAqbqPfrfjZJqu/F3ukw6kWgSUNmo+X2/4twkHXwkdnlA0xoVgdSEDrAAZcS08TCw2ADYWyha3Ef1Y5//a72n9am+nsTMuQJXcxTcD1u2QdAqSx1MrqKCpmXyG3ykJzsQLvxRd+yGSvRVv8ZXxjV56ttc+4Yd/f3tgoHVvoLkZgusebxq2vk+nVRjHMn9wd7HEVekXtVPyS/UZDGoMxv8PFNJVmMM462s/KscgX96JDqGZsMouAR05uEqy8fqLkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WkNGOBOviwhxhqyOxku/SGMP9tcmir4SFe/Lpf45Le4=; b=RgoxrTk5cnnDXjEgBAkXvogSpW1TaVa3XfFYZLh0DZMWH8mhxg0akx3VLKd/e6yD6jEN1NneOD/oFdS7C/twK0paMbV9MqLSI/HJnbJTGXMfuNXqQxO9/Wxd5S3X+HHGPQC4p36gawgChccY1OfavpCz3MnTFs24cl/B1DBQppQ2q+/X7MzFG56Qa6T9oPEf33ak9jGfW45u4DtzsJFkYp3OCFjJfGRcI7V4iH1lkkGglwY5+UFGyqAJ0IeYrEztL9A71IwKc5LRJu4hbkdtdUXT7iyn5bxncByn2sluXDE6cFzUezq9gs58G/AZ9cQi7UEaEOXCL4eHQWKdXtY26Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WkNGOBOviwhxhqyOxku/SGMP9tcmir4SFe/Lpf45Le4=; b=nGUN5onVWSlt828nhD69MFYJXgvyKDbKmX9H+Og4v8BvHAyeGYB3jwbTtqjrzf5sOjKU7BRDfsDRgXB663FLWdvd6TkyMIhNLRSqu0wibOoIEqK6mmoXYCY2p8nOrAxa4VUl4yjR5scubEZzhgXXTmUcps/9qKEDd+GRPxfiFOj0l2iiqlCYdJMIpkHoQtZXJCAsZYBA04dYJ5NwaUfgheEAktMC+sN2L/7SvvGnNv4A5vCgjKxa4jRaRiLp/KmYB/QYOkBSdRq4g5tIx3lcZzld1Njmhuk4toC1I/pnLlcmzL1D7q0DAmldJBkuJCH6MeL01QdyjFD2CZz3QDs+rg== Received: from BN9PR03CA0169.namprd03.prod.outlook.com (2603:10b6:408:f4::24) by PH7PR12MB7915.namprd12.prod.outlook.com (2603:10b6:510:27c::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.33; Tue, 13 Jun 2023 16:59:22 +0000 Received: from BN8NAM11FT063.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f4:cafe::ea) by BN9PR03CA0169.outlook.office365.com (2603:10b6:408:f4::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.35 via Frontend Transport; Tue, 13 Jun 2023 16:59:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN8NAM11FT063.mail.protection.outlook.com (10.13.177.110) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6500.23 via Frontend Transport; Tue, 13 Jun 2023 16:59:22 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Tue, 13 Jun 2023 09:59:03 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Tue, 13 Jun 2023 09:59:02 -0700 From: Viacheslav Ovsiienko To: Subject: [PATCH v2 1/5] app/testpmd: add trace save command Date: Tue, 13 Jun 2023 19:58:41 +0300 Message-ID: <20230613165845.19109-2-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20230613165845.19109-1-viacheslavo@nvidia.com> References: <20230420100803.494-1-viacheslavo@nvidia.com> <20230613165845.19109-1-viacheslavo@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT063:EE_|PH7PR12MB7915:EE_ X-MS-Office365-Filtering-Correlation-Id: d32ea677-75dc-469a-8441-08db6c2f894d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TPJ0lBVQwVNRZ7Ml/GCjpFfXN7SsIcprDUqoAqeSodD27o8MfUMJsbPzsEBLAJjx7dg6dfPpUEFzHzjhMxxZYOW/uEqC2/yrWGRHFuoe5fypPYRTUwWfJjOCfBVqFEZvNkOlCLjyFT/EPoVKZdqg99tsFllVMEDSTdKr3UeFCfnPnOTcca9UzT2YYmd+3GTCW2D70lluqxi1ltyA43+iiF4ikcaUNH1rhkA3mwDOnmjYvhVAfBDsx1C6CIO9li1p+7CvwbERrn34DjBXc0cWfavnSoCJlLtSLA9ZYnbLE2baHbi6j+NSVQQqhDVzxI2f/5uksk4McGGgCQha5Iyiumn7/xmRLQiF4CW6xYYzeOhNs8yc/x+MmzLHscsMvShCPE9a7uSiD0zOGCaIXak2CqlhjxJN7mxJIr2oMzypnkLFxYJsYF2730t/9QVUhbWu4uxtN85hgMgXxpdLqzmd6pQQi36PPszv5TIpbIpxdU5PewXjo1tUZgJg+lrDFYUepu5Etyr7cm5U2HzLIHsfe8jtnw+jlq0QenqbpcjUrAXVQFZRG8LZkzq5BX6ck9AVxjnSI/m968g6z9RnPxmEW9JifamLQ1Zl+N92nS9e8+yHCsDE4ne9FR5j15YRbTrju8TYIBa0OZlfmiPF+p7Moj5B5I7Q/X9Z8dSGMeO/YXTme3HtZ0IvZ0sMYQtF+3DPKh91gyRZnvYpe6jv/bkxoy1t4qeM/JwG6rnFcJqLWyXTVlctlZvmRI7lahzRTyif X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(346002)(39860400002)(376002)(136003)(396003)(451199021)(36840700001)(46966006)(40470700004)(356005)(8676002)(7636003)(40460700003)(6916009)(316002)(8936002)(86362001)(36756003)(41300700001)(5660300002)(82310400005)(40480700001)(55016003)(2906002)(70586007)(2616005)(6666004)(26005)(70206006)(336012)(426003)(1076003)(7696005)(16526019)(186003)(47076005)(6286002)(36860700001)(82740400003)(478600001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jun 2023 16:59:22.2285 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d32ea677-75dc-469a-8441-08db6c2f894d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT063.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7915 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The "save_trace" CLI command is added to trigger saving the trace dumps to the trace directory. Signed-off-by: Viacheslav Ovsiienko --- app/test-pmd/cmdline.c | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index a15a442a06..db71ce2028 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -39,6 +39,7 @@ #include #endif #include +#include #include #include @@ -12745,6 +12746,40 @@ static cmdline_parse_inst_t cmd_config_tx_affinity_map = { }, }; +#ifndef RTE_EXEC_ENV_WINDOWS +/* *** SAVE_TRACE *** */ + +struct cmd_save_trace_result { + cmdline_fixed_string_t save; +}; + +static void cmd_save_trace_parsed(__rte_unused void *parsed_result, + __rte_unused struct cmdline *cl, + __rte_unused void *data) +{ + int rc; + + rc = rte_trace_save(); + if (rc) + printf("Save trace failed with error: %d\n", rc); + else + printf("Trace saved successfully\n"); +} + +static cmdline_parse_token_string_t cmd_save_trace_save = + TOKEN_STRING_INITIALIZER(struct cmd_save_trace_result, save, "save_trace"); + +static cmdline_parse_inst_t cmd_save_trace = { + .f = cmd_save_trace_parsed, + .data = NULL, + .help_str = "save_trace: save tracing buffer", + .tokens = { + (void *)&cmd_save_trace_save, + NULL, + }, +}; +#endif + /* ******************************************************************************** */ /* list of instructions */ @@ -12979,6 +13014,9 @@ static cmdline_parse_ctx_t builtin_ctx[] = { (cmdline_parse_inst_t *)&cmd_show_port_cman_config, (cmdline_parse_inst_t *)&cmd_set_port_cman_config, (cmdline_parse_inst_t *)&cmd_config_tx_affinity_map, +#ifndef RTE_EXEC_ENV_WINDOWS + (cmdline_parse_inst_t *)&cmd_save_trace, +#endif NULL, }; From patchwork Tue Jun 13 16:58:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 128635 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 98B9F42CA7; Tue, 13 Jun 2023 18:59:25 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 49D4241611; Tue, 13 Jun 2023 18:59:21 +0200 (CEST) Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2077.outbound.protection.outlook.com [40.107.101.77]) by mails.dpdk.org (Postfix) with ESMTP id 19E3440A8A for ; Tue, 13 Jun 2023 18:59:20 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=i/jmHSBo3DPNk/bOxzQa31UB0JFKke4ETofhfLCRXKYXeFphjsnEGxxwmZ7/yGT+2DfNtefJDGPZ3CVietagSACYB3DFb2k+SyHujquboA7wadUXOzT3u4kgd2HDOgADo7nX5/e5nKK3wLwE3AVxt51fR33LXiqvQPD8RwKJsGaGHiqStkU0KHlfym+rPiehVFbMm5VoGsDqYfyCr3Busg4Z+zSsRxuu3p26uvYRmxxHyQFW/+tw72/oUh/8hQQtOlmVJC0hFwCjrc4HjkqAr06aV6SoD3lG05AYwm/phjhseQcqQGOZO30sK53nnrDtHa2968l7NMXuMwzu8Evi7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KkiRht1cQyEH0p3tHocnTu0n32/D32DfJu4k0Rs4m+s=; b=BehCa3S4ky+uhwcrfFC1sZfSyozSoFqG0WXw2ZzPuAPi9GLBDD+sIHu5xN26N0GRcJzsVNQVHKoVq6o/8ZEXkDU28UGn3FjcGRQqOPzUo75ax67D77v1XfeYq0pINU/+NSUru0Jc21pHmNM0JIJNAuIsZQoytS0SYLrWB8RfOaL5ZByU9HEt0JUgXdQx0gx3iOd3/e7AlMTRluNb1aWN+8QxNRaWK0cr8IkAF0ruw7uTbUjkNVzdTwDXfxHP3i2R47kqTt7cx1aXfVsDk9O6F9uR5/H52j6tzN/OC3hRyKIOHzcBC5CNeoSCs4GIWE/t57m67wYGgfpojIgirbK0mw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KkiRht1cQyEH0p3tHocnTu0n32/D32DfJu4k0Rs4m+s=; b=PTzPZDF2Kkz4Kiz2JReNOt4Ultohb9ZC2jZAB+8nl6/rW4QLCU1M7RWW6T/y73NquwK4bZYUl+/spKWLXiIsKNCwrIqJOGqIFBOlJtNwJmTxR7U4e7KXlPSmMsrUHvUkd548xcBFuWgNENZ5SCPNVqK2yrJOw6KbmD3GqZWs6Ki4vhw5AJKEy1kBhTJ4JHP4Wn1lf/h5QeSSvfZ+Vq3ej24K/HEeA6aFgowPiHWd6UNH3+ooksump4ai5WeOxFY13Q2+66UdIAP9GbdH+A8SS/FNpXchlc5JOOuZEtBcjT07DvKpGhzkS6ViGmCiZXto+CwQS3BgztDY2S0szz6QbA== Received: from DS7PR03CA0178.namprd03.prod.outlook.com (2603:10b6:5:3b2::33) by CH3PR12MB8484.namprd12.prod.outlook.com (2603:10b6:610:158::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.29; Tue, 13 Jun 2023 16:59:18 +0000 Received: from DM6NAM11FT075.eop-nam11.prod.protection.outlook.com (2603:10b6:5:3b2:cafe::79) by DS7PR03CA0178.outlook.office365.com (2603:10b6:5:3b2::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.35 via Frontend Transport; Tue, 13 Jun 2023 16:59:18 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DM6NAM11FT075.mail.protection.outlook.com (10.13.173.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.23 via Frontend Transport; Tue, 13 Jun 2023 16:59:17 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Tue, 13 Jun 2023 09:59:05 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Tue, 13 Jun 2023 09:59:04 -0700 From: Viacheslav Ovsiienko To: Subject: [PATCH v2 2/5] common/mlx5: introduce tracepoints for mlx5 drivers Date: Tue, 13 Jun 2023 19:58:42 +0300 Message-ID: <20230613165845.19109-3-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20230613165845.19109-1-viacheslavo@nvidia.com> References: <20230420100803.494-1-viacheslavo@nvidia.com> <20230613165845.19109-1-viacheslavo@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT075:EE_|CH3PR12MB8484:EE_ X-MS-Office365-Filtering-Correlation-Id: aae1019d-554c-401c-0b17-08db6c2f869b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mHA/z4ai3u5wXFrlLxGvNdKTSwpQy3PNUViej4T60Wthr2ZTxA4QmRsVH8jcRJTUnkfBI971zjZTsMBawxSiYZo+pomf4ksLzY8x+9iAmWXnHGDfG92pP2bFHJlwSzryblFo50/E6IuHJKng22ZNVAqh9K/dk+1Q2uzjQmRgvCo1u78BEKnEcGM78/ovxVIzoV2uSEIkkuyzAUPfrwk7DVWlz8j5zXH4UkPV3HNWcnmX1PO66l7+8ySESdXXkICyyn6fHaLs2Mcx229vwFjxnK9I15W6+TC1YbPdVgz05KjlMBdi/laeXHjTBZUTUf88MFMO0dLu4qAL2QUA0j3zmgNbPOcktCF/MmUE2xLkoND9Y4o2lm+44OreKD8qH4TL2wxv8gG/3hKNWiwr6tYJA+TPmg6WiiBTu4J3n2Ls3C4qff03lHjEZiq+05pmBE4j0R+gWMYzoKbXts3Lb6iGw8u0MWCkATKjJz6hhzzqKpKBsmrlmUII3ZPcZ95EomZKNhXTK42Ld+va4QUyhKOLGJMqa68GCFL/O9Ew+bkyakS5KEkc/lAmFqwBa6fbBZK3mpglhwgp3m/d3Nn/xwB5mvG/TyNiXQr4tVAmR4AB7sJVSvusD040mE56FxmXT/MCATaL9VETKYacuqZl5FRYnjb7/ydibPQh/BGq569LpADGfbNZit0A+x1lEicx8dr769x2KDG6r0JLYZKIcuZZ6FfasDYJ104koftD5mxvC3c= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(39860400002)(136003)(376002)(396003)(346002)(451199021)(46966006)(40470700004)(36840700001)(426003)(336012)(2616005)(47076005)(36860700001)(7636003)(82740400003)(356005)(86362001)(40480700001)(55016003)(82310400005)(36756003)(40460700003)(478600001)(7696005)(6666004)(8936002)(8676002)(2906002)(5660300002)(70206006)(70586007)(6916009)(316002)(186003)(16526019)(6286002)(41300700001)(26005)(1076003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jun 2023 16:59:17.9065 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: aae1019d-554c-401c-0b17-08db6c2f869b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT075.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8484 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org There is an intention to engage DPDK tracing capabilities for mlx5 PMDs monitoring and profiling in various modes. The patch introduces tracepoints for the Tx datapath in the ethernet device driver. Signed-off-by: Viacheslav Ovsiienko --- drivers/common/mlx5/meson.build | 1 + drivers/common/mlx5/mlx5_trace.c | 25 +++++++++++ drivers/common/mlx5/mlx5_trace.h | 72 ++++++++++++++++++++++++++++++++ drivers/common/mlx5/version.map | 8 ++++ 4 files changed, 106 insertions(+) create mode 100644 drivers/common/mlx5/mlx5_trace.c create mode 100644 drivers/common/mlx5/mlx5_trace.h diff --git a/drivers/common/mlx5/meson.build b/drivers/common/mlx5/meson.build index 1eefc02f06..28bfbfa324 100644 --- a/drivers/common/mlx5/meson.build +++ b/drivers/common/mlx5/meson.build @@ -19,6 +19,7 @@ sources += files( 'mlx5_common_mp.c', 'mlx5_common_mr.c', 'mlx5_malloc.c', + 'mlx5_trace.c', 'mlx5_common_pci.c', 'mlx5_common_devx.c', 'mlx5_common_utils.c', diff --git a/drivers/common/mlx5/mlx5_trace.c b/drivers/common/mlx5/mlx5_trace.c new file mode 100644 index 0000000000..b9f14413ad --- /dev/null +++ b/drivers/common/mlx5/mlx5_trace.c @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2022 NVIDIA Corporation & Affiliates + */ + +#include +#include + +RTE_TRACE_POINT_REGISTER(rte_pmd_mlx5_trace_tx_entry, + pmd.net.mlx5.tx.entry) + +RTE_TRACE_POINT_REGISTER(rte_pmd_mlx5_trace_tx_exit, + pmd.net.mlx5.tx.exit) + +RTE_TRACE_POINT_REGISTER(rte_pmd_mlx5_trace_tx_wqe, + pmd.net.mlx5.tx.wqe) + +RTE_TRACE_POINT_REGISTER(rte_pmd_mlx5_trace_tx_wait, + pmd.net.mlx5.tx.wait) + +RTE_TRACE_POINT_REGISTER(rte_pmd_mlx5_trace_tx_push, + pmd.net.mlx5.tx.push) + +RTE_TRACE_POINT_REGISTER(rte_pmd_mlx5_trace_tx_complete, + pmd.net.mlx5.tx.complete) + diff --git a/drivers/common/mlx5/mlx5_trace.h b/drivers/common/mlx5/mlx5_trace.h new file mode 100644 index 0000000000..57512e654f --- /dev/null +++ b/drivers/common/mlx5/mlx5_trace.h @@ -0,0 +1,72 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2022 NVIDIA Corporation & Affiliates + */ + +#ifndef RTE_PMD_MLX5_TRACE_H_ +#define RTE_PMD_MLX5_TRACE_H_ + +/** + * @file + * + * API for mlx5 PMD trace support + */ + +#ifdef __cplusplus +extern "C" { +#endif + +#include +#include +#include + +RTE_TRACE_POINT_FP( + rte_pmd_mlx5_trace_tx_entry, + RTE_TRACE_POINT_ARGS(uint16_t port_id, uint16_t queue_id), + rte_trace_point_emit_u16(port_id); + rte_trace_point_emit_u16(queue_id); +) + +RTE_TRACE_POINT_FP( + rte_pmd_mlx5_trace_tx_exit, + RTE_TRACE_POINT_ARGS(uint16_t nb_sent, uint16_t nb_req), + rte_trace_point_emit_u16(nb_sent); + rte_trace_point_emit_u16(nb_req); +) + +RTE_TRACE_POINT_FP( + rte_pmd_mlx5_trace_tx_wqe, + RTE_TRACE_POINT_ARGS(uint32_t opcode), + rte_trace_point_emit_u32(opcode); +) + +RTE_TRACE_POINT_FP( + rte_pmd_mlx5_trace_tx_wait, + RTE_TRACE_POINT_ARGS(uint64_t ts), + rte_trace_point_emit_u64(ts); +) + + +RTE_TRACE_POINT_FP( + rte_pmd_mlx5_trace_tx_push, + RTE_TRACE_POINT_ARGS(const struct rte_mbuf *mbuf, uint16_t wqe_id), + rte_trace_point_emit_ptr(mbuf); + rte_trace_point_emit_u32(mbuf->pkt_len); + rte_trace_point_emit_u16(mbuf->nb_segs); + rte_trace_point_emit_u16(wqe_id); +) + +RTE_TRACE_POINT_FP( + rte_pmd_mlx5_trace_tx_complete, + RTE_TRACE_POINT_ARGS(uint16_t port_id, uint16_t queue_id, + uint16_t wqe_id, uint64_t ts), + rte_trace_point_emit_u16(port_id); + rte_trace_point_emit_u16(queue_id); + rte_trace_point_emit_u64(ts); + rte_trace_point_emit_u16(wqe_id); +) + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PMD_MLX5_TRACE_H_ */ diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map index e05e1aa8c5..d0ec8571e6 100644 --- a/drivers/common/mlx5/version.map +++ b/drivers/common/mlx5/version.map @@ -158,5 +158,13 @@ INTERNAL { mlx5_os_interrupt_handler_create; # WINDOWS_NO_EXPORT mlx5_os_interrupt_handler_destroy; # WINDOWS_NO_EXPORT + + __rte_pmd_mlx5_trace_tx_entry; + __rte_pmd_mlx5_trace_tx_exit; + __rte_pmd_mlx5_trace_tx_wqe; + __rte_pmd_mlx5_trace_tx_wait; + __rte_pmd_mlx5_trace_tx_push; + __rte_pmd_mlx5_trace_tx_complete; + local: *; }; From patchwork Tue Jun 13 16:58:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 128640 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2184842CA7; Tue, 13 Jun 2023 19:00:05 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8E2D642D42; Tue, 13 Jun 2023 18:59:29 +0200 (CEST) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2085.outbound.protection.outlook.com [40.107.237.85]) by mails.dpdk.org (Postfix) with ESMTP id 5E7FD42D2C for ; Tue, 13 Jun 2023 18:59:27 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cDQoI9lUK4i9A6cNN5t18HFEB8er/OcLLZ/uBL6p0FIdCJc5O2WiNPwoqoNXfEN8PeRMuU58/I5SnKBGz09s0Iq927q82gUgTRm0fevQG9cOpMzvth+1kjdImuWLqcy/005Kx/FDqsb6J0y4VRVKT/RPNs+k0u6YqlQ+JUbf5A5t5U2PQn7nqnRnszdqWSPMqGW029WIf2jQC8jA7f5e1+19cR4/CNpH8syR5uj36RfubeGJ3TEwWPkmvvKoVPxj52QQGdj3PX+26UlO4DfaqG1hf3aFz2xzm5RvbY9Bgm9+NoUH9w+L6rNPcJD20GRb1T/2e5EiC20kXJCRsb7SPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CTMbpJCZgFOGxkQemJU1R0Xaa5rsz6XaY8JH3nRkNPY=; b=XPu895MjVpH5jZn5Kd/zEY2GQ9KA4ffR5oms1YnQmw0aOnRhHdbPk9AbuhyW0Ime3T3FmcyQAIXH0XjQ9PIPhFSAe47RqmLijqzPt5ZyOxIM3QHkUTq6I5dD0CVS5waVwGLX/GoaMdB1dl7gOWPaCJYnXHXMfsxAiHbfmGLz0aN1iYnCAviTPNPRYalLHIkyCBAU9q9WQvp+iYqLu/3TAB8g/md35MfVmYiDApsGYQL9aRltnOpSxGAyeyVXvxRp9M7xaunVuWyo7+kRZVtzmYlTvs51wsooT6MCXy5gj2ay1JzXt0SA/ngXJKoFHNbK0LDCWZhJym8IikviQLiRUA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CTMbpJCZgFOGxkQemJU1R0Xaa5rsz6XaY8JH3nRkNPY=; b=b6fUPM6BpoNvmq4wSDE3qbnqDNaUQlhqC0XoRWMOkLXSUtoN223mE+SrPmE6JCUd9TwEcSkyP6iEq1gBzdltq2Etvgnl76zN77NKphs1f09fmM5Mg7pBdDHt+dv9d+xe2YMKRLg6z+Zd/bYOhxf5A833dhDWtYmdGakHuf8qCYDJovSegfNDpFEitALKX9+xCTEtVqjoc2RVupikzuo/DbHmpxHqm8Ys29M7VrB/xF/cZOh8bLz2hmqKX/cpzAlj8lqoXug9VL68qvskRnwdMVivtLx4+7PkGpcxTXljl+Iq7/gymt45vpXO4hxsOdWAtohU0DKeChm/Xqz8Gry3vQ== Received: from BN7PR06CA0041.namprd06.prod.outlook.com (2603:10b6:408:34::18) by IA1PR12MB8538.namprd12.prod.outlook.com (2603:10b6:208:455::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.42; Tue, 13 Jun 2023 16:59:25 +0000 Received: from BN8NAM11FT103.eop-nam11.prod.protection.outlook.com (2603:10b6:408:34:cafe::44) by BN7PR06CA0041.outlook.office365.com (2603:10b6:408:34::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.35 via Frontend Transport; Tue, 13 Jun 2023 16:59:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN8NAM11FT103.mail.protection.outlook.com (10.13.176.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6500.23 via Frontend Transport; Tue, 13 Jun 2023 16:59:25 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Tue, 13 Jun 2023 09:59:06 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Tue, 13 Jun 2023 09:59:05 -0700 From: Viacheslav Ovsiienko To: Subject: [PATCH v2 3/5] net/mlx5: add Tx datapath tracing Date: Tue, 13 Jun 2023 19:58:43 +0300 Message-ID: <20230613165845.19109-4-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20230613165845.19109-1-viacheslavo@nvidia.com> References: <20230420100803.494-1-viacheslavo@nvidia.com> <20230613165845.19109-1-viacheslavo@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT103:EE_|IA1PR12MB8538:EE_ X-MS-Office365-Filtering-Correlation-Id: 4af0c7f2-f344-4239-b4e0-08db6c2f8b14 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: y8BUsSWpWO/ZzbHnnRjkJn+Sv8m1armdRcopvj7ZgyaAw0uWiRlYozxQI4yfXl8IUMpr6rhmF81MVCEQCkjX179mIANVkdkUcRl7jn7mi9wwsWI+jum2n6vvlMbi2lT7aIXyJ7FJEfj4fq8/xaHkVrTGMpj8XWkDVTjXFrWhCdLBo+JyziZxVw+ByXO13wVYGbaODZgOVV1R3LWU6Y99oYMFH/yVnPXfZuGSg7n7r9TzRZ8+z9E/tNGk7wwkmQ5fI3TLF+w3LP+qcXgbfG0gYkld43M8S+OuDRZv/8zX1ZCMKHbyQbPyfKbOUWbajEMcoO311TC6qfOb84MTCumnHtoICVSV8MK4yMqcwcz012wiIIsl3SPc/O32YaQZtc3FSnDj2+PMmTtAp0fXCijWwhJ4U/I3IoRBMka/y3qoAJbDbYabCc24oRIN/yVclXWhpDSRohnIpRuA2vmHweNj6D098a2t/aRkQh8PW4oHfT6omuRU4Rsyb7DAY9dV6YaKBKlfDZlrl61OFrHWXW/wVSehAJfV6K9DcfVDYNX6NQkxyNvFXtYAvMZjs8WRA0pOm8L8Ebk3ubtd94kA5iMG0+EgG7+A5ZQz53ZrzA6tolkvoWU1cxkLthZVKBv/pW7H8dn0zu1saj+3TYT6X5gxuTsx5kGvU+O3CG/yQRTcQEJk97OewH+7TvByNqiWrlvd5xMKai+uxeMzoDWBcKDlXGeV/4SeQCe1RMgwKGV07BU7cef9hKAIZvUPgtgfCRKQ X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(136003)(39860400002)(346002)(376002)(396003)(451199021)(40470700004)(46966006)(36840700001)(40480700001)(55016003)(40460700003)(7696005)(478600001)(82740400003)(7636003)(356005)(316002)(6916009)(8936002)(8676002)(70206006)(41300700001)(2616005)(70586007)(36860700001)(186003)(16526019)(6286002)(83380400001)(47076005)(6666004)(426003)(26005)(336012)(1076003)(5660300002)(82310400005)(86362001)(2906002)(36756003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jun 2023 16:59:25.2585 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4af0c7f2-f344-4239-b4e0-08db6c2f8b14 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT103.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8538 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The patch adds tracing capability to Tx datapath. To engage this tracing capability the following steps should be taken: - meson option -Denable_trace_fp=true - meson option -Dc_args='-DALLOW_EXPERIMENTAL_API' - EAL command line parameter --trace=pmd.net.mlx5.tx.* The Tx datapath tracing allows to get information how packets are pushed into hardware descriptors, time stamping for scheduled wait and send completions, etc. To provide the human readable form of trace results the dedicated post-processing script is presumed. Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5_rx.h | 19 ------------------- drivers/net/mlx5/mlx5_rxtx.h | 19 +++++++++++++++++++ drivers/net/mlx5/mlx5_tx.c | 9 +++++++++ drivers/net/mlx5/mlx5_tx.h | 25 +++++++++++++++++++++++-- 4 files changed, 51 insertions(+), 21 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rx.h b/drivers/net/mlx5/mlx5_rx.h index 52c35c83f8..ed912ffb99 100644 --- a/drivers/net/mlx5/mlx5_rx.h +++ b/drivers/net/mlx5/mlx5_rx.h @@ -376,25 +376,6 @@ mlx5_rx_mb2mr(struct mlx5_rxq_data *rxq, struct rte_mbuf *mb) return mlx5_mr_mempool2mr_bh(mr_ctrl, mb->pool, addr); } -/** - * Convert timestamp from HW format to linear counter - * from Packet Pacing Clock Queue CQE timestamp format. - * - * @param sh - * Pointer to the device shared context. Might be needed - * to convert according current device configuration. - * @param ts - * Timestamp from CQE to convert. - * @return - * UTC in nanoseconds - */ -static __rte_always_inline uint64_t -mlx5_txpp_convert_rx_ts(struct mlx5_dev_ctx_shared *sh, uint64_t ts) -{ - RTE_SET_USED(sh); - return (ts & UINT32_MAX) + (ts >> 32) * NS_PER_S; -} - /** * Set timestamp in mbuf dynamic field. * diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h index 876aa14ae6..b109d50758 100644 --- a/drivers/net/mlx5/mlx5_rxtx.h +++ b/drivers/net/mlx5/mlx5_rxtx.h @@ -43,4 +43,23 @@ int mlx5_queue_state_modify_primary(struct rte_eth_dev *dev, int mlx5_queue_state_modify(struct rte_eth_dev *dev, struct mlx5_mp_arg_queue_state_modify *sm); +/** + * Convert timestamp from HW format to linear counter + * from Packet Pacing Clock Queue CQE timestamp format. + * + * @param sh + * Pointer to the device shared context. Might be needed + * to convert according current device configuration. + * @param ts + * Timestamp from CQE to convert. + * @return + * UTC in nanoseconds + */ +static __rte_always_inline uint64_t +mlx5_txpp_convert_rx_ts(struct mlx5_dev_ctx_shared *sh, uint64_t ts) +{ + RTE_SET_USED(sh); + return (ts & UINT32_MAX) + (ts >> 32) * NS_PER_S; +} + #endif /* RTE_PMD_MLX5_RXTX_H_ */ diff --git a/drivers/net/mlx5/mlx5_tx.c b/drivers/net/mlx5/mlx5_tx.c index 14e1487e59..1fe9521dfc 100644 --- a/drivers/net/mlx5/mlx5_tx.c +++ b/drivers/net/mlx5/mlx5_tx.c @@ -232,6 +232,15 @@ mlx5_tx_handle_completion(struct mlx5_txq_data *__rte_restrict txq, MLX5_ASSERT((txq->fcqs[txq->cq_ci & txq->cqe_m] >> 16) == cqe->wqe_counter); #endif + if (__rte_trace_point_fp_is_enabled()) { + uint64_t ts = rte_be_to_cpu_64(cqe->timestamp); + uint16_t wqe_id = rte_be_to_cpu_16(cqe->wqe_counter); + + if (txq->rt_timestamp) + ts = mlx5_txpp_convert_rx_ts(NULL, ts); + rte_pmd_mlx5_trace_tx_complete(txq->port_id, txq->idx, + wqe_id, ts); + } ring_doorbell = true; ++txq->cq_ci; last_cqe = cqe; diff --git a/drivers/net/mlx5/mlx5_tx.h b/drivers/net/mlx5/mlx5_tx.h index cc8f7e98aa..7f624de58e 100644 --- a/drivers/net/mlx5/mlx5_tx.h +++ b/drivers/net/mlx5/mlx5_tx.h @@ -19,6 +19,8 @@ #include "mlx5.h" #include "mlx5_autoconf.h" +#include "mlx5_trace.h" +#include "mlx5_rxtx.h" /* TX burst subroutines return codes. */ enum mlx5_txcmp_code { @@ -764,6 +766,9 @@ mlx5_tx_cseg_init(struct mlx5_txq_data *__rte_restrict txq, cs->flags = RTE_BE32(MLX5_COMP_ONLY_FIRST_ERR << MLX5_COMP_MODE_OFFSET); cs->misc = RTE_BE32(0); + if (__rte_trace_point_fp_is_enabled() && !loc->pkts_sent) + rte_pmd_mlx5_trace_tx_entry(txq->port_id, txq->idx); + rte_pmd_mlx5_trace_tx_wqe((txq->wqe_ci << 8) | opcode); } /** @@ -1692,6 +1697,7 @@ mlx5_tx_schedule_send(struct mlx5_txq_data *restrict txq, if (txq->wait_on_time) { /* The wait on time capability should be used. */ ts -= sh->txpp.skew; + rte_pmd_mlx5_trace_tx_wait(ts); mlx5_tx_cseg_init(txq, loc, wqe, 1 + sizeof(struct mlx5_wqe_wseg) / MLX5_WSEG_SIZE, @@ -1706,6 +1712,7 @@ mlx5_tx_schedule_send(struct mlx5_txq_data *restrict txq, if (unlikely(wci < 0)) return MLX5_TXCMP_CODE_SINGLE; /* Build the WAIT WQE with specified completion. */ + rte_pmd_mlx5_trace_tx_wait(ts - sh->txpp.skew); mlx5_tx_cseg_init(txq, loc, wqe, 1 + sizeof(struct mlx5_wqe_qseg) / MLX5_WSEG_SIZE, @@ -1810,6 +1817,7 @@ mlx5_tx_packet_multi_tso(struct mlx5_txq_data *__rte_restrict txq, wqe = txq->wqes + (txq->wqe_ci & txq->wqe_m); loc->wqe_last = wqe; mlx5_tx_cseg_init(txq, loc, wqe, 0, MLX5_OPCODE_TSO, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); ds = mlx5_tx_mseg_build(txq, loc, wqe, vlan, inlen, 1, olx); wqe->cseg.sq_ds = rte_cpu_to_be_32(txq->qp_num_8s | ds); txq->wqe_ci += (ds + 3) / 4; @@ -1892,6 +1900,7 @@ mlx5_tx_packet_multi_send(struct mlx5_txq_data *__rte_restrict txq, wqe = txq->wqes + (txq->wqe_ci & txq->wqe_m); loc->wqe_last = wqe; mlx5_tx_cseg_init(txq, loc, wqe, ds, MLX5_OPCODE_SEND, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); mlx5_tx_eseg_none(txq, loc, wqe, olx); dseg = &wqe->dseg[0]; do { @@ -2115,6 +2124,7 @@ mlx5_tx_packet_multi_inline(struct mlx5_txq_data *__rte_restrict txq, wqe = txq->wqes + (txq->wqe_ci & txq->wqe_m); loc->wqe_last = wqe; mlx5_tx_cseg_init(txq, loc, wqe, 0, MLX5_OPCODE_SEND, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); ds = mlx5_tx_mseg_build(txq, loc, wqe, vlan, inlen, 0, olx); wqe->cseg.sq_ds = rte_cpu_to_be_32(txq->qp_num_8s | ds); txq->wqe_ci += (ds + 3) / 4; @@ -2318,8 +2328,8 @@ mlx5_tx_burst_tso(struct mlx5_txq_data *__rte_restrict txq, */ wqe = txq->wqes + (txq->wqe_ci & txq->wqe_m); loc->wqe_last = wqe; - mlx5_tx_cseg_init(txq, loc, wqe, ds, - MLX5_OPCODE_TSO, olx); + mlx5_tx_cseg_init(txq, loc, wqe, ds, MLX5_OPCODE_TSO, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); dseg = mlx5_tx_eseg_data(txq, loc, wqe, vlan, hlen, 1, olx); dptr = rte_pktmbuf_mtod(loc->mbuf, uint8_t *) + hlen - vlan; dlen -= hlen - vlan; @@ -2688,6 +2698,7 @@ mlx5_tx_burst_empw_simple(struct mlx5_txq_data *__rte_restrict txq, /* Update sent data bytes counter. */ slen += dlen; #endif + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); mlx5_tx_dseg_ptr (txq, loc, dseg, rte_pktmbuf_mtod(loc->mbuf, uint8_t *), @@ -2926,6 +2937,7 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *__rte_restrict txq, tlen += sizeof(struct rte_vlan_hdr); if (room < tlen) break; + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); dseg = mlx5_tx_dseg_vlan(txq, loc, dseg, dptr, dlen, olx); #ifdef MLX5_PMD_SOFT_COUNTERS @@ -2935,6 +2947,7 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *__rte_restrict txq, } else { if (room < tlen) break; + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); dseg = mlx5_tx_dseg_empw(txq, loc, dseg, dptr, dlen, olx); } @@ -2980,6 +2993,7 @@ mlx5_tx_burst_empw_inline(struct mlx5_txq_data *__rte_restrict txq, if (MLX5_TXOFF_CONFIG(VLAN)) MLX5_ASSERT(!(loc->mbuf->ol_flags & RTE_MBUF_F_TX_VLAN)); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); mlx5_tx_dseg_ptr(txq, loc, dseg, dptr, dlen, olx); /* We have to store mbuf in elts.*/ txq->elts[txq->elts_head++ & txq->elts_m] = loc->mbuf; @@ -3194,6 +3208,7 @@ mlx5_tx_burst_single_send(struct mlx5_txq_data *__rte_restrict txq, loc->wqe_last = wqe; mlx5_tx_cseg_init(txq, loc, wqe, seg_n, MLX5_OPCODE_SEND, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); mlx5_tx_eseg_data(txq, loc, wqe, vlan, inlen, 0, olx); txq->wqe_ci += wqe_n; @@ -3256,6 +3271,7 @@ mlx5_tx_burst_single_send(struct mlx5_txq_data *__rte_restrict txq, loc->wqe_last = wqe; mlx5_tx_cseg_init(txq, loc, wqe, ds, MLX5_OPCODE_SEND, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); dseg = mlx5_tx_eseg_data(txq, loc, wqe, vlan, txq->inlen_mode, 0, olx); @@ -3297,6 +3313,7 @@ mlx5_tx_burst_single_send(struct mlx5_txq_data *__rte_restrict txq, loc->wqe_last = wqe; mlx5_tx_cseg_init(txq, loc, wqe, 4, MLX5_OPCODE_SEND, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); mlx5_tx_eseg_dmin(txq, loc, wqe, vlan, olx); dptr = rte_pktmbuf_mtod(loc->mbuf, uint8_t *) + MLX5_ESEG_MIN_INLINE_SIZE - vlan; @@ -3338,6 +3355,7 @@ mlx5_tx_burst_single_send(struct mlx5_txq_data *__rte_restrict txq, loc->wqe_last = wqe; mlx5_tx_cseg_init(txq, loc, wqe, 3, MLX5_OPCODE_SEND, olx); + rte_pmd_mlx5_trace_tx_push(loc->mbuf, txq->wqe_ci); mlx5_tx_eseg_none(txq, loc, wqe, olx); mlx5_tx_dseg_ptr (txq, loc, &wqe->dseg[0], @@ -3707,6 +3725,9 @@ mlx5_tx_burst_tmpl(struct mlx5_txq_data *__rte_restrict txq, #endif if (MLX5_TXOFF_CONFIG(INLINE) && loc.mbuf_free) __mlx5_tx_free_mbuf(txq, pkts, loc.mbuf_free, olx); + /* Trace productive bursts only. */ + if (__rte_trace_point_fp_is_enabled() && loc.pkts_sent) + rte_pmd_mlx5_trace_tx_exit(loc.pkts_sent, pkts_n); return loc.pkts_sent; } From patchwork Tue Jun 13 16:58:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 128637 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id CD10842CA7; Tue, 13 Jun 2023 18:59:41 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CC26542BFE; Tue, 13 Jun 2023 18:59:25 +0200 (CEST) Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2058.outbound.protection.outlook.com [40.107.101.58]) by mails.dpdk.org (Postfix) with ESMTP id 1050A42D10 for ; Tue, 13 Jun 2023 18:59:24 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=G7fD3Hbq/9xXYxRVHg1EIaLT3nDcbEv3U4pbq4Sipe8Z3fN1xO85v56EiHgMfWlf9EJphLCnd/0LnmjhoYK98wb/LVASWolFbl49t6VDFpPozslsi/7gMVIuZ5QwhjVBckFXn/vHCnDam3ilXN8jb5pWo426GC066b7SEaevThlvuctyI7ifiEQS6Ar9n1+eYLW4Qe96LBi1hbnTX8amAEuYJ5ajqN3mUN26x0gVd+lyKY+f9azQrmbmIPYK1VgghwpPcFJP1UqF97cnW5wDNQcRh5FjJJjzkSseqXwwgYzm5qVuIqYcSn7Wn1jZEhLvr0fmcC/Ud76jNa+8EE/bVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZPZy0912N+8rrr56xuslSXObdPBbtqNskbc/7siw6LA=; b=GxU29YBHN79bFpGABXjrxuumQLE1hSGDXwNgAazrN+XJ6e3tjQ2DPxMPXtCGUXoScHz3/nOYkCGvw//uC/k2YY3t9QQ/XOSm4h0nKJu5XlLdxuC0s9472Qf/6NtPxml+6mOliw1Pm1GihS57tCIBdxjMBvnUqpM2wdgszZ/uUwMpuXcy11tmv+FRC7U342M4g0KQnBSOlf97igiV7xFaZT0aDmmwh3WAEL2WyR78vAOk9Xw0dkwfVkKHfQn5Hvrzl4i4BSoYj4VzAuns1NU7MkKef8PgdwVhGkpJL1mdO4EtNFAxnXW3nQCz4+uTgO3tIBCmqWEA2TUJzd960bBoJw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZPZy0912N+8rrr56xuslSXObdPBbtqNskbc/7siw6LA=; b=qMrOvPLEchvax0qt0xJp+3pAy4S+Lkrnjqydz0WX4647XS/Ogmw7P41q4PRMadvEn2KT5nFR5tFk9OdmIojujVuDQqg3E63v9pkAf4qKVQQB+fxeYW/+NoftjJxTKy5msGetMBncjRoTpeZSB8Tt1FYGzCdqQH6k2nQr96VmgQCj3wdy9UuZ/CfHxAZdvcKdiW8s7NCj86WXRzJbpu7ZpQMnZiijHKv1JP4Reox5ilnuUye32zkUNez6d164T48wlrNIVnQgm/dDAj4eHLdpvSYlE8rhuxuQlQoNlLvMRpZF51q8i1cBsq9RM8n5IL/mwEIviX6c6ofDqsaXlyaEWQ== Received: from DS7PR05CA0078.namprd05.prod.outlook.com (2603:10b6:8:57::24) by CH0PR12MB5284.namprd12.prod.outlook.com (2603:10b6:610:d7::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.46; Tue, 13 Jun 2023 16:59:22 +0000 Received: from DM6NAM11FT103.eop-nam11.prod.protection.outlook.com (2603:10b6:8:57:cafe::d8) by DS7PR05CA0078.outlook.office365.com (2603:10b6:8:57::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6500.23 via Frontend Transport; Tue, 13 Jun 2023 16:59:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DM6NAM11FT103.mail.protection.outlook.com (10.13.172.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6500.23 via Frontend Transport; Tue, 13 Jun 2023 16:59:21 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Tue, 13 Jun 2023 09:59:07 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Tue, 13 Jun 2023 09:59:06 -0700 From: Viacheslav Ovsiienko To: Subject: [PATCH v2 4/5] net/mlx5: add comprehensive send completion trace Date: Tue, 13 Jun 2023 19:58:44 +0300 Message-ID: <20230613165845.19109-5-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20230613165845.19109-1-viacheslavo@nvidia.com> References: <20230420100803.494-1-viacheslavo@nvidia.com> <20230613165845.19109-1-viacheslavo@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT103:EE_|CH0PR12MB5284:EE_ X-MS-Office365-Filtering-Correlation-Id: 4983cfeb-0a8b-4148-f78e-08db6c2f88a6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1COEwsR0rix8MPIYhlraFO8s5mQl4vdaQ6Jhi2SHUQa5P6uNsPlWIA/8vVUrD51SgXTzht0RhnskiFj0R9iGTnKI7r7kU191+2M/Wy12emhTwlTbeZ3GoRvXURX6NXOA+HK3FmLXxSM8OD2+vE3ZwerzGm43l535Iqa0Jzf4CeZ5dZuw0Xie21s/GdUutn5xVZ8HJ2VYMLK557BRBUHwMVPNJnjCJkCkJTNe+ZaQHec/SdslmNsl+PdNT0ajCktIL1z6RZROddfYLNMXo8u+Q7UMRNkAmwSJdqAVkEDfGtLLfJO+S/lj1MtQo17sVNjgRb3uHe51bQ/cqHHNX78VZd7hf7FOKGH1IpgFP52V1wjdA1XY2d78o56dIVCQ6jKBuhHcFDuiSJ3sUZwZU+WgAW6gI2/iswAQ+KMjjKiR3AEihDj/1miw/ezblmBi646rr9mDC49VC4uA2imrw1i1d4mgldAsKQY+FMeJaSOE/+M7sxsXNIBYTJwYSplxDUIhWSCQJlkN9v7XS7tOuugE5s2pSjXKqZw6MLrgbeyGWzRtjmJif+JcGQh8jTl/3YtvQPXq1aJ36z4S0JG+0awkJxqE6vBBD73kktBocWAuDGq4Ms9D8/HCEt8mqsphKBGn2bZbqh7Jf/mqzrcpFsaaP/NgppdUnEOCyQg10qVXsUKhZNjXLtUJ1CZTf3MYCZjcDlOoFmFiMDafdOvzsRa7xlHT3GJz/gjitZr+8KvYmC/I55ef2iuWYMge5mkVs0r5 X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(376002)(346002)(136003)(396003)(39860400002)(451199021)(40470700004)(46966006)(36840700001)(40460700003)(336012)(83380400001)(40480700001)(36860700001)(426003)(55016003)(47076005)(356005)(70206006)(478600001)(70586007)(6916009)(1076003)(26005)(6666004)(2616005)(186003)(6286002)(16526019)(86362001)(7696005)(2906002)(8676002)(8936002)(5660300002)(7636003)(82740400003)(41300700001)(36756003)(316002)(82310400005); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jun 2023 16:59:21.3557 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4983cfeb-0a8b-4148-f78e-08db6c2f88a6 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT103.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB5284 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org There is the demand to trace the send completions of every WQE if time scheduling is enabled. The patch extends the size of completion queue and requests completion on every issued WQE in the send queue. As the result hardware provides CQE on each completed WQE and driver is able to fetch completion timestamp for dedicated operation. The add code is under conditional compilation RTE_ENABLE_TRACE_FP flag and does not impact the release code. Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/linux/mlx5_verbs.c | 8 +++- drivers/net/mlx5/mlx5_devx.c | 8 +++- drivers/net/mlx5/mlx5_tx.h | 63 +++++++++++++++++++++++++++-- 3 files changed, 71 insertions(+), 8 deletions(-) diff --git a/drivers/net/mlx5/linux/mlx5_verbs.c b/drivers/net/mlx5/linux/mlx5_verbs.c index 7233c2c7fa..b54f3ccd9a 100644 --- a/drivers/net/mlx5/linux/mlx5_verbs.c +++ b/drivers/net/mlx5/linux/mlx5_verbs.c @@ -968,8 +968,12 @@ mlx5_txq_ibv_obj_new(struct rte_eth_dev *dev, uint16_t idx) rte_errno = EINVAL; return -rte_errno; } - cqe_n = desc / MLX5_TX_COMP_THRESH + - 1 + MLX5_TX_COMP_THRESH_INLINE_DIV; + if (__rte_trace_point_fp_is_enabled() && + txq_data->offloads & RTE_ETH_TX_OFFLOAD_SEND_ON_TIMESTAMP) + cqe_n = UINT16_MAX / 2 - 1; + else + cqe_n = desc / MLX5_TX_COMP_THRESH + + 1 + MLX5_TX_COMP_THRESH_INLINE_DIV; txq_obj->cq = mlx5_glue->create_cq(priv->sh->cdev->ctx, cqe_n, NULL, NULL, 0); if (txq_obj->cq == NULL) { diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index 4369d2557e..5082a7e178 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -1465,8 +1465,12 @@ mlx5_txq_devx_obj_new(struct rte_eth_dev *dev, uint16_t idx) MLX5_ASSERT(ppriv); txq_obj->txq_ctrl = txq_ctrl; txq_obj->dev = dev; - cqe_n = (1UL << txq_data->elts_n) / MLX5_TX_COMP_THRESH + - 1 + MLX5_TX_COMP_THRESH_INLINE_DIV; + if (__rte_trace_point_fp_is_enabled() && + txq_data->offloads & RTE_ETH_TX_OFFLOAD_SEND_ON_TIMESTAMP) + cqe_n = UINT16_MAX / 2 - 1; + else + cqe_n = (1UL << txq_data->elts_n) / MLX5_TX_COMP_THRESH + + 1 + MLX5_TX_COMP_THRESH_INLINE_DIV; log_desc_n = log2above(cqe_n); cqe_n = 1UL << log_desc_n; if (cqe_n > UINT16_MAX) { diff --git a/drivers/net/mlx5/mlx5_tx.h b/drivers/net/mlx5/mlx5_tx.h index 7f624de58e..9f29df280f 100644 --- a/drivers/net/mlx5/mlx5_tx.h +++ b/drivers/net/mlx5/mlx5_tx.h @@ -728,6 +728,54 @@ mlx5_tx_request_completion(struct mlx5_txq_data *__rte_restrict txq, } } +/** + * Set completion request flag for all issued WQEs. + * This routine is intended to be used with enabled fast path tracing + * and send scheduling on time to provide the detailed report in trace + * for send completions on every WQE. + * + * @param txq + * Pointer to TX queue structure. + * @param loc + * Pointer to burst routine local context. + * @param olx + * Configured Tx offloads mask. It is fully defined at + * compile time and may be used for optimization. + */ +static __rte_always_inline void +mlx5_tx_request_completion_trace(struct mlx5_txq_data *__rte_restrict txq, + struct mlx5_txq_local *__rte_restrict loc, + unsigned int olx) +{ + uint16_t head = txq->elts_comp; + + while (txq->wqe_comp != txq->wqe_ci) { + volatile struct mlx5_wqe *wqe; + uint32_t wqe_n; + + MLX5_ASSERT(loc->wqe_last); + wqe = txq->wqes + (txq->wqe_comp & txq->wqe_m); + if (wqe == loc->wqe_last) { + head = txq->elts_head; + head += MLX5_TXOFF_CONFIG(INLINE) ? + 0 : loc->pkts_sent - loc->pkts_copy; + txq->elts_comp = head; + } + /* Completion request flag was set on cseg constructing. */ +#ifdef RTE_LIBRTE_MLX5_DEBUG + txq->fcqs[txq->cq_pi++ & txq->cqe_m] = head | + (wqe->cseg.opcode >> 8) << 16; +#else + txq->fcqs[txq->cq_pi++ & txq->cqe_m] = head; +#endif + /* A CQE slot must always be available. */ + MLX5_ASSERT((txq->cq_pi - txq->cq_ci) <= txq->cqe_s); + /* Advance to the next WQE in the queue. */ + wqe_n = rte_be_to_cpu_32(wqe->cseg.sq_ds) & 0x3F; + txq->wqe_comp += RTE_ALIGN(wqe_n, 4) / 4; + } +} + /** * Build the Control Segment with specified opcode: * - MLX5_OPCODE_SEND @@ -754,7 +802,7 @@ mlx5_tx_cseg_init(struct mlx5_txq_data *__rte_restrict txq, struct mlx5_wqe *__rte_restrict wqe, unsigned int ds, unsigned int opcode, - unsigned int olx __rte_unused) + unsigned int olx) { struct mlx5_wqe_cseg *__rte_restrict cs = &wqe->cseg; @@ -763,8 +811,12 @@ mlx5_tx_cseg_init(struct mlx5_txq_data *__rte_restrict txq, opcode = MLX5_OPCODE_TSO | MLX5_OPC_MOD_MPW << 24; cs->opcode = rte_cpu_to_be_32((txq->wqe_ci << 8) | opcode); cs->sq_ds = rte_cpu_to_be_32(txq->qp_num_8s | ds); - cs->flags = RTE_BE32(MLX5_COMP_ONLY_FIRST_ERR << - MLX5_COMP_MODE_OFFSET); + if (MLX5_TXOFF_CONFIG(TXPP) && __rte_trace_point_fp_is_enabled()) + cs->flags = RTE_BE32(MLX5_COMP_ALWAYS << + MLX5_COMP_MODE_OFFSET); + else + cs->flags = RTE_BE32(MLX5_COMP_ONLY_FIRST_ERR << + MLX5_COMP_MODE_OFFSET); cs->misc = RTE_BE32(0); if (__rte_trace_point_fp_is_enabled() && !loc->pkts_sent) rte_pmd_mlx5_trace_tx_entry(txq->port_id, txq->idx); @@ -3662,7 +3714,10 @@ mlx5_tx_burst_tmpl(struct mlx5_txq_data *__rte_restrict txq, if (unlikely(loc.pkts_sent == loc.pkts_loop)) goto burst_exit; /* Request CQE generation if limits are reached. */ - mlx5_tx_request_completion(txq, &loc, olx); + if (MLX5_TXOFF_CONFIG(TXPP) && __rte_trace_point_fp_is_enabled()) + mlx5_tx_request_completion_trace(txq, &loc, olx); + else + mlx5_tx_request_completion(txq, &loc, olx); /* * Ring QP doorbell immediately after WQE building completion * to improve latencies. The pure software related data treatment From patchwork Tue Jun 13 16:58:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Slava Ovsiienko X-Patchwork-Id: 128639 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9DE3842CA7; Tue, 13 Jun 2023 18:59:57 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5591842D37; Tue, 13 Jun 2023 18:59:28 +0200 (CEST) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2067.outbound.protection.outlook.com [40.107.93.67]) by mails.dpdk.org (Postfix) with ESMTP id 0851F42BB1 for ; Tue, 13 Jun 2023 18:59:25 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UieyLunvadGbtyVK513HakwLilTThrl7qdMJ3hA02uw5eCXnZ+FCbP7a6qEkNvsCZulA+UTRhqRCPB+zyVPkoilGdTQM99Nl0KTNHqZsRo4C5bzCmCCo2fpWyu3eu4jJRHrv4lyEm4BaxJ9ngDPXgYZxA0zex1x6FTthcqkX/bWuJl6XnnL0EsRIkHvG+UP1whcbkBhwarP4O3dQq50NYR3b2PoFNZBvF56Uf+9oX5/KIan81qspwDPxWisi1Q2l+tUyPwA/X97nXwe3N2CICAXwO5TlKJM8qFdzpLK4xa82dl/L5iOdDDBZVp/VGwMHbbbkgetUXPArmS8lWZxKaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MYiacIuwMsvJ4xeh+TMKQLFoa38kzzHwJ+42WXNm0ZU=; b=ias7oYhjwGWZsbTr1mVBxWkXiA4zFzBWAJ/uwGMeNyLhKyqGzaSnJOmXHtheBarxwiJQSVOtPChYp4vBm7QVrHMTH1TnY9VY9tML7WgW+gluJ4yq7LZu/aBp7TXh06zldYN3J7qxuTpFbR2tYeNmrCR3C7fJhYeZUn1uqoVhe9vCKh28O3tqMA/4vjTedC4dyJ/H7mqsoGci2aNAQcs1Osab1HtMsOqhQzgNnM3/egZjd8KN8Q9ZdGYoVUPrGyqLc6McOqNnGl+7VlXp5d4fCC+Xno11AiPcs3MLPNSEPQ4A4MWcOyuAjuL1TIHNCYwQ685jwoEvTAUhIySTNzkCwA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MYiacIuwMsvJ4xeh+TMKQLFoa38kzzHwJ+42WXNm0ZU=; b=LbdZQp+TbrPViA0T7w9IH44NnmJefuTKH1Kb+7zNglnMSRKrv1CCtng2FpWQQfUnUcs4qE8gQvOGsG0SQNHI30QNI33cYgDaZK/towewS33Rq0qqJSsEgRaUrmmKJNrsdaJ7DmyUQrsxjIuIrNvI9QdoOljb4Fs+zIk0JU0p+pa2QQENTalWdFVN9St6kTskmJY7Bnm8q3OHuqHwCWCrNezw+mgoafq8f+Li1D1VtWw4ySbLhBOENZE5SqBjhaFlx9BxFMYcIBkeNdyG3CQpUc+/JypQKWabY9wyMwLzoysi7iynoqymI7F769I3U0xYnzCtn6z7fQczA5Xwu96tgQ== Received: from DM6PR05CA0055.namprd05.prod.outlook.com (2603:10b6:5:335::24) by BL1PR12MB5094.namprd12.prod.outlook.com (2603:10b6:208:312::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6455.46; Tue, 13 Jun 2023 16:59:23 +0000 Received: from DM6NAM11FT069.eop-nam11.prod.protection.outlook.com (2603:10b6:5:335:cafe::52) by DM6PR05CA0055.outlook.office365.com (2603:10b6:5:335::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6500.23 via Frontend Transport; Tue, 13 Jun 2023 16:59:23 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DM6NAM11FT069.mail.protection.outlook.com (10.13.173.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6500.22 via Frontend Transport; Tue, 13 Jun 2023 16:59:22 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.5; Tue, 13 Jun 2023 09:59:09 -0700 Received: from nvidia.com (10.126.231.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.37; Tue, 13 Jun 2023 09:59:07 -0700 From: Viacheslav Ovsiienko To: Subject: [PATCH v2 5/5] net/mlx5: add Tx datapath trace analyzing script Date: Tue, 13 Jun 2023 19:58:45 +0300 Message-ID: <20230613165845.19109-6-viacheslavo@nvidia.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20230613165845.19109-1-viacheslavo@nvidia.com> References: <20230420100803.494-1-viacheslavo@nvidia.com> <20230613165845.19109-1-viacheslavo@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [10.126.231.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT069:EE_|BL1PR12MB5094:EE_ X-MS-Office365-Filtering-Correlation-Id: a1975ebb-bebe-42c7-05b5-08db6c2f898a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oT9L2TuTGldT7jiPhte9ZqSOdL/NTNnTEKFdnE3S92oX39S9nYfM37VpPOqunl8cRI38Li683MvOwlpDq9FTHGFpNzp5daJF63kCqsnN+VOOz3tRkfifq9grbSzkVaZvf7IxwSzjllhG1XokenFVO5rgo0Y7+CLwyyyHcVlLwhRivy00Gq72D4qIsztU0aisIojkITRDFRZNm20p7ac4fTswm5LKo4ZvsGN0ymavsqlkgZRJTI9b+A532XNVQhMiMRuM/Nt54xqEVktn0AlOxx+cIgrkrQw4K+ka1TuSYFDcmLya1wIa558ivc0wq5Hy/loQyPaa0bzu+Eq663VpCDx4o5o19Yu2dJaQd5qs1PlE2WOQtJ8WBw98d7RrKSMHPtylneo3g8HOZD0oxqsbosW4hqOMmNQvrVENyBI36XmTY+8UyPqS5n37Ad0m4TsSnHgKbH2a0ByCTR5Jy5PQ27qZW1oEvmBAcOsfB2zAGOhIgr7IP0GS970BkU+XHd8OcuKXR7TOr62sV7O3rLTJ9CZyM1e9SVMJZUDXHyJYdB28Xp+epECDz6l+ZzwJayG/lh4GQ8e21wADDJ/l2d/GQIim8uH+2icB9QShvsxPHh+EOtG8VW8zY2BclytvtrVEQNM31LQP0ISizuyv96Jb4qz9gg0DsDxZHA7lcBFdWzq0xsPTbHssaieAz/v04lJr53hSQfOSJ+Du1bFb/y8qpAmg5FA8Y6TiOuDkST6fue5IDJBnwDiYJLGjLKyWSO4T X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230028)(4636009)(39860400002)(136003)(346002)(396003)(376002)(451199021)(36840700001)(46966006)(40470700004)(40460700003)(6666004)(7696005)(478600001)(36860700001)(47076005)(1076003)(336012)(6286002)(426003)(186003)(16526019)(83380400001)(36756003)(82310400005)(86362001)(26005)(2616005)(82740400003)(356005)(7636003)(40480700001)(55016003)(6916009)(70206006)(70586007)(316002)(8936002)(5660300002)(8676002)(41300700001)(2906002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jun 2023 16:59:22.8437 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a1975ebb-bebe-42c7-05b5-08db6c2f898a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT069.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5094 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org The Python script is intended to analyze mlx5 PMD datapath traces and report: - tx_burst routine timings - how packets are pushed to WQEs - how packet sending is completed with timings Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/tools/mlx5_trace.py | 271 +++++++++++++++++++++++++++ 1 file changed, 271 insertions(+) create mode 100755 drivers/net/mlx5/tools/mlx5_trace.py diff --git a/drivers/net/mlx5/tools/mlx5_trace.py b/drivers/net/mlx5/tools/mlx5_trace.py new file mode 100755 index 0000000000..c8fa63a7b9 --- /dev/null +++ b/drivers/net/mlx5/tools/mlx5_trace.py @@ -0,0 +1,271 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: BSD-3-Clause +# Copyright (c) 2023 NVIDIA Corporation & Affiliates + +''' +Analyzing the mlx5 PMD datapath tracings +''' +import sys +import argparse +import pathlib +import bt2 + +PFX_TX = "pmd.net.mlx5.tx." +PFX_TX_LEN = len(PFX_TX) + +tx_blst = {} # current Tx bursts per CPU +tx_qlst = {} # active Tx queues per port/queue +tx_wlst = {} # wait timestamp list per CPU + +class mlx5_queue(object): + def __init__(self): + self.done_burst = [] # completed bursts + self.wait_burst = [] # waiting for completion + self.pq_id = 0 + + def log(self): + for txb in self.done_burst: + txb.log() + + +class mlx5_mbuf(object): + def __init__(self): + self.wqe = 0 # wqe id + self.ptr = None # first packet mbuf pointer + self.len = 0 # packet data length + self.nseg = 0 # number of segments + + def log(self): + out = " %X: %u" % (self.ptr, self.len) + if self.nseg != 1: + out += " (%d segs)" % self.nseg + print(out) + + +class mlx5_wqe(object): + def __init__(self): + self.mbuf = [] # list of mbufs in WQE + self.wait_ts = 0 # preceding wait/push timestamp + self.comp_ts = 0 # send/recv completion timestamp + self.opcode = 0 + + def log(self): + id = (self.opcode >> 8) & 0xFFFF + op = self.opcode & 0xFF + fl = self.opcode >> 24 + out = " %04X: " % id + if op == 0xF: + out += "WAIT" + elif op == 0x29: + out += "EMPW" + elif op == 0xE: + out += "TSO " + elif op == 0xA: + out += "SEND" + else: + out += "0x%02X" % op + if self.comp_ts != 0: + out += " (%d, %d)" % (self.wait_ts, self.comp_ts - self.wait_ts) + else: + out += " (%d)" % self.wait_ts + print(out) + for mbuf in self.mbuf: + mbuf.log() + + # return 0 if WQE in not completed + def comp(self, wqe_id, ts): + if self.comp_ts != 0: + return 1 + id = (self.opcode >> 8) & 0xFFFF + if id > wqe_id: + id -= wqe_id + if id <= 0x8000: + return 0 + else: + id = wqe_id - id + if id >= 0x8000: + return 0 + self.comp_ts = ts + return 1 + + +class mlx5_burst(object): + def __init__(self): + self.wqes = [] # issued burst WQEs + self.done = 0 # number of sent/recv packets + self.req = 0 # requested number of packets + self.call_ts = 0 # burst routine invocation + self.done_ts = 0 # burst routine done + self.queue = None + + def log(self): + port = self.queue.pq_id >> 16 + queue = self.queue.pq_id & 0xFFFF + if self.req == 0: + print("%u: tx(p=%u, q=%u, %u/%u pkts (incomplete)" % + (self.call_ts, port, queue, self.done, self.req)) + else: + print("%u: tx(p=%u, q=%u, %u/%u pkts in %u" % + (self.call_ts, port, queue, self.done, self.req, + self.done_ts - self.call_ts)) + for wqe in self.wqes: + wqe.log() + + # return 0 if not all of WQEs in burst completed + def comp(self, wqe_id, ts): + wlen = len(self.wqes) + if wlen == 0: + return 0 + for wqe in self.wqes: + if wqe.comp(wqe_id, ts) == 0: + return 0 + return 1 + + +def do_tx_entry(msg): + event = msg.event + cpu_id = event["cpu_id"] + burst = tx_blst.get(cpu_id) + if burst is not None: + # continue existing burst after WAIT + return + # allocate the new burst and append to the queue + burst = mlx5_burst() + burst.call_ts = msg.default_clock_snapshot.ns_from_origin + tx_blst[cpu_id] = burst + pq_id = event["port_id"] << 16 | event["queue_id"] + queue = tx_qlst.get(pq_id) + if queue is None: + # queue does not exist - allocate the new one + queue = mlx5_queue(); + queue.pq_id = pq_id + tx_qlst[pq_id] = queue + burst.queue = queue + queue.wait_burst.append(burst) + + +def do_tx_exit(msg): + event = msg.event + cpu_id = event["cpu_id"] + burst = tx_blst.get(cpu_id) + if burst is None: + return + burst.done_ts = msg.default_clock_snapshot.ns_from_origin + burst.req = event["nb_req"] + burst.done = event["nb_sent"] + tx_blst.pop(cpu_id) + + +def do_tx_wqe(msg): + event = msg.event + cpu_id = event["cpu_id"] + burst = tx_blst.get(cpu_id) + if burst is None: + return + wqe = mlx5_wqe() + wqe.wait_ts = tx_wlst.get(cpu_id) + if wqe.wait_ts is None: + wqe.wait_ts = msg.default_clock_snapshot.ns_from_origin + wqe.opcode = event["opcode"] + burst.wqes.append(wqe) + + +def do_tx_wait(msg): + event = msg.event + cpu_id = event["cpu_id"] + tx_wlst[cpu_id] = event["ts"] + + +def do_tx_push(msg): + event = msg.event + cpu_id = event["cpu_id"] + burst = tx_blst.get(cpu_id) + if burst is None: + return + if not burst.wqes: + return + wqe = burst.wqes[-1] + mbuf = mlx5_mbuf() + mbuf.wqe = event["wqe_id"] + mbuf.ptr = event["mbuf"] + mbuf.len = event["mbuf_pkt_len"] + mbuf.nseg = event["mbuf_nb_segs"] + wqe.mbuf.append(mbuf) + + +def do_tx_complete(msg): + event = msg.event + pq_id = event["port_id"] << 16 | event["queue_id"] + queue = tx_qlst.get(pq_id) + if queue is None: + return + qlen = len(queue.wait_burst) + if qlen == 0: + return + wqe_id = event["wqe_id"] + ts = event["ts"] + rmv = 0 + while rmv < qlen: + burst = queue.wait_burst[rmv] + if burst.comp(wqe_id, ts) == 0: + break + rmv += 1 + # mode completed burst to done list + if rmv != 0: + idx = 0 + while idx < rmv: + queue.done_burst.append(burst) + idx += 1 + del queue.wait_burst[0:rmv] + + +def do_tx(msg): + name = msg.event.name[PFX_TX_LEN:] + if name == "entry": + do_tx_entry(msg) + elif name == "exit": + do_tx_exit(msg) + elif name == "wqe": + do_tx_wqe(msg) + elif name == "wait": + do_tx_wait(msg) + elif name == "push": + do_tx_push(msg) + elif name == "complete": + do_tx_complete(msg) + else: + print("Error: unrecognized Tx event name: %s" % msg.event.name) + sys.exit(1) + + +def do_log(msg_it): + for msg in msg_it: + if type(msg) is not bt2._EventMessageConst: + continue + event = msg.event + if event.name.startswith(PFX_TX): + do_tx(msg) + # Handling of other log event cathegories can be added here + + +def do_print(): + for pq_id in tx_qlst: + queue = tx_qlst.get(pq_id) + queue.log() + + +def main(args): + parser = argparse.ArgumentParser() + parser.add_argument("path", + nargs = 1, + type = str, + help = "input trace folder") + args = parser.parse_args() + + msg_it = bt2.TraceCollectionMessageIterator(args.path) + do_log(msg_it) + do_print() + exit(0) + +if __name__ == "__main__": + main(sys.argv)