From patchwork Thu Feb 2 09:13:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 122888 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9C57941BAB; Thu, 2 Feb 2023 10:14:02 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5D53242D44; Thu, 2 Feb 2023 10:13:59 +0100 (CET) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2042.outbound.protection.outlook.com [40.107.92.42]) by mails.dpdk.org (Postfix) with ESMTP id 4204140689 for ; Thu, 2 Feb 2023 10:13:56 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=b6oLW8RBASnM91PVVFjKq82HqWEIsvUK7xbFVMz1WEUjk+C/2AIidVQisFUcSREB5ui+ZxogQVBTnT4upxfKofYmqpMOYiNO291JxiDkUxq8AgrXjwmTKlAnNeCsATlL/RIIb1AVnrgY70bX5FOpNkW3CgKPYUc88pjRnodEDDgrpXNFkG6JRA9qm+tHfEC1uePYPu0W520QP8BG58qspSsP3igvM6CuO/on8wFxFeMOX22MJh+FpvLgQNeNlhErG0ZfdJpob1T9I5tqvVPJ9iWPpJExnDh9Bv8u0E9Mrv5dtOh6BXyxmQU09x42stdWTheWRRhh0uEn5knk2poV3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CmebHCPFoD0g7iW23UtegGjIEgKkYliN82KvAgo+9XA=; b=AeM41ku3gh/WbjZ0zpwq1dE3IEa3gVreYQfISaFuDVDPqvXw7zwMD44I4/x8A1nsVBjwuLCIgGVKhp5ld/zxM7sh0BkCaPzrz/fgnHW+2tVcU/sn13EQtQ7vpb9yyD5jarSGLIvhkELhr8LYYf7UmxhctojIalb70wdFd0rHQpVRtAmiBxZhh7TK8h8A8dCi+J90E4ICW8xc1RmBokqMUHmWq/jfk7Ykj/YB871Qy3Op0jJkdtwbm7Cxlj3+4E8zUynql3FTHBsBDFnLwDpEXUyXwwM61hSPFySs8IZZc9l9G20xigy67fjV3HBuoGfR0wW7NgQwhmWliFiv2k0RlQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CmebHCPFoD0g7iW23UtegGjIEgKkYliN82KvAgo+9XA=; b=OvCOD/Gn9H1cwgLwY8NPcnPoBzM9XUPT2KfNu5swSEG+bsTbH9d3yXLjIMQFzEGyMBTSPfFX0qOMo0DxJcIR7e5VEbnwn4mHl1tSYbdC6db7f/aWTxKM7vZ+hJjKUzzh/Audx+N69ZocSwYEJRkya7WAxVRL95dll9X+bk3d2wxVyVWTGGOYt9lX5sdExyYQTc5T3NuUt7MQfoqDsDtWvk3Eclpqod/UCc3iAE3ZCmTPEDho8qen6on5bDN5MpMc7+CUB+JTkAnxMdX/3DjuJgt4/AAEAKYcA/S+HM17UHiy1FBi2i10D+ryOF+Uu43IPjFNFurIN31rr1kHQAlqGw== Received: from BN9PR03CA0264.namprd03.prod.outlook.com (2603:10b6:408:ff::29) by PH0PR12MB8049.namprd12.prod.outlook.com (2603:10b6:510:28f::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.27; Thu, 2 Feb 2023 09:13:54 +0000 Received: from BN8NAM11FT060.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ff:cafe::3c) by BN9PR03CA0264.outlook.office365.com (2603:10b6:408:ff::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.27 via Frontend Transport; Thu, 2 Feb 2023 09:13:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN8NAM11FT060.mail.protection.outlook.com (10.13.177.211) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.27 via Frontend Transport; Thu, 2 Feb 2023 09:13:53 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 2 Feb 2023 01:13:40 -0800 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 2 Feb 2023 01:13:40 -0800 Received: from nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36 via Frontend Transport; Thu, 2 Feb 2023 01:13:38 -0800 From: Michael Baum To: CC: Matan Azrad , Akhil Goyal , "Ashish Gupta" , Fiona Trahe , "Thomas Monjalon" Subject: [PATCH v2 1/4] compressdev: add LZ4 algorithm support Date: Thu, 2 Feb 2023 11:13:29 +0200 Message-ID: <20230202091332.1037078-2-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202091332.1037078-1-michaelba@nvidia.com> References: <20230109074526.2507344-1-michaelba@nvidia.com> <20230202091332.1037078-1-michaelba@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT060:EE_|PH0PR12MB8049:EE_ X-MS-Office365-Filtering-Correlation-Id: 78b8f023-94c7-4dae-a7f0-08db04fdce85 X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: QsLwG9qIWG/5tFOT/O5GMR+LPuG1juvpnNGGsSsRJEBg1CsbrRlIiXNdQpiN3rCsKEzT6ld0Fa95eiG0D4AWgNreZ+DZGWzFZpZdB3S5QAmHli8TRSljMmDnOH9nYtfkyRSQPPTKtnbsabYzhz+ZBcMCnCMyx7/QY+K49htVbtJCG4AfLpPGQz7D83/7GSgaGBtB4z0aIdTKmzimmAMgqxFl8rkZU1wcJVDDrkG0ynxtSTesn9lXpuysFP7FDv60gTkKnMjoR3ks1jT6PkQUpycwkrI50xFDFSOxXSvHj4rHegqMkFGVckMuxnX7VXSqsGuCx9+C1c3AcAP5+OYwiKL8/A7AQrbCAHmnlZf/EK7qCYTDVM8sXjjhl6TYBctELZ+9O0MwzytwlD5qn+mpkBFlIvP8MxbS7xVwxT6yb2ZRuGqX97Qis9QKKFmf0TgHWjD/LENbfc7XN6pD1/weZexno+P0GW2lkbESByP8qjBfstQ8N1OptrbbV8fcuY6wqymcSXnEotpENBJLEEekxHikHKPUr6hLwSI0HSWWsAblM+sDGeT9pD3AYqqG/L2O4DVC4AQEEXIGIoPI6uA3TdgxTnohZykx5FIvTT/8x1w0rCnFv4kkPxr8mMKpM7jRRBF2dOnMoqwOoxuG9DZW8R/yTqc5cI1FXqH6Bpxg8znWiINDVDf7NNFH0/irfjxzbqtCN3eChAC9DhVsWlDGQ+feK9o0x1NGblQkhUNliCbk19QGH50koOp0x5lNGF0a4L+TwF/PtQ0LOzz4OfADzTj0S8ImBYJa6PSksHFRcFF9esrxXD+XZKnxewGjV1vB X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230025)(4636009)(136003)(396003)(39860400002)(346002)(376002)(451199018)(46966006)(36840700001)(40470700004)(478600001)(966005)(47076005)(7696005)(6286002)(2616005)(186003)(1076003)(6666004)(83380400001)(426003)(40460700003)(336012)(26005)(5660300002)(2906002)(82310400005)(36756003)(316002)(54906003)(8676002)(55016003)(40480700001)(41300700001)(8936002)(6916009)(70206006)(70586007)(4326008)(7636003)(82740400003)(356005)(36860700001)(86362001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2023 09:13:53.9285 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 78b8f023-94c7-4dae-a7f0-08db04fdce85 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT060.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB8049 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add support for LZ4 algorithm: - Add Lz4 param structure to XFORM structures. - Add capabilities flags for LZ4 params. - Add xxHash-32 checksum and capabilities flag. Signed-off-by: Michael Baum --- doc/guides/compressdevs/features/default.ini | 7 ++ doc/guides/rel_notes/release_23_03.rst | 7 ++ lib/compressdev/rte_comp.c | 12 +++ lib/compressdev/rte_comp.h | 79 +++++++++++++++++++- 4 files changed, 103 insertions(+), 2 deletions(-) diff --git a/doc/guides/compressdevs/features/default.ini b/doc/guides/compressdevs/features/default.ini index e1419ee8db..2f178c5efd 100644 --- a/doc/guides/compressdevs/features/default.ini +++ b/doc/guides/compressdevs/features/default.ini @@ -20,8 +20,15 @@ OOP SGL In LB Out = OOP LB In SGL Out = Deflate = LZS = +LZ4 = Adler32 = Crc32 = Adler32&Crc32 = +xxHash32 = Fixed = Dynamic = +LZ4 Dictionary ID = +LZ4 Content Checksum = +LZ4 Content Size = +LZ4 Block Checksum = +LZ4 Block Independence = diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst index 73f5d94e14..2722296180 100644 --- a/doc/guides/rel_notes/release_23_03.rst +++ b/doc/guides/rel_notes/release_23_03.rst @@ -78,6 +78,13 @@ New Features ``rte_event_dev_config::nb_single_link_event_port_queues`` parameter required for eth_rx, eth_tx, crypto and timer eventdev adapters. +* **Added LZ4 algorithm in Compressdev Library.** + + Added new compression algorithm, including: + + * Added support for ``RTE_COMP_ALGO_LZ4``. + * Added support for ``RTE_COMP_CHECKSUM_XXHASH32``. + Removed Items ------------- diff --git a/lib/compressdev/rte_comp.c b/lib/compressdev/rte_comp.c index 320c6dab92..f060c68557 100644 --- a/lib/compressdev/rte_comp.c +++ b/lib/compressdev/rte_comp.c @@ -39,6 +39,18 @@ rte_comp_get_feature_name(uint64_t flag) return "HUFFMAN_FIXED"; case RTE_COMP_FF_HUFFMAN_DYNAMIC: return "HUFFMAN_DYNAMIC"; + case RTE_COMP_FF_XXHASH32_CHECKSUM: + return "XXHASH32_CHECKSUM"; + case RTE_COMP_FF_LZ4_DICT_ID: + return "LZ4_DICT_ID"; + case RTE_COMP_FF_LZ4_CONTENT_WITH_CHECKSUM: + return "LZ4_CONTENT_WITH_CHECKSUM"; + case RTE_COMP_FF_LZ4_CONTENT_SIZE: + return "LZ4_CONTENT_SIZE"; + case RTE_COMP_FF_LZ4_BLOCK_INDEPENDENCE: + return "LZ4_BLOCK_INDEPENDENCE"; + case RTE_COMP_FF_LZ4_BLOCK_WITH_CHECKSUM: + return "LZ4_BLOCK_WITH_CHECKSUM"; default: return NULL; } diff --git a/lib/compressdev/rte_comp.h b/lib/compressdev/rte_comp.h index 5bd711fda1..2096fb2407 100644 --- a/lib/compressdev/rte_comp.h +++ b/lib/compressdev/rte_comp.h @@ -67,6 +67,18 @@ extern "C" { /**< Fixed huffman encoding is supported */ #define RTE_COMP_FF_HUFFMAN_DYNAMIC (1ULL << 14) /**< Dynamic huffman encoding is supported */ +#define RTE_COMP_FF_XXHASH32_CHECKSUM (1ULL << 15) +/**< xxHash-32 Checksum is supported */ +#define RTE_COMP_FF_LZ4_DICT_ID (1ULL << 16) +/**< LZ4 dictionary ID is supported */ +#define RTE_COMP_FF_LZ4_CONTENT_WITH_CHECKSUM (1ULL << 17) +/**< LZ4 content with checksum is supported */ +#define RTE_COMP_FF_LZ4_CONTENT_SIZE (1ULL << 18) +/**< LZ4 content size is supported */ +#define RTE_COMP_FF_LZ4_BLOCK_INDEPENDENCE (1ULL << 19) +/**< LZ4 block independent is supported */ +#define RTE_COMP_FF_LZ4_BLOCK_WITH_CHECKSUM (1ULL << 20) +/**< LZ4 block with checksum is supported */ /** Status of comp operation */ enum rte_comp_op_status { @@ -109,6 +121,10 @@ enum rte_comp_algorithm { /**< LZS compression algorithm * https://tools.ietf.org/html/rfc2395 */ + RTE_COMP_ALGO_LZ4, + /**< LZ4 compression algorithm + * https://github.com/lz4/lz4 + */ }; /** Compression Hash Algorithms */ @@ -147,9 +163,12 @@ enum rte_comp_checksum_type { /**< Generates both Adler-32 and CRC32 checksums, concatenated. * CRC32 is in the lower 32bits, Adler-32 in the upper 32 bits. */ + RTE_COMP_CHECKSUM_XXHASH32, + /**< Generates a xxHash-32 checksum, as used by lz4. + * https://github.com/Cyan4973/xxHash/blob/dev/doc/xxhash_spec.md + */ }; - /** Compression Huffman Type - used by DEFLATE algorithm */ enum rte_comp_huffman { RTE_COMP_HUFFMAN_DEFAULT, @@ -206,13 +225,63 @@ enum rte_comp_op_type { */ }; - /** Parameters specific to the deflate algorithm */ struct rte_comp_deflate_params { enum rte_comp_huffman huffman; /**< Compression huffman encoding type */ }; +/** + * Dictionary ID flag + * If this flag is set, a 4-bytes Dict-ID field will be present, after the + * descriptor flags and the Content Size. + */ +#define RTE_COMP_LZ4_FLAG_DICT_ID (1 << 0) + +/** + * Content Checksum flag + * If this flag is set, a 32-bits content checksum will be appended after the + * EndMark. + */ +#define RTE_COMP_LZ4_FLAG_CONTENT_CHECKSUM (1 << 2) + +/** + * Content Size flag + * If this flag is set, the uncompressed size of data included within the frame + * will be present as an 8 bytes unsigned little-endian value, after the flags. + * Content Size usage is optional. + */ +#define RTE_COMP_LZ4_FLAG_CONTENT_SIZE (1 << 3) + +/** + * Block Checksum flag. + * If this flag is set, each data block will be followed by a 4-bytes checksum, + * calculated by using the xxHash-32 algorithm on the raw (compressed) data + * block. The intention is to detect data corruption (storage or transmission + * errors) immediately, before decoding. Block checksum usage is optional. + */ +#define RTE_COMP_LZ4_FLAG_BLOCK_CHECKSUM (1 << 4) + +/** + * Block Independence flag. + * If this flag is set to 1, blocks are independent. + * If this flag is set to 0, each block depends on previous ones (up to LZ4 + * window size, which is 64 KB). In such case, it is necessary to decode all + * blocks in sequence. + * Block dependency improves compression ratio, especially for small blocks. On + * the other hand, it makes random access or multi-threaded decoding impossible. + */ +#define RTE_COMP_LZ4_FLAG_BLOCK_INDEPENDENCE (1 << 5) + +/** Parameters specific to the LZ4 algorithm */ +struct rte_comp_lz4_params { + uint8_t flags; + /**< Compression LZ4 parameter flags. + * Based on LZ4 standard flags: + * https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md#frame-descriptor + */ +}; + /** Setup Data for compression */ struct rte_comp_compress_xform { enum rte_comp_algorithm algo; @@ -220,6 +289,8 @@ struct rte_comp_compress_xform { union { struct rte_comp_deflate_params deflate; /**< Parameters specific to the deflate algorithm */ + struct rte_comp_lz4_params lz4; + /**< Parameters specific to the LZ4 algorithm */ }; /**< Algorithm specific parameters */ int level; /**< Compression level */ @@ -249,6 +320,10 @@ struct rte_comp_decompress_xform { * compressed data. If window size can't be supported by the PMD then * setup of stream or private_xform should fail. */ + union { + struct rte_comp_lz4_params lz4; + /**< Parameters specific to the LZ4 algorithm */ + }; /**< Algorithm specific parameters */ enum rte_comp_hash_algorithm hash_algo; /**< Hash algorithm to be used with decompress operation. Hash is always * done on plaintext. From patchwork Thu Feb 2 09:13:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 122889 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A824041BAB; Thu, 2 Feb 2023 10:14:12 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CD62942D59; Thu, 2 Feb 2023 10:14:01 +0100 (CET) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2073.outbound.protection.outlook.com [40.107.220.73]) by mails.dpdk.org (Postfix) with ESMTP id 1D85F42D49 for ; Thu, 2 Feb 2023 10:14:00 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PVtxWl/wvXrVuAR76Z+EldPZ5J9QV8Onwrz85z5DJ9KimyQcSt+7iau7RFRxTZ1Y7SO7f8X0vynX+iph/3QpTfhVBp3cvNqxNL3ID96MZc34PWFSklAy7Y0E9qJoSDuTxJoGcGBAwXxOOKw8ZgAgL06Cjjbc+jOd7V+y8N7I2ddW1HA8sXG49s+WcyuOe4S/ur6JSjdItkMpPyM3M9AoraTR3u0QEonYPDw0PunFp5csTfQZxC7Fx8gfB+gnKc+2ceMouTtPXsyF6YgE3F6v2drd46QCnhhxsfQde7jKwRzx8cJDJCkTuKoNFtNuUk+9r9X0cmr+ifLTM/U/GO6I5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pofdpSpu2+hwZcp4y+nPG2NU1JjGVX7/JQCzjWWzraI=; b=kro0+wf0uZki1hlgpiqiBn+uCOzmVFbEVq/niysPzlXFnC1VP/E8m77tmShqLDNr5+vmHAuNNzw0tElaqPORAbc4viaH7rYHDSzOsIoAB1roGfCocw+fELCIppXoaNPkJbCWHBMQcTMuQdjyCezB92hNu7GnEJ9+I7y6wHsEgmC3wu446T2wMtk5U8Oa6TLaohKD+6aYCn5YW9vMQ6PnaPTc9gsVAig3uy5otJbg6lrfCgpt6jDxioJMnUsFKwNevVjXzrb42hH+0aoesjxs24BQe+dMuq3v85qXdLLBkEarwYZXdlIjYTZp1jJE1e+XVTnovQ6Jdf0A5xfBBstIFA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pofdpSpu2+hwZcp4y+nPG2NU1JjGVX7/JQCzjWWzraI=; b=dYolvaXNPuUv2LHPuU/bgF+iWES3ZmUs2rko/SLNzJsyuV/Wt0dCwiJI7UwVRTqLR1XaZh8hWzLdmaxpP8R6H61OLEep02xXnqZsFWpCjoE5ztHQubaTq/+knQ2C3O5CuRBeaRQHEK5nRXU8E5Fe/rZeWV4TfDgL5mNJQP0OCgsZmm1OBK+CQNkUrLMzMI/JK/eB+RQx6kamwaczI0wUSElPItDqLA2tsH94tGV+mN1ll67glpra/2rmLJNWGn2zoDac9S0oljGHCrCJhzO00ngoab3Qvp3Du8KjsWF2qsU1Ql4THrFwen1xDCsdJg5YuFEVCqnrIL1znLgUBWizpg== Received: from BN0PR08CA0001.namprd08.prod.outlook.com (2603:10b6:408:142::6) by PH7PR12MB6857.namprd12.prod.outlook.com (2603:10b6:510:1af::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.27; Thu, 2 Feb 2023 09:13:57 +0000 Received: from BN8NAM11FT110.eop-nam11.prod.protection.outlook.com (2603:10b6:408:142:cafe::2b) by BN0PR08CA0001.outlook.office365.com (2603:10b6:408:142::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.25 via Frontend Transport; Thu, 2 Feb 2023 09:13:57 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN8NAM11FT110.mail.protection.outlook.com (10.13.176.156) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.27 via Frontend Transport; Thu, 2 Feb 2023 09:13:57 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 2 Feb 2023 01:13:43 -0800 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 2 Feb 2023 01:13:42 -0800 Received: from nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36 via Frontend Transport; Thu, 2 Feb 2023 01:13:40 -0800 From: Michael Baum To: CC: Matan Azrad , Akhil Goyal , "Ashish Gupta" , Fiona Trahe , "Thomas Monjalon" Subject: [PATCH v2 2/4] app/test-compress-perf: allow test single compress operation Date: Thu, 2 Feb 2023 11:13:30 +0200 Message-ID: <20230202091332.1037078-3-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202091332.1037078-1-michaelba@nvidia.com> References: <20230109074526.2507344-1-michaelba@nvidia.com> <20230202091332.1037078-1-michaelba@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT110:EE_|PH7PR12MB6857:EE_ X-MS-Office365-Filtering-Correlation-Id: fb96c6d5-e828-4ec5-dbaf-08db04fdd07b X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3aRw6pcVV1PlVjiTPSf8UOH/+xVG/nrHATwDYDX41f1ikezotJCk++1+m2Voj7K0TTRvRMcpWAzweuPKAHO8kXJXUTiJmQZ80Tt+M3zn6ZL5uaOt7OMteF+89vbKfVnw8kehmjtr4VXkDQm8tLagyNn3m4aQEnOS6QLkfXMm+jBasXQV+L95jna9H9dxZCeAtMlbC4fZDqE4jX1KIRMX58ybKNeT/7ly+f/M/4DqLNt9Xd3KHidv+iLVnF+piauRjbUvp2lscL5KwNP0AauGVxJQ/AflXXk0s+z/QLaU9EzpNyBLN7PMf3H7jKanK5hmKBogHaZnBqsGf0+XMhMsDoQWAIo50wmMres20xOUpglafwb7evVf4tQL77wSaROAWrkDQTMLUqky8hABvLJeO7sE76dm/oKDdj02pu53PdX8fH+F2gf585J52hZKzOFnECSbPOzdhSnxT/eTM42Q9yByz3tmHKRVfWL1XAvhjdMjNSIfcasdYx8PdAFBDJ5uupdj2xc4jphREzfuHXjgrh3XRD2gxQcVzoVQ8W2dmX54hzyeaDY5IhyMxkwzFA07a8zK/4Kxc7oa7TLo0o6mI3CUbJWVMKp7yRyEzhe5HPC1VBrbPtoEJe7wt8Qzf3t/Xo9YaoF/VuLSGbtZdXtsnp2eforc8YTFGyor/ksUI49fLvQiUI9PLXuNQpwyng0ic75dcwxBYJ/tQ3M7/zerdw== X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230025)(4636009)(376002)(136003)(39860400002)(396003)(346002)(451199018)(40470700004)(46966006)(36840700001)(336012)(66899018)(2906002)(478600001)(36756003)(70586007)(30864003)(40460700003)(6666004)(1076003)(26005)(186003)(6286002)(2616005)(7696005)(55016003)(8936002)(40480700001)(47076005)(86362001)(5660300002)(356005)(41300700001)(426003)(82310400005)(8676002)(83380400001)(70206006)(4326008)(6916009)(316002)(82740400003)(36860700001)(7636003)(54906003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2023 09:13:57.2041 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fb96c6d5-e828-4ec5-dbaf-08db04fdd07b X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT110.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6857 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Part of the application options is to test only compress and only decompress but actually the application ignores this user option and tries to test always both compress and decompress. Allow testing only compress and only decompress. Signed-off-by: Matan Azrad Signed-off-by: Michael Baum --- app/test-compress-perf/comp_perf_options.h | 6 +- .../comp_perf_options_parse.c | 4 +- .../comp_perf_test_common.c | 126 +++++++++++++----- .../comp_perf_test_cyclecount.c | 71 ++++++---- .../comp_perf_test_throughput.c | 64 +++++---- .../comp_perf_test_verify.c | 64 +++++---- app/test-compress-perf/main.c | 8 ++ doc/guides/rel_notes/release_23_03.rst | 4 + 8 files changed, 230 insertions(+), 117 deletions(-) diff --git a/app/test-compress-perf/comp_perf_options.h b/app/test-compress-perf/comp_perf_options.h index 57dd146330..d00b299247 100644 --- a/app/test-compress-perf/comp_perf_options.h +++ b/app/test-compress-perf/comp_perf_options.h @@ -32,9 +32,9 @@ enum cperf_test_type { }; enum comp_operation { - COMPRESS_ONLY, - DECOMPRESS_ONLY, - COMPRESS_DECOMPRESS + COMPRESS = (1 << 0), + DECOMPRESS = (1 << 1), + COMPRESS_DECOMPRESS = (COMPRESS | DECOMPRESS), }; struct range_list { diff --git a/app/test-compress-perf/comp_perf_options_parse.c b/app/test-compress-perf/comp_perf_options_parse.c index 019eddb7bd..7a992bf43e 100644 --- a/app/test-compress-perf/comp_perf_options_parse.c +++ b/app/test-compress-perf/comp_perf_options_parse.c @@ -446,11 +446,11 @@ parse_op_type(struct comp_test_data *test_data, const char *arg) struct name_id_map optype_namemap[] = { { "comp", - COMPRESS_ONLY + COMPRESS }, { "decomp", - DECOMPRESS_ONLY + DECOMPRESS }, { "comp_and_decomp", diff --git a/app/test-compress-perf/comp_perf_test_common.c b/app/test-compress-perf/comp_perf_test_common.c index b402a0d839..cd60958944 100644 --- a/app/test-compress-perf/comp_perf_test_common.c +++ b/app/test-compress-perf/comp_perf_test_common.c @@ -227,23 +227,43 @@ comp_perf_allocate_memory(struct comp_test_data *test_data, { uint16_t comp_mbuf_size; uint16_t decomp_mbuf_size; + size_t comp_data_size; + size_t decomp_data_size; + size_t output_data_sz; test_data->out_seg_sz = find_buf_size(test_data->seg_sz); - /* Number of segments for input and output - * (compression and decompression) - */ - test_data->total_segs = DIV_CEIL(test_data->input_data_sz, - test_data->seg_sz); + if (test_data->test_op & COMPRESS) { + /* + * Number of segments for input and output + * (compression and decompression) + */ + test_data->total_segs = DIV_CEIL(test_data->input_data_sz, + test_data->seg_sz); + } else { + /* + * When application does decompression only, input data is + * compressed and smaller than the output. The expected size of + * uncompressed data given by the user in segment size argument. + */ + test_data->total_segs = test_data->max_sgl_segs; + } + + output_data_sz = (size_t) test_data->out_seg_sz * test_data->total_segs; + output_data_sz = + RTE_MAX(output_data_sz, (size_t) MIN_COMPRESSED_BUF_SIZE); if (test_data->use_external_mbufs != 0) { if (comp_perf_allocate_external_mbufs(test_data, mem) < 0) return -1; comp_mbuf_size = 0; decomp_mbuf_size = 0; - } else { + } else if (test_data->test_op & COMPRESS) { comp_mbuf_size = test_data->out_seg_sz + RTE_PKTMBUF_HEADROOM; decomp_mbuf_size = test_data->seg_sz + RTE_PKTMBUF_HEADROOM; + } else { + comp_mbuf_size = test_data->seg_sz + RTE_PKTMBUF_HEADROOM; + decomp_mbuf_size = test_data->out_seg_sz + RTE_PKTMBUF_HEADROOM; } char pool_name[32] = ""; @@ -287,26 +307,28 @@ comp_perf_allocate_memory(struct comp_test_data *test_data, return -1; } - /* - * Compressed data might be a bit larger than input data, - * if data cannot be compressed - */ - mem->compressed_data = rte_zmalloc_socket(NULL, - RTE_MAX( - (size_t) test_data->out_seg_sz * - test_data->total_segs, - (size_t) MIN_COMPRESSED_BUF_SIZE), - 0, - rte_socket_id()); + if (test_data->test_op & COMPRESS) { + /* + * Compressed data might be a bit larger than input data, + * if data cannot be compressed + */ + comp_data_size = output_data_sz; + decomp_data_size = test_data->input_data_sz; + } else { + comp_data_size = test_data->input_data_sz; + decomp_data_size = output_data_sz; + } + + mem->compressed_data = rte_zmalloc_socket(NULL, comp_data_size, 0, + rte_socket_id()); if (mem->compressed_data == NULL) { RTE_LOG(ERR, USER1, "Memory to hold the data from the input " "file could not be allocated\n"); return -1; } - mem->decompressed_data = rte_zmalloc_socket(NULL, - test_data->input_data_sz, 0, - rte_socket_id()); + mem->decompressed_data = rte_zmalloc_socket(NULL, decomp_data_size, 0, + rte_socket_id()); if (mem->decompressed_data == NULL) { RTE_LOG(ERR, USER1, "Memory to hold the data from the input " "file could not be allocated\n"); @@ -351,6 +373,7 @@ prepare_bufs(struct comp_test_data *test_data, struct cperf_mem_resources *mem) uint16_t segs_per_mbuf = 0; uint32_t cmz = 0; uint32_t dmz = 0; + bool decompress_only = !!(test_data->test_op == DECOMPRESS); for (i = 0; i < mem->total_bufs; i++) { /* Allocate data in input mbuf and copy data from input file */ @@ -361,8 +384,6 @@ prepare_bufs(struct comp_test_data *test_data, struct cperf_mem_resources *mem) return -1; } - data_sz = RTE_MIN(remaining_data, test_data->seg_sz); - if (test_data->use_external_mbufs != 0) { rte_pktmbuf_attach_extbuf(mem->decomp_bufs[i], mem->decomp_memzones[dmz]->addr, @@ -372,16 +393,23 @@ prepare_bufs(struct comp_test_data *test_data, struct cperf_mem_resources *mem) dmz++; } + if (!decompress_only) + data_sz = RTE_MIN(remaining_data, test_data->seg_sz); + else + data_sz = test_data->out_seg_sz; + data_addr = (uint8_t *) rte_pktmbuf_append( mem->decomp_bufs[i], data_sz); if (data_addr == NULL) { RTE_LOG(ERR, USER1, "Could not append data\n"); return -1; } - rte_memcpy(data_addr, input_data_ptr, data_sz); - input_data_ptr += data_sz; - remaining_data -= data_sz; + if (!decompress_only) { + rte_memcpy(data_addr, input_data_ptr, data_sz); + input_data_ptr += data_sz; + remaining_data -= data_sz; + } /* Already one segment in the mbuf */ segs_per_mbuf = 1; @@ -398,8 +426,6 @@ prepare_bufs(struct comp_test_data *test_data, struct cperf_mem_resources *mem) return -1; } - data_sz = RTE_MIN(remaining_data, test_data->seg_sz); - if (test_data->use_external_mbufs != 0) { rte_pktmbuf_attach_extbuf( next_seg, @@ -410,6 +436,12 @@ prepare_bufs(struct comp_test_data *test_data, struct cperf_mem_resources *mem) dmz++; } + if (!decompress_only) + data_sz = RTE_MIN(remaining_data, + test_data->seg_sz); + else + data_sz = test_data->out_seg_sz; + data_addr = (uint8_t *)rte_pktmbuf_append(next_seg, data_sz); @@ -418,9 +450,11 @@ prepare_bufs(struct comp_test_data *test_data, struct cperf_mem_resources *mem) return -1; } - rte_memcpy(data_addr, input_data_ptr, data_sz); - input_data_ptr += data_sz; - remaining_data -= data_sz; + if (!decompress_only) { + rte_memcpy(data_addr, input_data_ptr, data_sz); + input_data_ptr += data_sz; + remaining_data -= data_sz; + } if (rte_pktmbuf_chain(mem->decomp_bufs[i], next_seg) < 0) { @@ -447,16 +481,26 @@ prepare_bufs(struct comp_test_data *test_data, struct cperf_mem_resources *mem) cmz++; } - data_addr = (uint8_t *) rte_pktmbuf_append( - mem->comp_bufs[i], - test_data->out_seg_sz); + if (decompress_only) + data_sz = RTE_MIN(remaining_data, test_data->seg_sz); + else + data_sz = test_data->out_seg_sz; + + data_addr = (uint8_t *) rte_pktmbuf_append(mem->comp_bufs[i], + data_sz); if (data_addr == NULL) { RTE_LOG(ERR, USER1, "Could not append data\n"); return -1; } + if (decompress_only) { + rte_memcpy(data_addr, input_data_ptr, data_sz); + input_data_ptr += data_sz; + remaining_data -= data_sz; + } + /* Chain mbufs if needed for output mbufs */ - for (j = 1; j < segs_per_mbuf; j++) { + for (j = 1; j < segs_per_mbuf && remaining_data > 0; j++) { struct rte_mbuf *next_seg = rte_pktmbuf_alloc(mem->comp_buf_pool); @@ -476,13 +520,25 @@ prepare_bufs(struct comp_test_data *test_data, struct cperf_mem_resources *mem) cmz++; } + if (decompress_only) + data_sz = RTE_MIN(remaining_data, + test_data->seg_sz); + else + data_sz = test_data->out_seg_sz; + data_addr = (uint8_t *)rte_pktmbuf_append(next_seg, - test_data->out_seg_sz); + data_sz); if (data_addr == NULL) { RTE_LOG(ERR, USER1, "Could not append data\n"); return -1; } + if (decompress_only) { + rte_memcpy(data_addr, input_data_ptr, data_sz); + input_data_ptr += data_sz; + remaining_data -= data_sz; + } + if (rte_pktmbuf_chain(mem->comp_bufs[i], next_seg) < 0) { RTE_LOG(ERR, USER1, "Could not chain mbufs\n"); diff --git a/app/test-compress-perf/comp_perf_test_cyclecount.c b/app/test-compress-perf/comp_perf_test_cyclecount.c index 7473cb6277..ce6c4d7605 100644 --- a/app/test-compress-perf/comp_perf_test_cyclecount.c +++ b/app/test-compress-perf/comp_perf_test_cyclecount.c @@ -514,38 +514,55 @@ cperf_cyclecount_test_runner(void *test_ctx) if (cperf_verify_test_runner(&ctx->ver)) return EXIT_FAILURE; - /* - * Run the tests twice, discarding the first performance - * results, before the cache is warmed up - */ - - /* C O M P R E S S */ - for (i = 0; i < 2; i++) { - if (main_loop(ctx, RTE_COMP_COMPRESS) < 0) - return EXIT_FAILURE; - } + if (test_data->test_op & COMPRESS) { + /* + * Run the test twice, discarding the first performance + * results, before the cache is warmed up + */ + for (i = 0; i < 2; i++) { + if (main_loop(ctx, RTE_COMP_COMPRESS) < 0) + return EXIT_FAILURE; + } - ops_enq_retries_comp = ctx->ops_enq_retries; - ops_deq_retries_comp = ctx->ops_deq_retries; + ops_enq_retries_comp = ctx->ops_enq_retries; + ops_deq_retries_comp = ctx->ops_deq_retries; - duration_enq_per_op_comp = ctx->duration_enq / - (ctx->ver.mem.total_bufs * test_data->num_iter); - duration_deq_per_op_comp = ctx->duration_deq / - (ctx->ver.mem.total_bufs * test_data->num_iter); + duration_enq_per_op_comp = ctx->duration_enq / + (ctx->ver.mem.total_bufs * test_data->num_iter); + duration_deq_per_op_comp = ctx->duration_deq / + (ctx->ver.mem.total_bufs * test_data->num_iter); + } else { + ops_enq_retries_comp = 0; + ops_deq_retries_comp = 0; - /* D E C O M P R E S S */ - for (i = 0; i < 2; i++) { - if (main_loop(ctx, RTE_COMP_DECOMPRESS) < 0) - return EXIT_FAILURE; + duration_enq_per_op_comp = 0; + duration_deq_per_op_comp = 0; } - ops_enq_retries_decomp = ctx->ops_enq_retries; - ops_deq_retries_decomp = ctx->ops_deq_retries; + if (test_data->test_op & DECOMPRESS) { + /* + * Run the test twice, discarding the first performance + * results, before the cache is warmed up + */ + for (i = 0; i < 2; i++) { + if (main_loop(ctx, RTE_COMP_DECOMPRESS) < 0) + return EXIT_FAILURE; + } - duration_enq_per_op_decomp = ctx->duration_enq / - (ctx->ver.mem.total_bufs * test_data->num_iter); - duration_deq_per_op_decomp = ctx->duration_deq / - (ctx->ver.mem.total_bufs * test_data->num_iter); + ops_enq_retries_decomp = ctx->ops_enq_retries; + ops_deq_retries_decomp = ctx->ops_deq_retries; + + duration_enq_per_op_decomp = ctx->duration_enq / + (ctx->ver.mem.total_bufs * test_data->num_iter); + duration_deq_per_op_decomp = ctx->duration_deq / + (ctx->ver.mem.total_bufs * test_data->num_iter); + } else { + ops_enq_retries_decomp = 0; + ops_deq_retries_decomp = 0; + + duration_enq_per_op_decomp = 0; + duration_deq_per_op_decomp = 0; + } duration_setup_per_op = ctx->duration_op / (ctx->ver.mem.total_bufs * test_data->num_iter); @@ -563,7 +580,7 @@ cperf_cyclecount_test_runner(void *test_ctx) " [D-e] - decompression enqueue\n" " [D-d] - decompression dequeue\n" " - Cycles section: number of cycles per 'op' for the following operations:\n" - " setup/op - memory allocation, op configuration and memory dealocation\n" + " setup/op - memory allocation, op configuration and memory deallocation\n" " [C-e] - compression enqueue\n" " [C-d] - compression dequeue\n" " [D-e] - decompression enqueue\n" diff --git a/app/test-compress-perf/comp_perf_test_throughput.c b/app/test-compress-perf/comp_perf_test_throughput.c index 79cd2b2bf2..c9f8237626 100644 --- a/app/test-compress-perf/comp_perf_test_throughput.c +++ b/app/test-compress-perf/comp_perf_test_throughput.c @@ -359,41 +359,53 @@ cperf_throughput_test_runner(void *test_ctx) * First the verification part is needed */ if (cperf_verify_test_runner(&ctx->ver)) { - ret = EXIT_FAILURE; + ret = EXIT_FAILURE; goto end; } - /* - * Run the tests twice, discarding the first performance - * results, before the cache is warmed up - */ - for (i = 0; i < 2; i++) { - if (main_loop(ctx, RTE_COMP_COMPRESS) < 0) { - ret = EXIT_FAILURE; - goto end; + if (test_data->test_op & COMPRESS) { + /* + * Run the test twice, discarding the first performance + * results, before the cache is warmed up + */ + for (i = 0; i < 2; i++) { + if (main_loop(ctx, RTE_COMP_COMPRESS) < 0) { + ret = EXIT_FAILURE; + goto end; + } } - } - for (i = 0; i < 2; i++) { - if (main_loop(ctx, RTE_COMP_DECOMPRESS) < 0) { - ret = EXIT_FAILURE; - goto end; - } + ctx->comp_tsc_byte = + (double)(ctx->comp_tsc_duration[test_data->level]) / + test_data->input_data_sz; + ctx->comp_gbps = rte_get_tsc_hz() / ctx->comp_tsc_byte * 8 / + 1000000000; + } else { + ctx->comp_tsc_byte = 0; + ctx->comp_gbps = 0; } - ctx->comp_tsc_byte = - (double)(ctx->comp_tsc_duration[test_data->level]) / - test_data->input_data_sz; + if (test_data->test_op & DECOMPRESS) { + /* + * Run the test twice, discarding the first performance + * results, before the cache is warmed up + */ + for (i = 0; i < 2; i++) { + if (main_loop(ctx, RTE_COMP_DECOMPRESS) < 0) { + ret = EXIT_FAILURE; + goto end; + } + } - ctx->decomp_tsc_byte = + ctx->decomp_tsc_byte = (double)(ctx->decomp_tsc_duration[test_data->level]) / - test_data->input_data_sz; - - ctx->comp_gbps = rte_get_tsc_hz() / ctx->comp_tsc_byte * 8 / - 1000000000; - - ctx->decomp_gbps = rte_get_tsc_hz() / ctx->decomp_tsc_byte * 8 / - 1000000000; + test_data->input_data_sz; + ctx->decomp_gbps = rte_get_tsc_hz() / ctx->decomp_tsc_byte * 8 / + 1000000000; + } else { + ctx->decomp_tsc_byte = 0; + ctx->decomp_gbps = 0; + } exp = 0; if (__atomic_compare_exchange_n(&display_once, &exp, 1, 0, diff --git a/app/test-compress-perf/comp_perf_test_verify.c b/app/test-compress-perf/comp_perf_test_verify.c index 8964442891..7d6b6abecd 100644 --- a/app/test-compress-perf/comp_perf_test_verify.c +++ b/app/test-compress-perf/comp_perf_test_verify.c @@ -114,7 +114,8 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type) output_data_sz = &ctx->decomp_data_sz; input_bufs = mem->comp_bufs; output_bufs = mem->decomp_bufs; - out_seg_sz = test_data->seg_sz; + out_seg_sz = (test_data->test_op & COMPRESS) ? + test_data->seg_sz : test_data->out_seg_sz; } /* Create private xform */ @@ -226,7 +227,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type) op->status == RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE) { RTE_LOG(ERR, USER1, -"Out of space error occurred due to uncompressible input data expanding to larger than destination buffer. Increase the EXPANSE_RATIO constant to use this data.\n"); +"Out of space error occurred due to incompressible input data expanding to larger than destination buffer. Increase the EXPANSE_RATIO constant to use this data.\n"); res = -1; goto end; } else if (op->status != @@ -311,7 +312,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type) op->status == RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE) { RTE_LOG(ERR, USER1, -"Out of space error occurred due to uncompressible input data expanding to larger than destination buffer. Increase the EXPANSE_RATIO constant to use this data.\n"); +"Out of space error occurred due to incompressible input data expanding to larger than destination buffer. Increase the EXPANSE_RATIO constant to use this data.\n"); res = -1; goto end; } else if (op->status != @@ -392,44 +393,59 @@ cperf_verify_test_runner(void *test_ctx) int ret = EXIT_SUCCESS; static uint16_t display_once; uint32_t lcore = rte_lcore_id(); + uint16_t exp = 0; ctx->mem.lcore_id = lcore; test_data->ratio = 0; - if (main_loop(ctx, RTE_COMP_COMPRESS) < 0) { - ret = EXIT_FAILURE; - goto end; + if (test_data->test_op & COMPRESS) { + if (main_loop(ctx, RTE_COMP_COMPRESS) < 0) { + ret = EXIT_FAILURE; + goto end; + } } - if (main_loop(ctx, RTE_COMP_DECOMPRESS) < 0) { - ret = EXIT_FAILURE; - goto end; - } + if (test_data->test_op & DECOMPRESS) { + if (main_loop(ctx, RTE_COMP_DECOMPRESS) < 0) { + ret = EXIT_FAILURE; + goto end; + } - if (ctx->decomp_data_sz != test_data->input_data_sz) { - RTE_LOG(ERR, USER1, - "Decompressed data length not equal to input data length\n"); - RTE_LOG(ERR, USER1, - "Decompressed size = %zu, expected = %zu\n", - ctx->decomp_data_sz, test_data->input_data_sz); - ret = EXIT_FAILURE; - goto end; - } else { - if (memcmp(ctx->mem.decompressed_data, - test_data->input_data, - test_data->input_data_sz) != 0) { + if (!(test_data->test_op & COMPRESS)) { + /* + * For DECOMPRESS_ONLY mode there is no more + * verifications, reset the 'ratio' and 'comp_data_sz' + * fields for other tests report. + */ + ctx->comp_data_sz = 0; + ctx->ratio = 0; + goto end; + } + + if (ctx->decomp_data_sz != test_data->input_data_sz) { + RTE_LOG(ERR, USER1, + "Decompressed data length not equal to input data length\n"); RTE_LOG(ERR, USER1, - "Decompressed data is not the same as file data\n"); + "Decompressed size = %zu, expected = %zu\n", + ctx->decomp_data_sz, test_data->input_data_sz); ret = EXIT_FAILURE; goto end; + } else { + if (memcmp(ctx->mem.decompressed_data, + test_data->input_data, + test_data->input_data_sz) != 0) { + RTE_LOG(ERR, USER1, + "Decompressed data is not the same as file data\n"); + ret = EXIT_FAILURE; + goto end; + } } } ctx->ratio = (double) ctx->comp_data_sz / test_data->input_data_sz * 100; - uint16_t exp = 0; if (!ctx->silent) { if (__atomic_compare_exchange_n(&display_once, &exp, 1, 0, __ATOMIC_RELAXED, __ATOMIC_RELAXED)) { diff --git a/app/test-compress-perf/main.c b/app/test-compress-perf/main.c index 41b8edc2bd..bbb4c7917b 100644 --- a/app/test-compress-perf/main.c +++ b/app/test-compress-perf/main.c @@ -254,6 +254,14 @@ comp_perf_dump_input_data(struct comp_test_data *test_data) goto end; } + if (!(test_data->test_op & COMPRESS) && + test_data->input_data_sz > + (size_t) test_data->seg_sz * (size_t) test_data->max_sgl_segs) { + RTE_LOG(ERR, USER1, + "Size of input must be less than total segments\n"); + goto end; + } + test_data->input_data = rte_zmalloc_socket(NULL, test_data->input_data_sz, 0, rte_socket_id()); diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst index 2722296180..15fa8e812b 100644 --- a/doc/guides/rel_notes/release_23_03.rst +++ b/doc/guides/rel_notes/release_23_03.rst @@ -85,6 +85,10 @@ New Features * Added support for ``RTE_COMP_ALGO_LZ4``. * Added support for ``RTE_COMP_CHECKSUM_XXHASH32``. +* **Allowed test single compress operation in test-compress-perf.** + + Enable the application options for testing only compress and only decompress. + Removed Items ------------- From patchwork Thu Feb 2 09:13:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 122890 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D745241BAB; Thu, 2 Feb 2023 10:14:19 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D209E42D5F; Thu, 2 Feb 2023 10:14:02 +0100 (CET) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2066.outbound.protection.outlook.com [40.107.94.66]) by mails.dpdk.org (Postfix) with ESMTP id 8413942D4D for ; Thu, 2 Feb 2023 10:14:00 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QBcSPh2XCpGc2FbK+mO6uhFJlAdIjAVC/LzNHleagCfewQsYGphaNfX5Fktwq2ciajrNSOU2oNamQ9J+eVI0d76pfMr2uzmO4oyWBsor54GuJaPbI6Ko4KAhHJwQVion4Gj72QDG17lneLsk+5HI0GQg+Eu8mAGh3egvMG0s8cu5kX0U/kl13RrN8nC8+bbXWu2kegPQTp8Mer3ntMYJSlNY14vEFiRAX4K9dXRPXlPVa3uyW469sHMFopfaZb+ckzIdfmHHYBuvN1zcSk+FGSx9j+zhkThN2JQmOIwNHdET3ZnEvA/3F68tHROfG8fEcIRixAWKg7i9GnZs8E0S2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5xkcEdvJ1opNOfT3DRu1/YWt45PNTiOd6lUAIvlubaw=; b=F/8P0/mHGPOD78IkuEGJMTb7wSSFN4/qhxu+O8lxXNyYFT9oMp4StOZkvwSKdyKcENvbRE0LkQGd+8c8eusgVW+UYd6Q7E9L/4w38xPi+8AhVTa3vdTKSMCGC/r93+knE7t2WzH2AHC0HgBNbcPxJjwb7ruDmG3j9R3z/2yhy54OZZnAAHjlLvvJ/lYCNi5aqRkC+HxquBzKOXZMWbi2vgpJVxM04Wj6guutzWW/YLEGYdZR0KDWVmeJqy65HbF8aoqze7y8qvQKmcm9+z8spk7fkyU/MIrq/fRjl5NU6Xye/Xqf1QZBdxBdtYGnBJFt5U4i67rwYSJa+Bl5cntXRA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5xkcEdvJ1opNOfT3DRu1/YWt45PNTiOd6lUAIvlubaw=; b=UdV6g/FJXnqBzRlQZb0Lu32r8GB/PjjjexNrqsEsRNs87VoPWoh/EsjHaSHkZeeFfScYMU5wAcKS77VVgnpvapo8GPmXqUyKE0ozL8oksbkpIqGwR2SAzXmHeWCfqGnaT+aX14U+E3povAUSHscDMaLiGfJ6xq1ii19AQMjfO8bl0nK2jRFke78nz/CLa2RE4wwkAmQxe62ODmyasKbCG/0LvUKTvA6dvUiGApAA3X2LNDUCFwwCiHZOpIksv8a9IRqq21YeUAErTtx3hxdlNfP70jViYQ9QkNcXiK5oAf/ip2YYJ/ytMfyXftbz9uYtDhagizGchPQLEKHzzVv4tw== Received: from BN9PR03CA0632.namprd03.prod.outlook.com (2603:10b6:408:13b::7) by SN7PR12MB6692.namprd12.prod.outlook.com (2603:10b6:806:270::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6043.38; Thu, 2 Feb 2023 09:13:58 +0000 Received: from BN8NAM11FT051.eop-nam11.prod.protection.outlook.com (2603:10b6:408:13b:cafe::8c) by BN9PR03CA0632.outlook.office365.com (2603:10b6:408:13b::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.27 via Frontend Transport; Thu, 2 Feb 2023 09:13:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BN8NAM11FT051.mail.protection.outlook.com (10.13.177.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.27 via Frontend Transport; Thu, 2 Feb 2023 09:13:58 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 2 Feb 2023 01:13:45 -0800 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 2 Feb 2023 01:13:45 -0800 Received: from nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36 via Frontend Transport; Thu, 2 Feb 2023 01:13:43 -0800 From: Michael Baum To: CC: Matan Azrad , Akhil Goyal , "Ashish Gupta" , Fiona Trahe , "Thomas Monjalon" Subject: [PATCH v2 3/4] app/test-compress-perf: add algo option Date: Thu, 2 Feb 2023 11:13:31 +0200 Message-ID: <20230202091332.1037078-4-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202091332.1037078-1-michaelba@nvidia.com> References: <20230109074526.2507344-1-michaelba@nvidia.com> <20230202091332.1037078-1-michaelba@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT051:EE_|SN7PR12MB6692:EE_ X-MS-Office365-Filtering-Correlation-Id: d3282fa5-db92-4c08-bf20-08db04fdd121 X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zrA2/tY9IMRQ3RFY3VbBAjpkzF73cZviTCdbegvBUAyoxXIexBJGPzdcmvOfnhGvOcSZnfQAlxoSrgnD82sJunHVGUgUHXXlZu8cBkGu2T5OW0RgIQWUfI68DwVlPmqPQs0yJoGG6ePJfk8Nvw3GLtIFTmFp7+Sh/5CMeWEIsL31+vIIUPfL1khGIsphSBRZGAPBQuI/DBePW3Afw0LTyCEKn5jW4a6kouKOXkmqWo/WR2nIDOuyhC3XOvHjvPVNJADYfrHvbUr88/fOSqIT2LfL5MZOzVsmnmSju5SqDGaWTPGTe3+qnBO+qwar9oAANh7wO/xveSDj/tfp/lh7UU47O5PLfeBzqmdvMLjLDM37buja/okL6sWhBje4YS6fQk0wq3pgB84zvzqQQ4pJCW5bRJSxycAgPsEhUbMHdJFyyY7KFURW9ww6cbg73Pz+2qv6LCa/x/4brFpNIRmCGJlPcJDhFJMXjMrs8oPw9Rm/z49wQrNd6E9HqjC1baO8lUCkzs4z743xmI1zs7IEHi/xZH88ELzZydmERp0S+BfaowCN9G8r8hEpSEDHvslvfv0rQPeQzEZ55wx8hJ6U7VFC4Q5J5Qj7I30Q9iBR6FNci46Pjc3B8ZcylLmVwjpLkuoTtGq/Z8nJm8Q4a9hj7d11ofDOLPEkyds1Vd+Nc4d7MumeBNf+h/vLm44DjIQxHakzb6v88SvWba4BpZN2iA== X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230025)(4636009)(136003)(376002)(346002)(396003)(39860400002)(451199018)(46966006)(36840700001)(40470700004)(47076005)(2616005)(36756003)(54906003)(6666004)(336012)(83380400001)(316002)(426003)(2906002)(36860700001)(82740400003)(7636003)(82310400005)(41300700001)(8676002)(478600001)(70586007)(70206006)(7696005)(4326008)(1076003)(26005)(6286002)(40460700003)(55016003)(186003)(356005)(40480700001)(5660300002)(30864003)(8936002)(86362001)(6916009); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2023 09:13:58.3069 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d3282fa5-db92-4c08-bf20-08db04fdd121 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT051.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB6692 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add a command line option `algo` to select the compress algorithm supported by the compress API: null (DMA), deflate and lzs. Default for deflate. Signed-off-by: Matan Azrad Signed-off-by: Michael Baum --- app/test-compress-perf/comp_perf_options.h | 2 + .../comp_perf_options_parse.c | 40 ++++++++++++++++++- .../comp_perf_test_cyclecount.c | 4 +- .../comp_perf_test_throughput.c | 4 +- .../comp_perf_test_verify.c | 4 +- app/test-compress-perf/main.c | 38 +++++++++++------- doc/guides/rel_notes/release_23_03.rst | 4 ++ doc/guides/tools/comp_perf.rst | 2 + 8 files changed, 76 insertions(+), 22 deletions(-) diff --git a/app/test-compress-perf/comp_perf_options.h b/app/test-compress-perf/comp_perf_options.h index d00b299247..5e5227a700 100644 --- a/app/test-compress-perf/comp_perf_options.h +++ b/app/test-compress-perf/comp_perf_options.h @@ -63,6 +63,8 @@ struct comp_test_data { enum rte_comp_huffman huffman_enc; enum comp_operation test_op; + enum rte_comp_algorithm test_algo; + int window_sz; struct range_list level_lst; uint8_t level; diff --git a/app/test-compress-perf/comp_perf_options_parse.c b/app/test-compress-perf/comp_perf_options_parse.c index 7a992bf43e..97ddff87ef 100644 --- a/app/test-compress-perf/comp_perf_options_parse.c +++ b/app/test-compress-perf/comp_perf_options_parse.c @@ -25,6 +25,7 @@ #define CPERF_MAX_SGL_SEGS ("max-num-sgl-segs") #define CPERF_NUM_ITER ("num-iter") #define CPERF_OPTYPE ("operation") +#define CPERF_ALGO ("algo") #define CPERF_HUFFMAN_ENC ("huffman-enc") #define CPERF_LEVEL ("compress-level") #define CPERF_WINDOW_SIZE ("window-sz") @@ -56,6 +57,8 @@ usage(char *progname) " compressed/decompressed (default: 10000)\n" " --operation [comp/decomp/comp_and_decomp]: perform test on\n" " compression, decompression or both operations\n" + " --algo [null/deflate/lzs]: perform test on algorithm\n" + " null(DMA), deflate or lzs (default: deflate)\n" " --huffman-enc [fixed/dynamic/default]: Huffman encoding\n" " (default: dynamic)\n" " --compress-level N: compression level, which could be a single value, list or range\n" @@ -470,6 +473,36 @@ parse_op_type(struct comp_test_data *test_data, const char *arg) return 0; } +static int +parse_algo(struct comp_test_data *test_data, const char *arg) +{ + struct name_id_map algo_namemap[] = { + { + "null", + RTE_COMP_ALGO_NULL + }, + { + "deflate", + RTE_COMP_ALGO_DEFLATE + }, + { + "lzs", + RTE_COMP_ALGO_LZS + } + }; + + int id = get_str_key_id_mapping(algo_namemap, + RTE_DIM(algo_namemap), arg); + if (id < 0) { + RTE_LOG(ERR, USER1, "Invalid algorithm specified\n"); + return -1; + } + + test_data->test_algo = (enum rte_comp_algorithm)id; + + return 0; +} + static int parse_huffman_enc(struct comp_test_data *test_data, const char *arg) { @@ -491,7 +524,7 @@ parse_huffman_enc(struct comp_test_data *test_data, const char *arg) int id = get_str_key_id_mapping(huffman_namemap, RTE_DIM(huffman_namemap), arg); if (id < 0) { - RTE_LOG(ERR, USER1, "Invalid Huffmane encoding specified\n"); + RTE_LOG(ERR, USER1, "Invalid Huffman encoding specified\n"); return -1; } @@ -507,7 +540,7 @@ parse_level(struct comp_test_data *test_data, const char *arg) /* * Try parsing the argument as a range, if it fails, - * arse it as a list + * parse it as a list */ if (parse_range(arg, &test_data->level_lst.min, &test_data->level_lst.max, @@ -572,6 +605,7 @@ static struct option lgopts[] = { { CPERF_MAX_SGL_SEGS, required_argument, 0, 0}, { CPERF_NUM_ITER, required_argument, 0, 0 }, { CPERF_OPTYPE, required_argument, 0, 0 }, + { CPERF_ALGO, required_argument, 0, 0 }, { CPERF_HUFFMAN_ENC, required_argument, 0, 0 }, { CPERF_LEVEL, required_argument, 0, 0 }, { CPERF_WINDOW_SIZE, required_argument, 0, 0 }, @@ -594,6 +628,7 @@ comp_perf_opts_parse_long(int opt_idx, struct comp_test_data *test_data) { CPERF_MAX_SGL_SEGS, parse_max_num_sgl_segs }, { CPERF_NUM_ITER, parse_num_iter }, { CPERF_OPTYPE, parse_op_type }, + { CPERF_ALGO, parse_algo }, { CPERF_HUFFMAN_ENC, parse_huffman_enc }, { CPERF_LEVEL, parse_level }, { CPERF_WINDOW_SIZE, parse_window_sz }, @@ -649,6 +684,7 @@ comp_perf_options_default(struct comp_test_data *test_data) test_data->num_iter = 10000; test_data->huffman_enc = RTE_COMP_HUFFMAN_DYNAMIC; test_data->test_op = COMPRESS_DECOMPRESS; + test_data->test_algo = RTE_COMP_ALGO_DEFLATE; test_data->window_sz = -1; test_data->level_lst.min = RTE_COMP_LEVEL_MIN; test_data->level_lst.max = RTE_COMP_LEVEL_MAX; diff --git a/app/test-compress-perf/comp_perf_test_cyclecount.c b/app/test-compress-perf/comp_perf_test_cyclecount.c index ce6c4d7605..81c3d30038 100644 --- a/app/test-compress-perf/comp_perf_test_cyclecount.c +++ b/app/test-compress-perf/comp_perf_test_cyclecount.c @@ -193,7 +193,7 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type) xform = (struct rte_comp_xform) { .type = RTE_COMP_COMPRESS, .compress = { - .algo = RTE_COMP_ALGO_DEFLATE, + .algo = test_data->test_algo, .deflate.huffman = test_data->huffman_enc, .level = test_data->level, .window_size = test_data->window_sz, @@ -208,7 +208,7 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type) xform = (struct rte_comp_xform) { .type = RTE_COMP_DECOMPRESS, .decompress = { - .algo = RTE_COMP_ALGO_DEFLATE, + .algo = test_data->test_algo, .chksum = RTE_COMP_CHECKSUM_NONE, .window_size = test_data->window_sz, .hash_algo = RTE_COMP_HASH_ALGO_NONE diff --git a/app/test-compress-perf/comp_perf_test_throughput.c b/app/test-compress-perf/comp_perf_test_throughput.c index c9f8237626..2545ee9925 100644 --- a/app/test-compress-perf/comp_perf_test_throughput.c +++ b/app/test-compress-perf/comp_perf_test_throughput.c @@ -84,7 +84,7 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type) xform = (struct rte_comp_xform) { .type = RTE_COMP_COMPRESS, .compress = { - .algo = RTE_COMP_ALGO_DEFLATE, + .algo = test_data->test_algo, .deflate.huffman = test_data->huffman_enc, .level = test_data->level, .window_size = test_data->window_sz, @@ -99,7 +99,7 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type) xform = (struct rte_comp_xform) { .type = RTE_COMP_DECOMPRESS, .decompress = { - .algo = RTE_COMP_ALGO_DEFLATE, + .algo = test_data->test_algo, .chksum = RTE_COMP_CHECKSUM_NONE, .window_size = test_data->window_sz, .hash_algo = RTE_COMP_HASH_ALGO_NONE diff --git a/app/test-compress-perf/comp_perf_test_verify.c b/app/test-compress-perf/comp_perf_test_verify.c index 7d6b6abecd..88f4f41851 100644 --- a/app/test-compress-perf/comp_perf_test_verify.c +++ b/app/test-compress-perf/comp_perf_test_verify.c @@ -87,7 +87,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type) xform = (struct rte_comp_xform) { .type = RTE_COMP_COMPRESS, .compress = { - .algo = RTE_COMP_ALGO_DEFLATE, + .algo = test_data->test_algo, .deflate.huffman = test_data->huffman_enc, .level = test_data->level, .window_size = test_data->window_sz, @@ -104,7 +104,7 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type) xform = (struct rte_comp_xform) { .type = RTE_COMP_DECOMPRESS, .decompress = { - .algo = RTE_COMP_ALGO_DEFLATE, + .algo = test_data->test_algo, .chksum = RTE_COMP_CHECKSUM_NONE, .window_size = test_data->window_sz, .hash_algo = RTE_COMP_HASH_ALGO_NONE diff --git a/app/test-compress-perf/main.c b/app/test-compress-perf/main.c index bbb4c7917b..d049527ba1 100644 --- a/app/test-compress-perf/main.c +++ b/app/test-compress-perf/main.c @@ -57,29 +57,39 @@ comp_perf_check_capabilities(struct comp_test_data *test_data, uint8_t cdev_id) { const struct rte_compressdev_capabilities *cap; - cap = rte_compressdev_capability_get(cdev_id, - RTE_COMP_ALGO_DEFLATE); + cap = rte_compressdev_capability_get(cdev_id, test_data->test_algo); if (cap == NULL) { RTE_LOG(ERR, USER1, - "Compress device does not support DEFLATE\n"); + "Compress device does not support %u algorithm\n", + test_data->test_algo); return -1; } uint64_t comp_flags = cap->comp_feature_flags; - /* Huffman encoding */ - if (test_data->huffman_enc == RTE_COMP_HUFFMAN_FIXED && - (comp_flags & RTE_COMP_FF_HUFFMAN_FIXED) == 0) { - RTE_LOG(ERR, USER1, - "Compress device does not supported Fixed Huffman\n"); - return -1; - } + /* Algorithm type */ + switch (test_data->test_algo) { + case RTE_COMP_ALGO_DEFLATE: + /* Huffman encoding */ + if (test_data->huffman_enc == RTE_COMP_HUFFMAN_FIXED && + (comp_flags & RTE_COMP_FF_HUFFMAN_FIXED) == 0) { + RTE_LOG(ERR, USER1, + "Compress device does not supported Fixed Huffman\n"); + return -1; + } - if (test_data->huffman_enc == RTE_COMP_HUFFMAN_DYNAMIC && - (comp_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) { - RTE_LOG(ERR, USER1, - "Compress device does not supported Dynamic Huffman\n"); + if (test_data->huffman_enc == RTE_COMP_HUFFMAN_DYNAMIC && + (comp_flags & RTE_COMP_FF_HUFFMAN_DYNAMIC) == 0) { + RTE_LOG(ERR, USER1, + "Compress device does not supported Dynamic Huffman\n"); + return -1; + } + break; + case RTE_COMP_ALGO_LZS: + case RTE_COMP_ALGO_NULL: + break; + default: return -1; } diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst index 15fa8e812b..da345ded48 100644 --- a/doc/guides/rel_notes/release_23_03.rst +++ b/doc/guides/rel_notes/release_23_03.rst @@ -89,6 +89,10 @@ New Features Enable the application options for testing only compress and only decompress. +* **Added algo option in test-compress-perf.** + + Added support for testing other algorithms except for DEFLAT. + Removed Items ------------- diff --git a/doc/guides/tools/comp_perf.rst b/doc/guides/tools/comp_perf.rst index 9d2f4dbe4a..cf9035cc23 100644 --- a/doc/guides/tools/comp_perf.rst +++ b/doc/guides/tools/comp_perf.rst @@ -84,6 +84,8 @@ Application Options ``--operation [comp/decomp/comp_and_decomp]``: perform test on compression, decompression or both operations + ``--algo [null/deflate/lzs]`` : perform test on algorithm null(DMA), Deflate or lzs (default: Deflate) + ``--huffman-enc [fixed/dynamic/default]``: Huffman encoding (default: dynamic) ``--compress-level N``: compression level, which could be a single value, list or range (default: range between 1 and 9) From patchwork Thu Feb 2 09:13:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Baum X-Patchwork-Id: 122891 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 32F1341BAB; Thu, 2 Feb 2023 10:14:29 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CB4E342D56; Thu, 2 Feb 2023 10:14:06 +0100 (CET) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2063.outbound.protection.outlook.com [40.107.244.63]) by mails.dpdk.org (Postfix) with ESMTP id 1391F42D49 for ; Thu, 2 Feb 2023 10:14:05 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=itMjLB/LtNhz9BiolE52z/qqONhgSNV+QIB5Wx1DM1xaVmofxojjuPojWjJN2EwdATsFi/WrJrOekJ0QjejqpZHLTlHTpgaoVpSlinhYuv/jPcEvSLnH+2bo8nm55XfnAXvAVgFPE68e2+QWmXxD5Oqs/vCU4LFbeWsEpFtPbJfWKSHctQXoFH3kL6QAzqu3Ts3UAhnZ2Vnly9VyDJQ2eYKDwc6ivc0zsmwwV5zZajyGNIHTl6DKXTzvgxDIouER68ldifQbOqMiPZ5p2ltH1wh2x3HO5ZbHm2RGEwJXNbFExdvcT7bv6YcdfvGLQ6spjLWK4mpBd5E8tR5FDQ8MBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KOz7e5/JCOKDCqNBCL4k/FnsokPIuqv22ajhH9Se7pw=; b=ep3e9sQ7jIBEVjDhlPiwkVR6JI+5hen+w2/YH8tIsxDFEH29CU/4+T6nBrd+8ENVoutFKnseO3M4z1OkCK4eIEsRra65pe2+jkBJZ4OW+Z0U9zm7fR0UQoJB0vZ1fUkPP5yaXdVAjS9alxMwujYxPbbz088Knrw/A+0asuf8iAgeYNncE2obqokLLkRb52MNWQ/Xdyf20EVsDWz/4Vi7b3amspPwwPm4IYCkTY26UcWxo6TZvkA1jdlkCuvrmN4TihEnPqD3KUx7KoBOTLATzDDB/0ScZOLTeMmLzlSVO3I8aaB6pFsoKVuaW+LdsH3FVEDn4auFeqZmhfnRye0u8g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=dpdk.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KOz7e5/JCOKDCqNBCL4k/FnsokPIuqv22ajhH9Se7pw=; b=YyXy2PQmx2fZahbRg2VYPtQ+i8RdWqzF1WVPiwWCyxEiC54wzTg7x5mX08YEXN7+7qdp1wIUoNy9OfTjFVXrQW2AmdyLVgisDpNRXtL1i1xqKZ47FM+BapyRssBGXPumt5WCAjpG/Xwf7y9UdvXpVmh+zaZU8dhRyxm1INi8pSIDWYzLgvQuJk7sAtlb3CKv0wFtyzgrygSaAqXkn0DCBdybhsCBx4er4U2k4NFjBGLMMmov5CADg2ztNUNOUD6px3ehI58QfeUtXhgxiOjXuJUHo0tqF3Mv+cK2JzwmJM7d/7lflnfKVWeGRVlUrHDNLaflbHeyet5zP+hcCjn2Mw== Received: from BN9PR03CA0319.namprd03.prod.outlook.com (2603:10b6:408:112::24) by DS7PR12MB6021.namprd12.prod.outlook.com (2603:10b6:8:87::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.27; Thu, 2 Feb 2023 09:14:03 +0000 Received: from BN8NAM11FT016.eop-nam11.prod.protection.outlook.com (2603:10b6:408:112:cafe::c5) by BN9PR03CA0319.outlook.office365.com (2603:10b6:408:112::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.27 via Frontend Transport; Thu, 2 Feb 2023 09:14:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN8NAM11FT016.mail.protection.outlook.com (10.13.176.97) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6064.27 via Frontend Transport; Thu, 2 Feb 2023 09:14:02 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 2 Feb 2023 01:13:47 -0800 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Thu, 2 Feb 2023 01:13:47 -0800 Received: from nvidia.com (10.127.8.13) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36 via Frontend Transport; Thu, 2 Feb 2023 01:13:45 -0800 From: Michael Baum To: CC: Matan Azrad , Akhil Goyal , "Ashish Gupta" , Fiona Trahe , "Thomas Monjalon" Subject: [PATCH v2 4/4] app/test-compress-perf: add LZ4 support Date: Thu, 2 Feb 2023 11:13:32 +0200 Message-ID: <20230202091332.1037078-5-michaelba@nvidia.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230202091332.1037078-1-michaelba@nvidia.com> References: <20230109074526.2507344-1-michaelba@nvidia.com> <20230202091332.1037078-1-michaelba@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT016:EE_|DS7PR12MB6021:EE_ X-MS-Office365-Filtering-Correlation-Id: a69711fe-8f66-451e-15b7-08db04fdd397 X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ejCgw/NpG9+onkZmeQs/ZTA2r6OHRcYdp090JqkAIF9omf3xh8hI8ALO//qIcgP0IoqTOM+25AuCO65l1q8nWTSURk5OtOG2CA5sT3iEsF76PlvmOk63uD8bdC0l8t4VA5HFxl3RZjXE/TK/W09erYYPtEZU0TN+KQoJrXn3ON4Q9p9cP4uaP/Qs3YfxX+IanTwNISHv819KOehd3gN0IURAzfvmlIF+7zgJgepO/HXgrPyK3C9jeSd74I8dkUzyKJDk5FKpLuVUBRNOB+xZW9qu+g9hMcXNpyC+ZMJRILNzbFiI4/+JuamXwunp3hHoSprQXXr9M/KVDSinccw3n2uj4qtVU8yL+pb4Dj9KXcLLRWsofwDdS2HeBl4fjpfE5YLxyEDnN6OelI1Nq5iysngsyVE8knhk4/zwTRBS0WxE3sLw+VbbKSTK/D9U30d37sZIrZIlJxKn6Whs71CMamplYFmFNJMGWyZ327YTlxOXYnSEkKJ1jOHI0KmbmeZrlfbnhvsXsVOsxxQlrOsUk5+NPaC7cdnTqZL/OlbYaZ1l2lBkKSLfBL62UHYMGNGI2vFtQjzK8fvNj5pI/wdd5mgpX5ghK9KAg7o5y1oy9j2SjSpYv2/4qBTZZNDTvYhmKjDtdvhHab+DcSXCDAu5RzjsbWPgdBXTK8UXJELxnWunA7F/3ZhpwqhVGJQQuEiyL6GcsG54RZNyH8ySbIgBsmlB+pkAAnSg1VotHEBJ7fVeBJsUHtPT/pih4JNwiNa/kuMKy4Z4JLOVADVlRrGioRAEnAVeTIaedoZKqeT/QEk= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230025)(4636009)(376002)(136003)(39860400002)(396003)(346002)(451199018)(40470700004)(46966006)(36840700001)(336012)(2906002)(478600001)(36756003)(70586007)(30864003)(40460700003)(6666004)(1076003)(26005)(186003)(6286002)(2616005)(7696005)(55016003)(8936002)(40480700001)(47076005)(86362001)(5660300002)(356005)(41300700001)(426003)(82310400005)(8676002)(83380400001)(70206006)(4326008)(6916009)(316002)(82740400003)(36860700001)(7636003)(54906003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2023 09:14:02.4157 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a69711fe-8f66-451e-15b7-08db04fdd397 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT016.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6021 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add support for LZ4 algorithm and add a new parameter for it. Signed-off-by: Michael Baum --- app/test-compress-perf/comp_perf_options.h | 1 + .../comp_perf_options_parse.c | 43 ++++++++++++++++++- .../comp_perf_test_cyclecount.c | 7 ++- .../comp_perf_test_throughput.c | 7 ++- .../comp_perf_test_verify.c | 7 ++- app/test-compress-perf/main.c | 17 ++++++++ doc/guides/rel_notes/release_23_03.rst | 1 + doc/guides/tools/comp_perf.rst | 5 ++- 8 files changed, 82 insertions(+), 6 deletions(-) diff --git a/app/test-compress-perf/comp_perf_options.h b/app/test-compress-perf/comp_perf_options.h index 5e5227a700..828a7309d8 100644 --- a/app/test-compress-perf/comp_perf_options.h +++ b/app/test-compress-perf/comp_perf_options.h @@ -61,6 +61,7 @@ struct comp_test_data { uint16_t max_sgl_segs; uint32_t total_segs; + uint8_t lz4_flags; enum rte_comp_huffman huffman_enc; enum comp_operation test_op; enum rte_comp_algorithm test_algo; diff --git a/app/test-compress-perf/comp_perf_options_parse.c b/app/test-compress-perf/comp_perf_options_parse.c index 97ddff87ef..6d8c370fc2 100644 --- a/app/test-compress-perf/comp_perf_options_parse.c +++ b/app/test-compress-perf/comp_perf_options_parse.c @@ -27,6 +27,7 @@ #define CPERF_OPTYPE ("operation") #define CPERF_ALGO ("algo") #define CPERF_HUFFMAN_ENC ("huffman-enc") +#define CPERF_LZ4_FLAGS ("lz4-flags") #define CPERF_LEVEL ("compress-level") #define CPERF_WINDOW_SIZE ("window-sz") #define CPERF_EXTERNAL_MBUFS ("external-mbufs") @@ -57,10 +58,11 @@ usage(char *progname) " compressed/decompressed (default: 10000)\n" " --operation [comp/decomp/comp_and_decomp]: perform test on\n" " compression, decompression or both operations\n" - " --algo [null/deflate/lzs]: perform test on algorithm\n" - " null(DMA), deflate or lzs (default: deflate)\n" + " --algo [null/deflate/lzs/lz4]: perform test on algorithm\n" + " null(DMA), deflate, lzs or lz4 (default: deflate)\n" " --huffman-enc [fixed/dynamic/default]: Huffman encoding\n" " (default: dynamic)\n" + " --lz4-flags N: flags to configure LZ4 algorithm (default: 0)\n" " --compress-level N: compression level, which could be a single value, list or range\n" " (default: range between 1 and 9)\n" " --window-sz N: base two log value of compression window size\n" @@ -153,6 +155,23 @@ parse_uint16_t(uint16_t *value, const char *arg) return 0; } +static int +parse_uint8_t(uint8_t *value, const char *arg) +{ + uint32_t val = 0; + int ret = parse_uint32_t(&val, arg); + + if (ret < 0) + return ret; + + if (val > UINT8_MAX) + return -ERANGE; + + *value = (uint8_t) val; + + return 0; +} + static int parse_range(const char *arg, uint8_t *min, uint8_t *max, uint8_t *inc) { @@ -488,6 +507,10 @@ parse_algo(struct comp_test_data *test_data, const char *arg) { "lzs", RTE_COMP_ALGO_LZS + }, + { + "lz4", + RTE_COMP_ALGO_LZ4 } }; @@ -533,6 +556,19 @@ parse_huffman_enc(struct comp_test_data *test_data, const char *arg) return 0; } +static int +parse_lz4_flags(struct comp_test_data *test_data, const char *arg) +{ + int ret = parse_uint8_t(&test_data->lz4_flags, arg); + + if (ret) { + RTE_LOG(ERR, USER1, "Failed to parse LZ4 flags\n"); + return -1; + } + + return 0; +} + static int parse_level(struct comp_test_data *test_data, const char *arg) { @@ -607,6 +643,7 @@ static struct option lgopts[] = { { CPERF_OPTYPE, required_argument, 0, 0 }, { CPERF_ALGO, required_argument, 0, 0 }, { CPERF_HUFFMAN_ENC, required_argument, 0, 0 }, + { CPERF_LZ4_FLAGS, required_argument, 0, 0 }, { CPERF_LEVEL, required_argument, 0, 0 }, { CPERF_WINDOW_SIZE, required_argument, 0, 0 }, { CPERF_EXTERNAL_MBUFS, 0, 0, 0 }, @@ -630,6 +667,7 @@ comp_perf_opts_parse_long(int opt_idx, struct comp_test_data *test_data) { CPERF_OPTYPE, parse_op_type }, { CPERF_ALGO, parse_algo }, { CPERF_HUFFMAN_ENC, parse_huffman_enc }, + { CPERF_LZ4_FLAGS, parse_lz4_flags }, { CPERF_LEVEL, parse_level }, { CPERF_WINDOW_SIZE, parse_window_sz }, { CPERF_EXTERNAL_MBUFS, parse_external_mbufs }, @@ -682,6 +720,7 @@ comp_perf_options_default(struct comp_test_data *test_data) test_data->pool_sz = 8192; test_data->max_sgl_segs = 16; test_data->num_iter = 10000; + test_data->lz4_flags = 0; test_data->huffman_enc = RTE_COMP_HUFFMAN_DYNAMIC; test_data->test_op = COMPRESS_DECOMPRESS; test_data->test_algo = RTE_COMP_ALGO_DEFLATE; diff --git a/app/test-compress-perf/comp_perf_test_cyclecount.c b/app/test-compress-perf/comp_perf_test_cyclecount.c index 81c3d30038..4d336ec8d6 100644 --- a/app/test-compress-perf/comp_perf_test_cyclecount.c +++ b/app/test-compress-perf/comp_perf_test_cyclecount.c @@ -194,13 +194,16 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type) .type = RTE_COMP_COMPRESS, .compress = { .algo = test_data->test_algo, - .deflate.huffman = test_data->huffman_enc, .level = test_data->level, .window_size = test_data->window_sz, .chksum = RTE_COMP_CHECKSUM_NONE, .hash_algo = RTE_COMP_HASH_ALGO_NONE } }; + if (test_data->test_algo == RTE_COMP_ALGO_DEFLATE) + xform.compress.deflate.huffman = test_data->huffman_enc; + else if (test_data->test_algo == RTE_COMP_ALGO_LZ4) + xform.compress.lz4.flags = test_data->lz4_flags; input_bufs = mem->decomp_bufs; output_bufs = mem->comp_bufs; out_seg_sz = test_data->out_seg_sz; @@ -214,6 +217,8 @@ main_loop(struct cperf_cyclecount_ctx *ctx, enum rte_comp_xform_type type) .hash_algo = RTE_COMP_HASH_ALGO_NONE } }; + if (test_data->test_algo == RTE_COMP_ALGO_LZ4) + xform.decompress.lz4.flags = test_data->lz4_flags; input_bufs = mem->comp_bufs; output_bufs = mem->decomp_bufs; out_seg_sz = test_data->seg_sz; diff --git a/app/test-compress-perf/comp_perf_test_throughput.c b/app/test-compress-perf/comp_perf_test_throughput.c index 2545ee9925..1f7072d223 100644 --- a/app/test-compress-perf/comp_perf_test_throughput.c +++ b/app/test-compress-perf/comp_perf_test_throughput.c @@ -85,13 +85,16 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type) .type = RTE_COMP_COMPRESS, .compress = { .algo = test_data->test_algo, - .deflate.huffman = test_data->huffman_enc, .level = test_data->level, .window_size = test_data->window_sz, .chksum = RTE_COMP_CHECKSUM_NONE, .hash_algo = RTE_COMP_HASH_ALGO_NONE } }; + if (test_data->test_algo == RTE_COMP_ALGO_DEFLATE) + xform.compress.deflate.huffman = test_data->huffman_enc; + else if (test_data->test_algo == RTE_COMP_ALGO_LZ4) + xform.compress.lz4.flags = test_data->lz4_flags; input_bufs = mem->decomp_bufs; output_bufs = mem->comp_bufs; out_seg_sz = test_data->out_seg_sz; @@ -105,6 +108,8 @@ main_loop(struct cperf_benchmark_ctx *ctx, enum rte_comp_xform_type type) .hash_algo = RTE_COMP_HASH_ALGO_NONE } }; + if (test_data->test_algo == RTE_COMP_ALGO_LZ4) + xform.decompress.lz4.flags = test_data->lz4_flags; input_bufs = mem->comp_bufs; output_bufs = mem->decomp_bufs; out_seg_sz = test_data->seg_sz; diff --git a/app/test-compress-perf/comp_perf_test_verify.c b/app/test-compress-perf/comp_perf_test_verify.c index 88f4f41851..7bd18073cf 100644 --- a/app/test-compress-perf/comp_perf_test_verify.c +++ b/app/test-compress-perf/comp_perf_test_verify.c @@ -88,13 +88,16 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type) .type = RTE_COMP_COMPRESS, .compress = { .algo = test_data->test_algo, - .deflate.huffman = test_data->huffman_enc, .level = test_data->level, .window_size = test_data->window_sz, .chksum = RTE_COMP_CHECKSUM_NONE, .hash_algo = RTE_COMP_HASH_ALGO_NONE } }; + if (test_data->test_algo == RTE_COMP_ALGO_DEFLATE) + xform.compress.deflate.huffman = test_data->huffman_enc; + else if (test_data->test_algo == RTE_COMP_ALGO_LZ4) + xform.compress.lz4.flags = test_data->lz4_flags; output_data_ptr = ctx->mem.compressed_data; output_data_sz = &ctx->comp_data_sz; input_bufs = mem->decomp_bufs; @@ -110,6 +113,8 @@ main_loop(struct cperf_verify_ctx *ctx, enum rte_comp_xform_type type) .hash_algo = RTE_COMP_HASH_ALGO_NONE } }; + if (test_data->test_algo == RTE_COMP_ALGO_LZ4) + xform.decompress.lz4.flags = test_data->lz4_flags; output_data_ptr = ctx->mem.decompressed_data; output_data_sz = &ctx->decomp_data_sz; input_bufs = mem->comp_bufs; diff --git a/app/test-compress-perf/main.c b/app/test-compress-perf/main.c index d049527ba1..fa366123ed 100644 --- a/app/test-compress-perf/main.c +++ b/app/test-compress-perf/main.c @@ -86,6 +86,23 @@ comp_perf_check_capabilities(struct comp_test_data *test_data, uint8_t cdev_id) return -1; } break; + case RTE_COMP_ALGO_LZ4: + /* LZ4 flags */ + if ((test_data->lz4_flags & RTE_COMP_LZ4_FLAG_BLOCK_CHECKSUM) && + (comp_flags & RTE_COMP_FF_LZ4_BLOCK_WITH_CHECKSUM) == 0) { + RTE_LOG(ERR, USER1, + "Compress device does not support LZ4 block with checksum\n"); + return -1; + } + + if ((test_data->lz4_flags & + RTE_COMP_LZ4_FLAG_BLOCK_INDEPENDENCE) && + (comp_flags & RTE_COMP_FF_LZ4_BLOCK_INDEPENDENCE) == 0) { + RTE_LOG(ERR, USER1, + "Compress device does not support LZ4 independent blocks\n"); + return -1; + } + break; case RTE_COMP_ALGO_LZS: case RTE_COMP_ALGO_NULL: break; diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst index da345ded48..aedc5767ff 100644 --- a/doc/guides/rel_notes/release_23_03.rst +++ b/doc/guides/rel_notes/release_23_03.rst @@ -84,6 +84,7 @@ New Features * Added support for ``RTE_COMP_ALGO_LZ4``. * Added support for ``RTE_COMP_CHECKSUM_XXHASH32``. + * Added support for ``lz4`` in test-compress-perf algo options. * **Allowed test single compress operation in test-compress-perf.** diff --git a/doc/guides/tools/comp_perf.rst b/doc/guides/tools/comp_perf.rst index cf9035cc23..5eb05a095c 100644 --- a/doc/guides/tools/comp_perf.rst +++ b/doc/guides/tools/comp_perf.rst @@ -84,10 +84,13 @@ Application Options ``--operation [comp/decomp/comp_and_decomp]``: perform test on compression, decompression or both operations - ``--algo [null/deflate/lzs]`` : perform test on algorithm null(DMA), Deflate or lzs (default: Deflate) + ``--algo [null/deflate/lzs/lz4]`` : perform test on algorithm null(DMA), Deflate, lzs or lz4 (default: Deflate) ``--huffman-enc [fixed/dynamic/default]``: Huffman encoding (default: dynamic) + ``--lz4-flags N``: flags to for LZ4 parameters, + see `LZ4 Frame Descriptor `_ (default: no flags) + ``--compress-level N``: compression level, which could be a single value, list or range (default: range between 1 and 9) ``--window-sz N``: base two log value of compression window size (default: max supported by PMD)