From patchwork Thu Feb 22 19:47:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ajit Khaparde X-Patchwork-Id: 137054 X-Patchwork-Delegate: ajit.khaparde@broadcom.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 289FD43B84; Thu, 22 Feb 2024 20:49:12 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 24D91427DB; Thu, 22 Feb 2024 20:48:22 +0100 (CET) Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by mails.dpdk.org (Postfix) with ESMTP id 76CAD41156 for ; Thu, 22 Feb 2024 20:48:19 +0100 (CET) Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-6d9f94b9186so18836b3a.0 for ; Thu, 22 Feb 2024 11:48:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1708631298; x=1709236098; darn=dpdk.org; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=VZ6+cr9BFxmuPkoUzWDQ+z8Ovg+g8UOR0P9HnZikfRM=; b=G8hn7jTl2LAAj7M5cf24z8krC4zObU3LcSAbIba8tuurJbjJf3xPfWyLzvgLrfI3Uw O8o71fTXDK3IYRIOg2L2bK0OGolsaqTzQm57M7NPoH6svG4dpQL1og2RJKOH0dLNrjqp w3NBYizNhEuJWqnwNGLkgguBhOeu+YGvOY1wA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708631298; x=1709236098; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VZ6+cr9BFxmuPkoUzWDQ+z8Ovg+g8UOR0P9HnZikfRM=; b=GpX/Kmc9Nr3WtrlZsrKX5+rGpa5YcGmsZlmVRluHAiAFcFpDWKvRaNFQiGsDkK8aML DO2nUKQEfKbi6Y26ltxgTE1AptHCgXGSd578Vo/HTAgVPxQVo02mTFvDKBvq2HLCtkAR nMjcgVPIoR26ke7NkN9paFzpJN86iaSjqaesgTkzPVh3URXnHSKSmr7QfBDeiMMp5HTX 6tyEdnpa79Gm2KCijd0mQ/1ezqG08tn/ENFuQdaQ19DDehvsos3ci4EwTEZijuwikXZu 2+Dn/Nzh0pg4PBG2/vfLe3hQyXY0OB0GW4/kI25lAJHv84SOg+Hg9lTLTNZiHL7T2SKm zkjA== X-Gm-Message-State: AOJu0Yw5W2jLVeV8waMO8eOQAg+e5ihwzHaq6JWdWxBgWy/kCSCA62UX yqH1NDuwcFG+9EWP1BbF2JXoHky+ltu/PgvziQol45f+h/o3644xHvmzVQdC+sedtT9w+pmLqGB WrHewqXzrFrSzdZ30ZEzj+fMZcCeLEsaqII+uGIjWtgLMoKwjUt9Igc5P5A6BJCBcypcoas5m+X Hcj5AGFhlsD4v7VkqR0FMlsy4NnSao X-Google-Smtp-Source: AGHT+IGXR6VcCO8qewTJOlWuurFpPCzR1/t4LJU1LxsOjFMGiVCdc1BK0ct84e23aJIBRbsVYJJV/A== X-Received: by 2002:a05:6a00:ccc:b0:6e4:d354:160a with SMTP id b12-20020a056a000ccc00b006e4d354160amr2349805pfv.1.1708631297139; Thu, 22 Feb 2024 11:48:17 -0800 (PST) Received: from C02GC2QQMD6T.wifi.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id gu12-20020a056a004e4c00b006e025e1fa61sm9538452pfb.14.2024.02.22.11.48.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 22 Feb 2024 11:48:16 -0800 (PST) From: Ajit Khaparde To: dev@dpdk.org Cc: ferruh.yigit@amd.com, thomas@monjalon.net Subject: [PATCH v2 08/10] net/bnxt: fix compressed CQE handling Date: Thu, 22 Feb 2024 11:47:58 -0800 Message-Id: <20240222194800.80648-9-ajit.khaparde@broadcom.com> X-Mailer: git-send-email 2.39.2 (Apple Git-143) In-Reply-To: <20240222194800.80648-1-ajit.khaparde@broadcom.com> References: <20240221212044.27209-1-ajit.khaparde@broadcom.com> <20240222194800.80648-1-ajit.khaparde@broadcom.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org We were trying to reuse parts of 32-byte CQE handling for compressed CQE handling. And that was causing the packet errors to be misinterpreted. Fix it by using separate code for the compressed CQE handling. Fixes: 812fd99f8c4e ("net/bnxt: add SSE Rx for compressed CQE") Signed-off-by: Ajit Khaparde --- drivers/net/bnxt/bnxt_rxtx_vec_sse.c | 107 ++++++++++++++++++++++++++- 1 file changed, 105 insertions(+), 2 deletions(-) diff --git a/drivers/net/bnxt/bnxt_rxtx_vec_sse.c b/drivers/net/bnxt/bnxt_rxtx_vec_sse.c index 6c0e33200c..e367fbfbd1 100644 --- a/drivers/net/bnxt/bnxt_rxtx_vec_sse.c +++ b/drivers/net/bnxt/bnxt_rxtx_vec_sse.c @@ -112,6 +112,109 @@ descs_to_mbufs(__m128i mm_rxcmp[4], __m128i mm_rxcmp1[4], errors = _mm_or_si128(errors, _mm_srli_epi32(is_tunnel, 1)); index = _mm_or_si128(index, is_tunnel); + /* Update mbuf rearm_data for four packets. */ + GET_OL_FLAGS(rss_flags, index, errors, 0, ol_flags); + _mm_store_si128((void *)&mbuf[0]->rearm_data, + _mm_or_si128(mbuf_init, _mm_set_epi64x(ol_flags, 0))); + + GET_OL_FLAGS(rss_flags, index, errors, 1, ol_flags); + _mm_store_si128((void *)&mbuf[1]->rearm_data, + _mm_or_si128(mbuf_init, _mm_set_epi64x(ol_flags, 0))); + + GET_OL_FLAGS(rss_flags, index, errors, 2, ol_flags); + _mm_store_si128((void *)&mbuf[2]->rearm_data, + _mm_or_si128(mbuf_init, _mm_set_epi64x(ol_flags, 0))); + + GET_OL_FLAGS(rss_flags, index, errors, 3, ol_flags); + _mm_store_si128((void *)&mbuf[3]->rearm_data, + _mm_or_si128(mbuf_init, _mm_set_epi64x(ol_flags, 0))); + + /* Update mbuf rx_descriptor_fields1 for four packets. */ + GET_DESC_FIELDS(mm_rxcmp[0], mm_rxcmp1[0], shuf_msk, ptype_idx, 0, t0); + _mm_store_si128((void *)&mbuf[0]->rx_descriptor_fields1, t0); + + GET_DESC_FIELDS(mm_rxcmp[1], mm_rxcmp1[1], shuf_msk, ptype_idx, 1, t0); + _mm_store_si128((void *)&mbuf[1]->rx_descriptor_fields1, t0); + + GET_DESC_FIELDS(mm_rxcmp[2], mm_rxcmp1[2], shuf_msk, ptype_idx, 2, t0); + _mm_store_si128((void *)&mbuf[2]->rx_descriptor_fields1, t0); + + GET_DESC_FIELDS(mm_rxcmp[3], mm_rxcmp1[3], shuf_msk, ptype_idx, 3, t0); + _mm_store_si128((void *)&mbuf[3]->rx_descriptor_fields1, t0); +} + +static inline void +crx_descs_to_mbufs(__m128i mm_rxcmp[4], __m128i mm_rxcmp1[4], + __m128i mbuf_init, const __m128i shuf_msk, + struct rte_mbuf **mbuf, struct bnxt_rx_ring_info *rxr) +{ + const __m128i flags_type_mask = + _mm_set1_epi32(RX_PKT_COMPRESS_CMPL_FLAGS_ITYPE_MASK); + const __m128i flags2_mask1 = + _mm_set1_epi32(CMPL_FLAGS2_VLAN_TUN_MSK_CRX); + const __m128i flags2_mask2 = + _mm_set1_epi32(RX_PKT_COMPRESS_CMPL_FLAGS_IP_TYPE); + const __m128i rss_mask = + _mm_set1_epi32(RX_PKT_COMPRESS_CMPL_FLAGS_RSS_VALID); + const __m128i cs_err_mask = + _mm_set1_epi32(RX_PKT_COMPRESS_CMPL_CS_ERROR_CALC_MASK | + BNXT_RXC_METADATA1_VLAN_VALID); + const __m128i crx_flags_mask = + _mm_set1_epi32(BNXT_CRX_CQE_CSUM_CALC_MASK); + const __m128i crx_tun_cs = + _mm_set1_epi32(BNXT_CRX_TUN_CS_CALC); + __m128i t0, t1, flags_type, flags, index, errors, rss_flags; + __m128i ptype_idx, is_tunnel; + uint32_t ol_flags; + __m128i cs_err; + __m128i t3, t4; + + /* Validate ptype table indexing at build time. */ + bnxt_check_ptype_constants(); + + /* Compute packet type table indexes for four packets */ + t0 = _mm_unpacklo_epi32(mm_rxcmp[0], mm_rxcmp[1]); + t3 = _mm_unpackhi_epi32(mm_rxcmp[0], mm_rxcmp[1]); + t1 = _mm_unpacklo_epi32(mm_rxcmp[2], mm_rxcmp[3]); + t4 = _mm_unpackhi_epi32(mm_rxcmp[2], mm_rxcmp[3]); + flags_type = _mm_unpacklo_epi64(t0, t1); + ptype_idx = _mm_srli_epi32(_mm_and_si128(flags_type, flags_type_mask), + RX_PKT_CMPL_FLAGS_ITYPE_SFT - BNXT_PTYPE_TBL_TYPE_SFT); + + flags = _mm_unpacklo_epi64(t0, t1); + + ptype_idx = _mm_or_si128(ptype_idx, + _mm_srli_epi32(_mm_and_si128(flags, flags2_mask1), + RX_PKT_CMPL_FLAGS2_META_FORMAT_SFT - + BNXT_PTYPE_TBL_VLAN_SFT)); + ptype_idx = _mm_or_si128(ptype_idx, + _mm_srli_epi32(_mm_and_si128(flags, flags2_mask2), + RX_PKT_CMPL_FLAGS2_IP_TYPE_SFT - + BNXT_PTYPE_TBL_IP_VER_SFT)); + + /* Extract RSS valid flags for four packets. */ + rss_flags = _mm_srli_epi32(_mm_and_si128(flags, rss_mask), 9); + + /* Extract cs_err fields for four packets. */ + cs_err = _mm_unpacklo_epi64(t3, t4); + cs_err = _mm_and_si128(cs_err, cs_err_mask); + flags = _mm_and_si128(cs_err, crx_flags_mask); + + /* Compute ol_flags and checksum error indexes for four packets. */ + is_tunnel = _mm_and_si128(flags, crx_tun_cs); + is_tunnel = _mm_slli_epi32(is_tunnel, 0x20); + flags = _mm_or_si128(flags, is_tunnel); + + flags = _mm_srli_si128(flags, 1); + + errors = _mm_and_si128(cs_err, _mm_set1_epi32(0xF0)); + errors = _mm_and_si128(_mm_srli_epi32(errors, 4), flags); + + index = _mm_andnot_si128(errors, flags); + /* reuse is_tunnel - just shift right one bit to index correctly. */ + errors = _mm_or_si128(errors, _mm_srli_epi32(is_tunnel, 1)); + index = _mm_or_si128(index, is_tunnel); + /* Update mbuf rearm_data for four packets. */ GET_OL_FLAGS(rss_flags, index, errors, 0, ol_flags); _mm_store_si128((void *)&mbuf[0]->rearm_data, @@ -392,8 +495,8 @@ crx_burst_vec_sse(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) if (num_valid == 0) break; - descs_to_mbufs(rxcmp, rxcmp1, mbuf_init, shuf_msk, &rx_pkts[nb_rx_pkts], - rxr); + crx_descs_to_mbufs(rxcmp, rxcmp1, mbuf_init, shuf_msk, + &rx_pkts[nb_rx_pkts], rxr); nb_rx_pkts += num_valid; if (num_valid < BNXT_RX_DESCS_PER_LOOP_VEC128)