From patchwork Thu Nov 10 02:09:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rongwei Liu X-Patchwork-Id: 119690 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6F748A04FD; Thu, 10 Nov 2022 03:10:23 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5610440156; Thu, 10 Nov 2022 03:10:23 +0100 (CET) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2089.outbound.protection.outlook.com [40.107.92.89]) by mails.dpdk.org (Postfix) with ESMTP id 5CCF2400D4; Thu, 10 Nov 2022 03:10:22 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IwJY/JHm0mbgUOIvXb4jQtqr/Zc+Egx11cEFUOtpxIkDEPzyZtHf4rAH+vhWBSoPt59ct0/+zFsNmfozKfosFgGvg2Z2msLRVt5q5NdbLDf5LxHU8gHIWDiivfn3PvqpFLTiKkCwl7QkoZgazfbcT9sNL92I4LY75Y1ks4NFHurcMDSuaF43tOztNoOrT5aiDuos6KdDiAjqzwDACzaj+9dy9JzGds4d1qI6ahfB42lutlRfx2NJQlJVpmq3fMP5pm3+wScYO8mMbTYtpgAP2SksvDG+F3THQ/s8RoZAF6xcTR2we6504UDe5AGPLuRX4gf0GZqFtrDAlsOjgGBEhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8efnU3r28QqS+Zyda7Hle167gq/GG4EVsImlPf8ftPY=; b=Frt+jACYBSs14Lkd7cN8RWD8cXFMsnIrvkLP3orgKIp+cVv0r6S1CcwccIuJ1Dc3tmJxfbmKee7CwJd0B4/ADwfagFL+sFjyaFoVborq2H7Zitu6zYZWFTYlDWZ/8ybxy5PIE062/8KhcnG7DW3fOjd+lzkNPHI2K+JsFGg9K/OIedIBUhlqJ7XS28dJeP2/dDadVskDXn941uzQXrnwsOBpp+0B4Yxs+vBGeHo8eD9Vwm1jn+gAArrg3V2//QXUz4/UhZijBitqvY3ixoSfpdU4HGbzDHXwkHU6//w2VRmmNPUfhWAYXuefgMoxwu+4oHflWxY2DACfh+lFgDAPjg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=monjalon.net smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8efnU3r28QqS+Zyda7Hle167gq/GG4EVsImlPf8ftPY=; b=LQkDJvCa2o0XXRs2GKxC7DIe7jC0V2mM6wHk+VxrE8PN6pjWj1M/n6ISFmTWoI16EtkF83/q2JxGmYtHT7aYfL/Y27J6RSmWf5bewTlvm3wWbc4vBTP/AIJWGnVaNixRniddMCqgOC7+erAS32tBI9P4s5UTp0M7yKj0bD3h8fClH7rkmiwYBdUnjnPNmExS0PLLJEdb3QcMMgYeaIeDEL1/uRTyld+yhOyl57reiaxZ8rTEscDpMWk9t+dcH7nThMr7w7b7EcTA43jujBWipeVpvSPaP+uYjrST+D0EfVmELngTEGEfNRmtX7MMtabVUM6W5/hXxh+qj5rd/8vTuQ== Received: from BN0PR04CA0161.namprd04.prod.outlook.com (2603:10b6:408:eb::16) by DM4PR12MB5988.namprd12.prod.outlook.com (2603:10b6:8:6b::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.27; Thu, 10 Nov 2022 02:10:21 +0000 Received: from BN8NAM11FT011.eop-nam11.prod.protection.outlook.com (2603:10b6:408:eb:cafe::91) by BN0PR04CA0161.outlook.office365.com (2603:10b6:408:eb::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.27 via Frontend Transport; Thu, 10 Nov 2022 02:10:20 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN8NAM11FT011.mail.protection.outlook.com (10.13.176.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.12 via Frontend Transport; Thu, 10 Nov 2022 02:10:20 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Wed, 9 Nov 2022 18:10:06 -0800 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.36; Wed, 9 Nov 2022 18:10:03 -0800 From: Rongwei Liu To: , , , , Suanming Mou CC: , , Subject: [PATCH v1] net/mlx5: fix missing marks on received packets Date: Thu, 10 Nov 2022 04:09:49 +0200 Message-ID: <20221110020950.662431-1-rongweil@nvidia.com> X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT011:EE_|DM4PR12MB5988:EE_ X-MS-Office365-Filtering-Correlation-Id: 0df0d16c-edb6-45f9-125a-08dac2c0b824 X-LD-Processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qgNFHsARHiH2PMdc6ZdrxcSOyz4dd9fYrcqoEU/A9e9/397vOOoPadAfohwrw7QK1GmoNTj2ucCSYKL6WABAEMBV9CLgYSalYPPdrrHzNO1dL4eLKNBYp69HH1ZpQeK6fj7XMF+7wKYVH5MEMq5PdE6skBPLk9yB140UOmt0p94GX9mbn4Qe1642y9ZLLG3e3SwIGx99JbnSOh6oSIo7LZ1xXADtfqCAE4MCnnkZACuqXUAYpG8sDjpaYpnGFsn9YWadrXEnUijnWSSENk6dJgTAGnoFrvv86TDrPWyLPipfL0wkWSzVWXbgBwcN/kCmr8ssKyPreBFdETSbDKKQm+2zemDw9zbYpn3PWPExxIJ2WSPpB5uRc8O0WOtTxSdTwP6yn/C6jpbsBT/SlS1lrO+XKIyxzXycqFRUPdQphbYjWMEFnz6H1cvKo9AQ35hawCNU7f15ULMLl8igMzrdWjaIfnE8baIfss6dv0RtTkR8p5yyxkeyFazSPDhJcgOH06+5OwhkKuu+bUUDA3/BC4n/xC6AJyn5vCvSqsMy13dGuB2wSqswIUygZ9q0SAAK4YW3Mnw2bSrQzuNfgczBiAcdaPxT6hQDdMxbBM11lRdRp68/iJbN0DiLkgZyVPse27dtMK1NrW6aSo4t0VtPAn938RIdzVnIVGCbgyT6k6ltgwTDHc9sEFkhtbZNicDiA/JVzt08ENAu5N46mq5wUUkNb9YmiYnrJGQPSurVWHg7fa50zcXL0AA8Yj+cCOsEsRQM2p8UUOIY6olHhZwCbw== X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(376002)(39860400002)(136003)(396003)(346002)(451199015)(40470700004)(36840700001)(46966006)(110136005)(40480700001)(54906003)(6636002)(186003)(55016003)(316002)(47076005)(5660300002)(426003)(2906002)(2616005)(41300700001)(36756003)(336012)(70586007)(8936002)(6286002)(7696005)(70206006)(26005)(1076003)(16526019)(8676002)(4326008)(40460700003)(83380400001)(478600001)(356005)(82310400005)(7636003)(86362001)(36860700001)(6666004)(82740400003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2022 02:10:20.3196 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0df0d16c-edb6-45f9-125a-08dac2c0b824 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT011.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5988 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org If HW Steering is enabled, Rx queues were configured to receive MARKs when a table with MARK actions was created. After stopping the port, Rx queue configuration is released, but during starting the port the mark flag was not updated in the Rx queue configuration. This patch introduces a reference count on the MARK action and it increases/decreases per template_table create/destroy. When the port is stopped, Rx queue configuration is not cleared if reference count is not zero. Fixes: 3a2f674b6aa8 ("net/mlx5: add queue and RSS HW steering action") Cc: stable@dpdk.org Signed-off-by: Rongwei Liu Acked-by: Matan Azrad --- drivers/net/mlx5/mlx5.c | 3 ++- drivers/net/mlx5/mlx5.h | 1 + drivers/net/mlx5/mlx5_flow.c | 28 +++++++++++++++++++++++++++ drivers/net/mlx5/mlx5_flow.h | 2 ++ drivers/net/mlx5/mlx5_flow_hw.c | 34 ++++++--------------------------- drivers/net/mlx5/mlx5_trigger.c | 7 ++++++- 6 files changed, 45 insertions(+), 30 deletions(-) diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index 1cf6df6049..bef8e35b6d 100644 --- a/drivers/net/mlx5/mlx5.c +++ b/drivers/net/mlx5/mlx5.c @@ -2010,7 +2010,8 @@ mlx5_dev_close(struct rte_eth_dev *dev) * If default mreg copy action is removed at the stop stage, * the search will return none and nothing will be done anymore. */ - mlx5_flow_stop_default(dev); + if (priv->sh->config.dv_flow_en != 2) + mlx5_flow_stop_default(dev); mlx5_traffic_disable(dev); /* * If all the flows are already flushed in the device stop stage, diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index 7d82cb7c8f..02bee5808d 100644 --- a/drivers/net/mlx5/mlx5.h +++ b/drivers/net/mlx5/mlx5.h @@ -1747,6 +1747,7 @@ struct mlx5_priv { uint32_t flex_item_map; /* Map of allocated flex item elements. */ uint32_t nb_queue; /* HW steering queue number. */ struct mlx5_hws_cnt_pool *hws_cpool; /* HW steering's counter pool. */ + uint32_t hws_mark_refcnt; /* HWS mark action reference counter. */ #if defined(HAVE_IBV_FLOW_DV_SUPPORT) || !defined(HAVE_INFINIBAND_VERBS_H) /* Item template list. */ LIST_HEAD(flow_hw_itt, rte_flow_pattern_template) flow_hw_itt; diff --git a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index b3c1336ab8..a0cf677fb0 100644 --- a/drivers/net/mlx5/mlx5_flow.c +++ b/drivers/net/mlx5/mlx5_flow.c @@ -7499,6 +7499,34 @@ mlx5_flow_stop_default(struct rte_eth_dev *dev) flow_rxq_flags_clear(dev); } +/** + * Set rxq flag. + * + * @param[in] dev + * Pointer to the rte_eth_dev structure. + * @param[in] enable + * Flag to enable or not. + */ +void +flow_hw_rxq_flag_set(struct rte_eth_dev *dev, bool enable) +{ + struct mlx5_priv *priv = dev->data->dev_private; + unsigned int i; + + if ((!priv->mark_enabled && !enable) || + (priv->mark_enabled && enable)) + return; + for (i = 0; i < priv->rxqs_n; ++i) { + struct mlx5_rxq_ctrl *rxq_ctrl = mlx5_rxq_ctrl_get(dev, i); + + /* With RXQ start/stop feature, RXQ might be stopped. */ + if (!rxq_ctrl) + continue; + rxq_ctrl->rxq.mark = enable; + } + priv->mark_enabled = enable; +} + /** * Start all default actions for flows. * diff --git a/drivers/net/mlx5/mlx5_flow.h b/drivers/net/mlx5/mlx5_flow.h index 187e9a23bf..955383dd07 100644 --- a/drivers/net/mlx5/mlx5_flow.h +++ b/drivers/net/mlx5/mlx5_flow.h @@ -2516,6 +2516,8 @@ mlx5_get_tof(const struct rte_flow_item *items, enum mlx5_tof_rule_type *rule_type); void flow_hw_resource_release(struct rte_eth_dev *dev); +void +flow_hw_rxq_flag_set(struct rte_eth_dev *dev, bool enable); int flow_dv_action_validate(struct rte_eth_dev *dev, const struct rte_flow_indir_action_conf *conf, const struct rte_flow_action *action, diff --git a/drivers/net/mlx5/mlx5_flow_hw.c b/drivers/net/mlx5/mlx5_flow_hw.c index 7f1ae0fbb0..537bcf141d 100644 --- a/drivers/net/mlx5/mlx5_flow_hw.c +++ b/drivers/net/mlx5/mlx5_flow_hw.c @@ -155,34 +155,6 @@ static const struct rte_flow_item_eth ctrl_rx_eth_bcast_spec = { .type = 0, }; -/** - * Set rxq flag. - * - * @param[in] dev - * Pointer to the rte_eth_dev structure. - * @param[in] enable - * Flag to enable or not. - */ -static void -flow_hw_rxq_flag_set(struct rte_eth_dev *dev, bool enable) -{ - struct mlx5_priv *priv = dev->data->dev_private; - unsigned int i; - - if ((!priv->mark_enabled && !enable) || - (priv->mark_enabled && enable)) - return; - for (i = 0; i < priv->rxqs_n; ++i) { - struct mlx5_rxq_ctrl *rxq_ctrl = mlx5_rxq_ctrl_get(dev, i); - - /* With RXQ start/stop feature, RXQ might be stopped. */ - if (!rxq_ctrl) - continue; - rxq_ctrl->rxq.mark = enable; - } - priv->mark_enabled = enable; -} - /** * Set the hash fields according to the @p rss_desc information. * @@ -462,6 +434,10 @@ __flow_hw_action_template_destroy(struct rte_eth_dev *dev, mlx5_ipool_free(priv->acts_ipool, data->idx); } + if (acts->mark) + if (!__atomic_sub_fetch(&priv->hws_mark_refcnt, 1, __ATOMIC_RELAXED)) + flow_hw_rxq_flag_set(dev, false); + if (acts->jump) { struct mlx5_flow_group *grp; @@ -484,6 +460,7 @@ __flow_hw_action_template_destroy(struct rte_eth_dev *dev, if (acts->mhdr->action) mlx5dr_action_destroy(acts->mhdr->action); mlx5_free(acts->mhdr); + acts->mhdr = NULL; } if (mlx5_hws_cnt_id_valid(acts->cnt_id)) { mlx5_hws_cnt_shared_put(priv->hws_cpool, &acts->cnt_id); @@ -1422,6 +1399,7 @@ __flow_hw_actions_translate(struct rte_eth_dev *dev, goto err; acts->rule_acts[action_pos].action = priv->hw_tag[!!attr->group]; + __atomic_add_fetch(&priv->hws_mark_refcnt, 1, __ATOMIC_RELAXED); flow_hw_rxq_flag_set(dev, true); break; case RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN: diff --git a/drivers/net/mlx5/mlx5_trigger.c b/drivers/net/mlx5/mlx5_trigger.c index 4b821a1076..df7ebf4ee0 100644 --- a/drivers/net/mlx5/mlx5_trigger.c +++ b/drivers/net/mlx5/mlx5_trigger.c @@ -1408,7 +1408,12 @@ mlx5_dev_stop(struct rte_eth_dev *dev) mlx5_mp_os_req_stop_rxtx(dev); rte_delay_us_sleep(1000 * priv->rxqs_n); DRV_LOG(DEBUG, "port %u stopping device", dev->data->port_id); - mlx5_flow_stop_default(dev); + if (priv->sh->config.dv_flow_en == 2) { + if (!__atomic_load_n(&priv->hws_mark_refcnt, __ATOMIC_RELAXED)) + flow_hw_rxq_flag_set(dev, false); + } else { + mlx5_flow_stop_default(dev); + } /* Control flows for default traffic can be removed firstly. */ mlx5_traffic_disable(dev); /* All RX queue flags will be cleared in the flush interface. */