From patchwork Sat Oct 16 09:12:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Xueming(Steven) Li" X-Patchwork-Id: 101887 X-Patchwork-Delegate: rasland@nvidia.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B582BA034F; Sat, 16 Oct 2021 11:14:10 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5FB1E41156; Sat, 16 Oct 2021 11:13:55 +0200 (CEST) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2077.outbound.protection.outlook.com [40.107.92.77]) by mails.dpdk.org (Postfix) with ESMTP id 8ED88411B8 for ; Sat, 16 Oct 2021 11:13:53 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FDVJKS41rCb/Mnsh1Xghr+x4zGKyBODaqfGYtKjJidn2O2+iJ669d6knhfBz8zXx17+79U+Xpsc7/KO0wVsNNIBkgHwGi7AnL2KBj7AmNbCDAg7Aov4M8gcGG47Fxl31eyjlAT9d37PGQwEK027PcjOJ2UrvY+EZz29ToQQncAY+aIlSAs4P8d+xcDFla/Z4IS7o4w2k+hx0TGvC9qNXs05d1+Ndq6uDuAT/EKlBaHYwqOj013PayraI4xR49qVXH2MYWtqejs51FuwzubVds+nBr0aF94+Z1BudwGaYbpfCXLfARy5OSSHq5M1VrVDvwsyfKZdZOJBK22PBuaTBog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PveslJM7XShbl3DxfAuHwTBgl05FXE6B8UR2H6pGADc=; b=FGf3e5OuWUyzTaO5AGzoEyKca2HobtUA4b8eZJMN2JQ9PsiJyXaCtQlK66Ez+eiA7+loSswPo1VCd8eNBKFVBuB0RIHMawYs9094xIG0+GS6Xl7fjPJ63ODyJynKgrLTwY72pqfK2YCOIIqDzL/RwWpnHgJDU/gVkjyygxjvPbOVOpYQ26j8trO9n9FrAvTmZLNUOM9NERpbIAoblosShKkCOxzfJ8i9ol7ImIzxdZKzO0KSfw/Ojj3Ag/3m/bBR8ppP2KCkzjFZyN2UjR4dPeHwtNFrPcbPH1+DO605lv78PLZlMjiwOV3j3gLbssiXveNxKCM2fSJqrsVGCI1f6A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PveslJM7XShbl3DxfAuHwTBgl05FXE6B8UR2H6pGADc=; b=XkKHORvZTmKnTusHD9qCP9hRXU9mQajvIdBHdZ7KB5lUkyYP5JNDsGxYDGBWsHK5bhNFZQZCAv2fi0akEvumEJ9+zElehI1EKbzROe1KcXVkV+dz4xSCSWPGlzd9lYw3dXVkUOPF1Agewp14kCyXVTGlQg6LyxU3+xmZr+6jEXoU1ekAKfMZw5s82fy87xHoQO5oRPZJzBJnxfe7nZWt6Rv9HhbKiLGgVd73qynWWPhUQWxaWpiufcGrbFHp4yvvxT4VPuBqL3GU3L0iwVnFMlJnHLeNQ3oRKYkv02o1+gIcMrdqnlRDSvBEmmzb//7dPCe5TOYrd3Qnaykf2HkGnA== Received: from BN1PR13CA0027.namprd13.prod.outlook.com (2603:10b6:408:e2::32) by BY5PR12MB4179.namprd12.prod.outlook.com (2603:10b6:a03:211::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.17; Sat, 16 Oct 2021 09:13:51 +0000 Received: from BN8NAM11FT067.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e2:cafe::1a) by BN1PR13CA0027.outlook.office365.com (2603:10b6:408:e2::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.5 via Frontend Transport; Sat, 16 Oct 2021 09:13:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by BN8NAM11FT067.mail.protection.outlook.com (10.13.177.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4608.15 via Frontend Transport; Sat, 16 Oct 2021 09:13:50 +0000 Received: from nvidia.com (172.20.187.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Sat, 16 Oct 2021 09:13:47 +0000 From: Xueming Li To: CC: Viacheslav Ovsiienko , , "Lior Margalit" , Matan Azrad , "David Christensen" , Ruifeng Wang , Bruce Richardson , Konstantin Ananyev Date: Sat, 16 Oct 2021 17:12:13 +0800 Message-ID: <20211016091214.1831902-14-xuemingl@nvidia.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211016091214.1831902-1-xuemingl@nvidia.com> References: <20210926111904.237736-1-xuemingl@nvidia.com> <20211016091214.1831902-1-xuemingl@nvidia.com> MIME-Version: 1.0 X-Originating-IP: [172.20.187.6] X-ClientProxiedBy: HQMAIL111.nvidia.com (172.20.187.18) To HQMAIL107.nvidia.com (172.20.187.13) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 049b3258-5980-49b0-eb62-08d9908544d1 X-MS-TrafficTypeDiagnostic: BY5PR12MB4179: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:312; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jhyzx4SIph7+3cQj61xx9MdS08ozqH5SpSfCQU1AX2loPSW1nRSnDThgIAO6PSFNJqXx79jKeck0kMXPvslthg0MKQ4pVj7P2hox1M4c8I7PAoopBdlfXrSGMQomg51eKJkWP5VqbkHGn1rwc0nXXqCDV8uulAuDRmNzsbBTYpCC3y55waZPSjnIZStUj0sM2RsQxLSc95cTBgEonFdVY2rs4h8g7tVneWtNA0PlAnEhc6iU9V6gYfAxBBQOYEpmPDKmbarkYD4nEaXO4+HYBUm5SYeb/F5q+SfZg0ps017O9gGCzBD2AlVAxdXWQC7IPVX4N+J8evN6Onuhqvdzw8rDwwxs/xNV/BZntz9cpKGm9xn4cGhcRmtjxxBt8uR9/Cyzd34IsiM8CMDHOkVx+8nfWboGg9gBM5d8eW99xNkcg90lFO5zI4rSaiCcHqvkpDbRS0XXjNUdym7n8sdf/hXigZuq2fE4uDNMsyzkqiM00IyvpumgPFwl4yPs/xHxK7n+vTdAs1HDTks8yFzgz34DVLBD23unhYBi6PVin+IMb6mAySrUmHDuY7cOp03b1M53xWQrIYwqoSLvg4HFN4IcxsvkvIdIGdDQ/YojDPMDaogEfj6VsPD1QMvMFHj7ldTkmwbbp7xIUAVYjFZqqYEKBVY+EvqRI8Wvvb+oC84E/YTzuVfR1fS2waEZtadp7zrOP4jnYut+W2HZGoTJXw== X-Forefront-Antispam-Report: CIP:216.228.112.34; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:schybrid03.nvidia.com; CAT:NONE; SFS:(4636009)(46966006)(36840700001)(83380400001)(356005)(7696005)(7636003)(6286002)(426003)(2906002)(55016002)(4326008)(36860700001)(336012)(6666004)(2616005)(8676002)(86362001)(82310400003)(36756003)(1076003)(70206006)(316002)(70586007)(16526019)(6916009)(508600001)(186003)(26005)(54906003)(5660300002)(47076005)(8936002); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Oct 2021 09:13:50.7627 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 049b3258-5980-49b0-eb62-08d9908544d1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.112.34]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT067.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4179 Subject: [dpdk-dev] [PATCH v2 13/13] net/mlx5: add shared Rx queue port datapath support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Viacheslav Ovsiienko When receive packet, mlx5 PMD saves mbuf port number from rxq data. To support shared rxq, save port number into RQ context as user index. Received packet resolve port number from CQE user index which derived from RQ context. Legacy Verbs API doesn't support RQ user index setting, still read from rxq port number. Signed-off-by: Xueming Li Signed-off-by: Viacheslav Ovsiienko --- drivers/net/mlx5/mlx5_devx.c | 1 + drivers/net/mlx5/mlx5_rx.c | 1 + drivers/net/mlx5/mlx5_rxq.c | 3 ++- drivers/net/mlx5/mlx5_rxtx_vec_altivec.h | 6 ++++++ drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 12 +++++++++++- drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 8 +++++++- 6 files changed, 28 insertions(+), 3 deletions(-) diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c index 94253047141..11c426eee14 100644 --- a/drivers/net/mlx5/mlx5_devx.c +++ b/drivers/net/mlx5/mlx5_devx.c @@ -277,6 +277,7 @@ mlx5_rxq_create_devx_rq_resources(struct mlx5_rxq_priv *rxq) MLX5_WQ_END_PAD_MODE_NONE; rq_attr.wq_attr.pd = priv->sh->pdn; rq_attr.counter_set_id = priv->counter_set_id; + rq_attr.user_index = rte_cpu_to_be_16(priv->dev_data->port_id); if (rxq_data->shared) /* Create RMP based RQ. */ rxq->devx_rq.rmp = &rxq_ctrl->obj->devx_rmp; /* Create RQ using DevX API. */ diff --git a/drivers/net/mlx5/mlx5_rx.c b/drivers/net/mlx5/mlx5_rx.c index 3017a8da20c..6ee54b820f1 100644 --- a/drivers/net/mlx5/mlx5_rx.c +++ b/drivers/net/mlx5/mlx5_rx.c @@ -707,6 +707,7 @@ rxq_cq_to_mbuf(struct mlx5_rxq_data *rxq, struct rte_mbuf *pkt, { /* Update packet information. */ pkt->packet_type = rxq_cq_to_pkt_type(rxq, cqe, mcqe); + pkt->port = unlikely(rxq->shared) ? cqe->user_index_low : rxq->port_id; if (rxq->rss_hash) { uint32_t rss_hash_res = 0; diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c index 494c9e3517f..250922b0d7a 100644 --- a/drivers/net/mlx5/mlx5_rxq.c +++ b/drivers/net/mlx5/mlx5_rxq.c @@ -179,7 +179,8 @@ rxq_alloc_elts_sprq(struct mlx5_rxq_ctrl *rxq_ctrl) mbuf_init->data_off = RTE_PKTMBUF_HEADROOM; rte_mbuf_refcnt_set(mbuf_init, 1); mbuf_init->nb_segs = 1; - mbuf_init->port = rxq->port_id; + /* For shared queues port is provided in CQE */ + mbuf_init->port = rxq->shared ? 0 : rxq->port_id; if (priv->flags & RTE_PKTMBUF_POOL_F_PINNED_EXT_BUF) mbuf_init->ol_flags = EXT_ATTACHED_MBUF; /* diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h b/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h index 82586f012cb..115320a26f0 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_altivec.h @@ -1189,6 +1189,12 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, /* D.5 fill in mbuf - rearm_data and packet_type. */ rxq_cq_to_ptype_oflags_v(rxq, cqes, opcode, &pkts[pos]); + if (unlikely(rxq->shared)) { + pkts[pos]->port = cq[pos].user_index_low; + pkts[pos + p1]->port = cq[pos + p1].user_index_low; + pkts[pos + p2]->port = cq[pos + p2].user_index_low; + pkts[pos + p3]->port = cq[pos + p3].user_index_low; + } if (rxq->hw_timestamp) { int offset = rxq->timestamp_offset; if (rxq->rt_timestamp) { diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h index 5ff792f4cb5..9e78318129a 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h @@ -787,7 +787,17 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, /* C.4 fill in mbuf - rearm_data and packet_type. */ rxq_cq_to_ptype_oflags_v(rxq, ptype_info, flow_tag, opcode, &elts[pos]); - if (rxq->hw_timestamp) { + if (unlikely(rxq->shared)) { + elts[pos]->port = container_of(p0, struct mlx5_cqe, + pkt_info)->user_index_low; + elts[pos + 1]->port = container_of(p1, struct mlx5_cqe, + pkt_info)->user_index_low; + elts[pos + 2]->port = container_of(p2, struct mlx5_cqe, + pkt_info)->user_index_low; + elts[pos + 3]->port = container_of(p3, struct mlx5_cqe, + pkt_info)->user_index_low; + } + if (unlikely(rxq->hw_timestamp)) { int offset = rxq->timestamp_offset; if (rxq->rt_timestamp) { struct mlx5_dev_ctx_shared *sh = rxq->sh; diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h index adf991f0139..97eb0adc9e6 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h @@ -736,7 +736,13 @@ rxq_cq_process_v(struct mlx5_rxq_data *rxq, volatile struct mlx5_cqe *cq, *err |= _mm_cvtsi128_si64(opcode); /* D.5 fill in mbuf - rearm_data and packet_type. */ rxq_cq_to_ptype_oflags_v(rxq, cqes, opcode, &pkts[pos]); - if (rxq->hw_timestamp) { + if (unlikely(rxq->shared)) { + pkts[pos]->port = cq[pos].user_index_low; + pkts[pos + p1]->port = cq[pos + p1].user_index_low; + pkts[pos + p2]->port = cq[pos + p2].user_index_low; + pkts[pos + p3]->port = cq[pos + p3].user_index_low; + } + if (unlikely(rxq->hw_timestamp)) { int offset = rxq->timestamp_offset; if (rxq->rt_timestamp) { struct mlx5_dev_ctx_shared *sh = rxq->sh;