From patchwork Tue Jun 26 11:33:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongseok Koh X-Patchwork-Id: 41561 X-Patchwork-Delegate: shahafs@mellanox.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C270E1B4BA; Tue, 26 Jun 2018 13:33:54 +0200 (CEST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-eopbgr70082.outbound.protection.outlook.com [40.107.7.82]) by dpdk.org (Postfix) with ESMTP id 3389E1B4B6; Tue, 26 Jun 2018 13:33:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KK365Ug52n8M9VCiBbONcESEFuQEZH0ID19F6D2n58E=; b=VdQxDJQtQGyOVPgolP732Wkxki7oN6biBWCHi5o7UePZZrckLXen9h4NCiD916e23kKWZ583dIQX0ZiyVpqegE7LDswboUxwgM2PxECKY29PDXcpJO2zYFqEORDwOjxDQT5sPtsGstEAld8b9lCI+J3209gCMyf8+E7lCHw27+g= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; Received: from mellanox.com (209.116.155.178) by DB6PR0501MB2039.eurprd05.prod.outlook.com (2603:10a6:4:6::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.906.20; Tue, 26 Jun 2018 11:33:50 +0000 From: Yongseok Koh To: adrien.mazarguil@6wind.com, nelio.laranjeiro@6wind.com Cc: dev@dpdk.org, shahafs@mellanox.com, Yongseok Koh , stable@dpdk.org Date: Tue, 26 Jun 2018 04:33:35 -0700 Message-Id: <20180626113335.6854-1-yskoh@mellanox.com> X-Mailer: git-send-email 2.11.0 MIME-Version: 1.0 X-Originating-IP: [209.116.155.178] X-ClientProxiedBy: BYAPR02CA0003.namprd02.prod.outlook.com (2603:10b6:a02:ee::16) To DB6PR0501MB2039.eurprd05.prod.outlook.com (2603:10a6:4:6::21) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 81941fc2-42e1-42fb-6383-08d5db58b11b X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(8989117)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600026)(711020)(48565401081)(2017052603328)(7153060)(7193020); SRVR:DB6PR0501MB2039; X-Microsoft-Exchange-Diagnostics: 1; DB6PR0501MB2039; 3:m4XnDmw/nV2PspPWW1osz3EgHZNPQtyqnUY51Ae6OkNG7VlasCYLWxuYOwHs3VUEbG6/KlXulu0CbaI+P1YLAtz0gjBuO1fBmq0VGSZi02bYxfu1H3zIjedvE+0Kv+3F0VFoHBhhdj/kGWJ7KBRA2LWC4w8zVwroTnfp8NzRs7obRa/agLfiA6Nc+SwHVVX7VigobTakjyQhqgTU/qZHdVfjeCgrxeU1U4b7jwnxz4zP5ZAWSnUH78e6/6BFoxe4; 25:GT1AfzsWc5E2hg+CTytH3fKg+XgnHgm7wblR1+lBPUXJ2MBaZRla96YCpikZgiuJenZo+zlcr/oSU2eQZbxcctrxFPVz9LtlpOTu7wmO2HJ081vaFpjzK5ZPmY9rFfz8oUyFtLBCAYtR7jJC1sl9ziZ/wgjA89cY7Se8HLHn7i2Hwp7lZcYSdptv6IDNIaKe/Y6QISOPYWp/ITRPcP7MYt6NRGOoF52iaGhiXDFn8LBEi8BZ76uxDenQ+Guv1pArLAGhMCRDr3LSODYqS6TFnrPA1Qh9t/JHI0RnfodpxipgSiY5d5f676i1+Nsj212Wwv3cWFNeon0CkRU7Zll9rQ==; 31:3ZQklmL6A0mONK3rvnIDTYuuFoHu9lOKFnrxJp0H0btV2kjK12h3tjwMkmkFxeuoxsmIjdvvEny/5Uo8wLQ/1+NWek9zD2VOLkW9guPB9ShFooNkJlmab5+L2eYDQcvzvpBmepRolTk2LX6Ow2ZJ+IpKSgXA02vP0BSBLowff9zjKJxC1vU9mLToA/6aepILPDZqne5hzlXNaAVfoLqqPw3KAXC74sZtYCrP25Pc1/M= X-MS-TrafficTypeDiagnostic: DB6PR0501MB2039: X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; DB6PR0501MB2039; 20:g1jeEZeiu9sbYInmLr1MJPx9/lASciMOvDWNPu+k46WtmQ7kiYICk8fdpVQOGiORoaHUo4reWcOe7sz6gfUA96YnhTzOixw5SSiBgUpHoJiMzQDTcsFbhD+OBvnn8zpX4vBvrd4eIu//HEthDtrcQDAFxOG11jBHqkvWstgfOEgA1sp40LXBy14Jm5IgghBAtMEb/vL0ugh60tH40YQQjxTV6/BusH+YPO4KaYf/z5/JujveLi8PdfRjCEbNa98w/9rdrHM8NS2Z7DqnuvljFrgaIbTZo7zo3/G6To9scQkBbaydDtzeD15ndxzWZpPFngn5UlewYdR7dgkBw9CUKR7akm0m47L5MxUlFJJU7X8/7PAgi8hW3/pU9BS8UYcOAGMCFLbAlhRuEVxtFXCuOsZeRvKFSiqmd3LOEDrj6SNkJZiMht7Ozx9FQExLCEIoLmsWCe/1ggJDHoc6gNNkl7IYJBhO7w2daiMTloAkaPHeLLhSO/R/0Gal0ypSPoqE; 4:yBWGvx05DEiYGCsNdG++uSvHjtqLplNIZOkCnjB5nZrMJS5eLuawniGlXKRp85rG0Ls5p/Uzy161eiefuwNcCA8kVxnDtd0oAX0pCZT9wJP9qZzD3HQ34Bbr4Bh8pYO+dJ6DzW1HapJpd+DPruFGVgE0YY1VqCD9YnAYoOqJUYzOT9bB5YMuN6WmrYnsNCl/TfaiP7OclXBL20ln9bD5JPsICKorKDx3rdoRVJJhztILpquMdi8/4SIoo6/LJOcrHmDTJq0knVIddF6pkK+coQ== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(3231254)(944501410)(52105095)(3002001)(93006095)(93001095)(10201501046)(6055026)(149027)(150027)(6041310)(20161123558120)(20161123564045)(20161123562045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011)(7699016); SRVR:DB6PR0501MB2039; BCL:0; PCL:0; RULEID:; SRVR:DB6PR0501MB2039; X-Forefront-PRVS: 071518EF63 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(136003)(376002)(366004)(396003)(346002)(39860400002)(189003)(199004)(69596002)(316002)(55016002)(16586007)(478600001)(5660300001)(47776003)(305945005)(6666003)(575784001)(36756003)(86362001)(50226002)(66066001)(1076002)(48376002)(53936002)(6116002)(386003)(8936002)(7696005)(52116002)(21086003)(3846002)(51416003)(25786009)(956004)(68736007)(1857600001)(2616005)(186003)(16526019)(476003)(26005)(50466002)(105586002)(486006)(7736002)(4326008)(2906002)(97736004)(106356001)(8676002)(81166006)(81156014); DIR:OUT; SFP:1101; SCL:1; SRVR:DB6PR0501MB2039; H:mellanox.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DB6PR0501MB2039; 23:5w8c/WrwrN7rcS39/dmLjIFgrFn/1JFoPT2P8HW?= nS7/3rszecxpZhys6KifBSDxY2DZSph6X56l7vn7nY7O6Gt2r5W6WUu9YF9At0LCBnZXSCCBRZbcVdRMPi7eC2O81cXO98gGp3eUOKXPbWvK69vl1uPNRQiodqQK94V5QQmi8dv328mCr1+jtakXvTlSM5TjOTDVx0PVQLC91EcKjJ5wIh1OMkkfg8aKO4le1IEVxfau8HyCbmt76OtzDmGo4kA3YxdtdUjg9CB0v2dXGN4RN3aPWxpgcsOmPO3DU3Zi+0lZ+lz0A0c96PFe17qnFG9zxYzwNVfwoH5zkkjBCgm3m85WGp93QhFkbrrtnhVMyvH3Mb25tyvv28gmeCyvFEApnBcVTZnJBvONPzCrpYjCTaejLECEIINjiGNgIdxci/ukHGBhWj6NrrSIM3GK5p7LYWo/DKikPjSS/Ot6eUYwFH4ap+vj21tDYR62Tc4PEz0NyWI8JFht7D872VhafUskaqIAR24LLm/hrVfvY8IpMknuiY7lr+5lbhf2y5lGLmUhjo9NofiFaDQXfdxD5W8OKRSXbyQ0BbSknMnUbLzxPPRoa+0AWjr+FoJL75X8fH/4ca5xXLrGV0XW0VPFKkPJP3TkjD23Oyr2bmQAYw6Oz2CvTrpZufuLLCnkIjmeYx9OxFFV24hxhL7MeXtamtXToF6XhR/8wpwcIE8YorhNFasJGmC41ilYDH9qoOy/q8xauBmoUOsdTmXaiL5MiVZgrfsSkwPsBmWt5cY8R8wORaVVvib9IJPd/yFCYIN3KcC1Mj5K2cLY8ZIxYQOKilkJ5szvy4TLRbkZTUCc4wDWPecA1GrX52AvBiyigExbiXJ0t1E209xiLkfk0P0g6ZpEoId4i+zrYZ70JxoCS7MkEeya3MPM5mje4X32GJYzDECoPUBY9IUk6uuWU/JHhZCu7YIojBOpZudwAZK6Q1ysi9HQzw+v/RsxMZRMCamK6J0cbcQMTSVb+3tI8ROSMkhoZtFAyT2RtLeHZRsd4cvfTWPnQKpYOwZMDGwIiHaEk3PL5osDIRA9/rzknKwFVDEh6btQ+GS5ngkabuqWnteJYtJ/cn6wM0AK9RAGH15PbCyG73UUHdP087/sNa0xE11D59FvzfAYLcLgKItO7Y2LwnJ4HrsVztb23Ajib6gg= X-Microsoft-Antispam-Message-Info: zyQg3FYIdwyrqAv4wRD1Uzqa5fQY3Rnpi0tJKYbjY1c+ahNyMTCmee6KPs8umH467yy2lfo2mojFwgyrVJ/ZmdFIYlEsG7AJMTLxEfkZSR4l5Xvg0sa4+QWQnmLtWCWOq8cmwZo7Vin8moZm4vgawi1eiF09UiB+dHicDAmhGnvtSSWoN6x42q9CZJLs6uEHZm5wrlZsOiw1+Ft55GYwyFR8PA0k8lYGAsbxuBVMhVKOk9J/K7Ak1NbhitQylPb2BKFrD9Wgdu6CVNtuRxuJ79Ty/iFUvUSDcSA9yAPSTU61L8P/Phs8zWkUtQ7Nh/QDD5AsaVs0aaAq04a72vJkdtdfznSwzr7VVY/y4mtDR10= X-Microsoft-Exchange-Diagnostics: 1; DB6PR0501MB2039; 6:5aunvm4PkLygn2sfwiCdql/pbeC0tYBGEugGYNsqkpNC91bSSvlECsFObmqDnLLL2K8bZxAKUkaf8llj6XRirxp1u5NzEjL1qCi2F3F33jP1NKwyi3BeT7IDiHkRYtKghY1v1EmdPAHWYiHJuPfsp6NzzhkFLVmgateZL03+hLypE5M55c2XMsU8zbibdvUDxIS3QvhGONVFKaC7/JanvN5R6w/3bTAv+/JpdtAsoPWiGuFEUZXpTFpzXbNckvUAjVyiVZ6m6nkGlGj7gMWcMCamH2L+uHsumzNJFH51XOj6LPn8czsKq6pJYrHedbA25Fh1uUrEqp+YnvMqQ0gNtOHqR1JyawQJXGXJcXpjATDzsiRmoLO0pqybiPNXkCIAHqDi7YJrqhJj7na5w1To1USQuk47zJweRTOite0PUqQipMyWxmabeEHg+/Jj/ZygmlVi4JlLRqegwb13A+R77w==; 5:YCeQ11Vwks5M1TyUlSi4aWkG+NH6EOkoxbM/1ReimXn8DUmyVgt9IIBmbgWA1aC14Hp3fDUhwtra0CXYf/rr9i4cAsokCEzXYuLDLZG6wW6HBbHJb09hZeSe6X85WxtU7CYob5lUmPRiJMZ9oTi5e4aCo4F3mLsNh1+3i5ikV2k=; 24:JSGu8jVS2U/g0B99D1bI9j2DEqYSPQ8/jcBmWGKXyiqZxeJsreMVkRPMQ9sjbwKDnaJretQA/SN2uON2Z1GUzf2X2reuCO/VaHvyTDho5OI= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DB6PR0501MB2039; 7:a5kPjNgGR47ZZ3UZxJZQ4MLkGjLCFAIeSDTs8crWw2s5PXrWFxSqmPDSHRNJ+uYOAOEyf2/iBXoRJPOdxUHP1hSBXlO6gdz+MQ0VjwOeZhUOhentOiO+dwSQ4SUgxz6oFAKtVGGITNjLwduxvzEY4/7LPcD0s7bqEyxNlYpKX8w2ZzUPvZdl2gJ5pJPHW1l37ejVab6k+PmsYriRHxfIjvtOjLJUjnk5XMJnK6otZSL8srdxgtnF4+RnYMnTLnx9 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Jun 2018 11:33:50.6339 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 81941fc2-42e1-42fb-6383-08d5db58b11b X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0501MB2039 Subject: [dpdk-dev] [PATCH] net/mlx5: fix Rx buffer replenishment threshold X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The threshold of buffer replenishment for vectorized Rx burst is a constant value (64). If the size of Rx queue is comparatively small, device could run out of buffers. For example, if the size of Rx queue is 128, buffers are replenished only twice per a wraparound. This can cause jitter in receiving packets and the jitter can cause unnecessary retransmission for TCP connections. Fixes: 6cb559d67b83 ("net/mlx5: add vectorized Rx/Tx burst for x86") Fixes: 570acdb1da8a ("net/mlx5: add vectorized Rx/Tx burst for ARM") Cc: stable@dpdk.org Signed-off-by: Yongseok Koh Acked-by: Shahaf Shuler --- drivers/net/mlx5/mlx5_defs.h | 5 +++-- drivers/net/mlx5/mlx5_rxtx_vec.h | 4 ++-- drivers/net/mlx5/mlx5_rxtx_vec_neon.h | 2 +- drivers/net/mlx5/mlx5_rxtx_vec_sse.h | 2 +- 4 files changed, 7 insertions(+), 6 deletions(-) diff --git a/drivers/net/mlx5/mlx5_defs.h b/drivers/net/mlx5/mlx5_defs.h index 51124cdcc..b52f7dcbe 100644 --- a/drivers/net/mlx5/mlx5_defs.h +++ b/drivers/net/mlx5/mlx5_defs.h @@ -64,10 +64,11 @@ #define MLX5_VPMD_MIN_TXQS 4 /* Threshold of buffer replenishment for vectorized Rx. */ -#define MLX5_VPMD_RXQ_RPLNSH_THRESH 64U +#define MLX5_VPMD_RXQ_RPLNSH_THRESH(n) \ + (RTE_MIN(MLX5_VPMD_RX_MAX_BURST, (unsigned int)(n) >> 2)) /* Maximum size of burst for vectorized Rx. */ -#define MLX5_VPMD_RX_MAX_BURST MLX5_VPMD_RXQ_RPLNSH_THRESH +#define MLX5_VPMD_RX_MAX_BURST 64U /* * Maximum size of burst for vectorized Tx. This is related to the maximum size diff --git a/drivers/net/mlx5/mlx5_rxtx_vec.h b/drivers/net/mlx5/mlx5_rxtx_vec.h index 598dc7519..fb884f92f 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec.h @@ -91,9 +91,9 @@ mlx5_rx_replenish_bulk_mbuf(struct mlx5_rxq_data *rxq, uint16_t n) &((volatile struct mlx5_wqe_data_seg *)rxq->wqes)[elts_idx]; unsigned int i; - assert(n >= MLX5_VPMD_RXQ_RPLNSH_THRESH); + assert(n >= MLX5_VPMD_RXQ_RPLNSH_THRESH(q_n)); assert(n <= (uint16_t)(q_n - (rxq->rq_ci - rxq->rq_pi))); - assert(MLX5_VPMD_RXQ_RPLNSH_THRESH > MLX5_VPMD_DESCS_PER_LOOP); + assert(MLX5_VPMD_RXQ_RPLNSH_THRESH(q_n) > MLX5_VPMD_DESCS_PER_LOOP); /* Not to cross queue end. */ n = RTE_MIN(n - MLX5_VPMD_DESCS_PER_LOOP, q_n - elts_idx); if (rte_mempool_get_bulk(rxq->mp, (void *)elts, n) < 0) { diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h index 71a5eaf23..3ebf34823 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_neon.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_neon.h @@ -739,7 +739,7 @@ rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t pkts_n, * N - (rq_ci - rq_pi) := # of buffers consumed (to be replenished). */ repl_n = q_n - (rxq->rq_ci - rxq->rq_pi); - if (repl_n >= MLX5_VPMD_RXQ_RPLNSH_THRESH) + if (repl_n >= MLX5_VPMD_RXQ_RPLNSH_THRESH(q_n)) mlx5_rx_replenish_bulk_mbuf(rxq, repl_n); /* See if there're unreturned mbufs from compressed CQE. */ rcvd_pkt = rxq->cq_ci - rxq->rq_pi; diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h index 3e985d61d..f0d888831 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_sse.h +++ b/drivers/net/mlx5/mlx5_rxtx_vec_sse.h @@ -724,7 +724,7 @@ rxq_burst_v(struct mlx5_rxq_data *rxq, struct rte_mbuf **pkts, uint16_t pkts_n, * N - (rq_ci - rq_pi) := # of buffers consumed (to be replenished). */ repl_n = q_n - (rxq->rq_ci - rxq->rq_pi); - if (repl_n >= MLX5_VPMD_RXQ_RPLNSH_THRESH) + if (repl_n >= MLX5_VPMD_RXQ_RPLNSH_THRESH(q_n)) mlx5_rx_replenish_bulk_mbuf(rxq, repl_n); /* See if there're unreturned mbufs from compressed CQE. */ rcvd_pkt = rxq->cq_ci - rxq->rq_pi;