From patchwork Sat Oct 7 22:10:35 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yongseok Koh X-Patchwork-Id: 29879 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 82B622C2A; Sun, 8 Oct 2017 00:10:56 +0200 (CEST) Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40041.outbound.protection.outlook.com [40.107.4.41]) by dpdk.org (Postfix) with ESMTP id 4EB662C24; Sun, 8 Oct 2017 00:10:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=3ivp6XEo7q4yfDfHoTdfPOaoxPfWftgfuyVzfl2W4D4=; b=fzcvwe5YYPOHzlTqdKE0CeuAggRvcX8PcCtuCPRVDYaY0MCaubLOg87nnbAB5dQLOVMMGArXag7dVzTPR6lRu1sfBkWi4zG3hxP+l9Yt3LtpI+pLqjvqu2V2YQcw/MykQIbcalkaClwmOwib3bYW4Ub2MoHPuCcqsBsUfNz6EZ4= Received: from mellanox.com (209.116.155.178) by AM5PR0501MB2036.eurprd05.prod.outlook.com (2603:10a6:203:1a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Sat, 7 Oct 2017 22:10:48 +0000 From: Yongseok Koh To: adrien.mazarguil@6wind.com, nelio.laranjeiro@6wind.com Cc: martin.weiser@allegro-packets.com, dev@dpdk.org, Yongseok Koh , stable@dpdk.org Date: Sat, 7 Oct 2017 15:10:35 -0700 Message-Id: <09b07aed3dd9a25ae6ee575a725924dd66f04e79.1507413575.git.yskoh@mellanox.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171006221948.24238-1-yskoh@mellanox.com> References: <20171006221948.24238-1-yskoh@mellanox.com> MIME-Version: 1.0 X-Originating-IP: [209.116.155.178] X-ClientProxiedBy: CY4PR1201CA0016.namprd12.prod.outlook.com (2603:10b6:910:16::26) To AM5PR0501MB2036.eurprd05.prod.outlook.com (2603:10a6:203:1a::22) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9c0e30a1-1393-458e-99a7-08d50dd0449f X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(48565401081)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:AM5PR0501MB2036; X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2036; 3:V/EHRJXPMWhpxnQykAwYkLb3amBFJt5cDrrmHdsiUkA8pnEmLJD/MF9LblTIKh4T2L0jZ78/iWA2vBsBzbx2uIQ1tduMcyz1oxfscHCVYulnL1cI5i5yf6bD5+AJ2N6LDxDpxDE1SbFKcbpKBztDnYEl+AiYtxPSiRaEckA1WFbs0qiujqEW5OCIciY8Tm7LYlI5yQ80u9XxaU+JxJJytpHz/0nM/Z/WT4NCZzeWVGB4ynjYvyR3orgHLQcSeTz2; 25:X9o0AuIwBnLmUM6fn8ab54Pt0HheRCTeqPCpuqDsYuAvGBGG5Tpp2tVwEis60+QTgpSVaH8zHyGN2zsO4qFhg62zEBCSROzTnBzkqzx+sgTl3GsrkgnL3jIDXkTghXKT3UnkZfXr+fvcRGI7VuG1l9jpRb7mf4oHnAuhl50soyay4ez5NK9rLfMiuL6DmxAv7KhBPtSRpUiOoxnUEwCkT5dCQRCmUxyDHKfPMLg5QUzM1BYCtb92xENlvEF2PJZZ9Pm0JsOY7+Zrz0VihShCblYd298m7zDEwi/Mzh59w5f6gwmXqubnHbe5NlYuv4Qo4tXozxZ3FBqcmsMlNdccJA==; 31:PC2ky4qI4OXsZJm5bp49L6ywRBMRXR+DNSPUnT3qR/82FCxiy3BfH7icYZkJKlWow31HYwDQKBszOAiEJOo73rJ+OwQnRfvMgsyqqyMaGGb4uOlVQzQdyVgYbhzkQnZYcHznAZQITT0t5LsicU+IT+KojtOIfHCgQslPGqGDqkTe4WCkrEU8qmkD2Rou0GpLP0e+b11hZ6gAmYrKz6BAl6hun2CnGNVoyCruMJR5wgE= X-MS-TrafficTypeDiagnostic: AM5PR0501MB2036: Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; X-LD-Processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2036; 20:ZF6dF3C+dwT1q/uwAn3ECg/RPCxmyqAK5M7y34rLDAyWoFKxCTAipUdrafBEKycQidutSl8OcTMsscVnY3kS6ZYripxm6Qt//thM2OTE8W2XAvJ5RMHlddFew7l1nHCTYT4hin1xDpEkXAjqozzymC3wTzykazJuJbNmDklg8zBuBdl+CHaNUT1oY8HISgLJsogrb5G3V5meVp9FEUgVNHE0yE+J0cp24h719VYf6djDO0EdvFm4kbReYyjt6aHvUAFh3yeRs5NWkJYUKt6uOYip6m5IvBwpfk1VghRuAnxLJIJ2A4bv+Q50Wjmhc0zTRKUxiskwWDYSywwGR0pgPw6HAWXAyvhqNjSh+23Ejc4r9CVzFUk1STgFQiyjgnfXx0IsDisrDyhCy/JackfzThBkq6FK9atjzjOAt/juHjEi8gb4zfgmPwEs5ezvJ5wiMwC+TKyllLung1TxePJvvwtUxz981VE4PQsAG+i9cKFZVxcvnEWADPCdtAVSoeiu; 4:g1tFV8zXw8fUYHxN0IR8GPnztnPRROhkUYktZFkjKk5ia1q2NA9tcB5UDOOd8BtFvCeJASiUwjnQLR7ZqcJpvokhidgunekHzbJogxFVxsEQlUeRRaE891Np19/Z0rddh08g9CKs57TZdpDy8qNOVzA2Mvb5ap5dg5g3JTlbBqLadFCCvuvg1taOHmWiPXRcISGkoW9oF2CVRkqs119cXDu0+9xo+9t1AA29kWJlNmVyAcovH+7Jdsd9KSZfh4isxF/qZnprNgH8U1/97K04T0xeGlQZKFsFV7c8uLQ16bI= X-Exchange-Antispam-Report-Test: UriScan:(788757137089); X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(100000703101)(100105400095)(3002001)(6055026)(6041248)(20161123560025)(20161123555025)(20161123562025)(20161123564025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:AM5PR0501MB2036; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:AM5PR0501MB2036; X-Forefront-PRVS: 045315E1EE X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(346002)(376002)(199003)(189002)(66066001)(36756003)(76176999)(50986999)(47776003)(86362001)(5003940100001)(33646002)(48376002)(106356001)(5660300001)(97736004)(50466002)(118296001)(105586002)(50226002)(101416001)(81156014)(81166006)(7736002)(2906002)(478600001)(53936002)(25786009)(68736007)(189998001)(305945005)(6666003)(6116002)(3846002)(8676002)(316002)(55016002)(8936002)(16586007)(16526018)(2950100002)(4326008)(69596002)(21086003); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0501MB2036; H:mellanox.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM5PR0501MB2036; 23:8dHYh00Tb28nkeSazt28zSNWomkjJ/8sEkntV3l?= mkH1FuHMoLHRdgkjDJnIoCtdDD8rS+gS55nQqs/0flojE41VU+9nBC92DnBXSE50Diq2O6WMjbttimcz8ld2YhyuqbnxaWyisJLuMUEoMKGEuFb9Hwm6Fwk42cWF/NSacPNkfeiRf/KlOT4RBD6DjAkfMOmRqEVC5DVXukpRKmqQTzTKQDItNZ4BhGIZWX03p1+EPrHPKMYq20zTZ4nSNXjFM5Oh7V3sfovnNXlf/GUS0jq7ypApIlxfoq8S3iNiUUhZP2mTDCS4bV0AawfWferNuYxxNcMQv+aNDnjvZG9GwBQQafUGi2/CaHJdJlqOwRQZXTXmA9L3T0qvZ3y8EuWGnPMIHt5kEe7izcSqOw2pTlJS4PXZZqpNDBTaiHKvmyL4FK4Chtbi/xjquZNn3lQxuIvx/XMesWrA3ajscmxiDUdr0g7t3ZZ+/3M9kdO5dLr5aqa+YMJLZJzPOb8iSgO5on3UKq+E6HSn4NSXpSHqlqld5nXx4r9dCbiRNkh8jjsbcyMP8xzaTktLmfljZkU42/O5SEuP3lJOOmyHuZWtYswl33xwG5JAfmjDP7IIqhCSmJDecIx7wUjr8Q5R9KjHNNoMDlGf5X60ABwH6lt0OwdmINJXiLIiu2DZMaZZbYHdYpNZOq1KF+MOk7zcizsY1sg4gtFuVa8nWRR/woCvTX6hFUwhPKY/QFPS2r6uT3CJbqqs1vbvsj1lGngCRrtqJhqIHt+etRRRwgnGO5Kg2dYY+Y042V2WrfasbWU4b1pi3H4q6QCjAQ14lnqW1Q2Wjjs7f4O6Cy1YNhiiZphDivMXdF0Ggey7t7fnHtG5ukVSAkC7ArTldMe1y6rxDJGC6iYfpr04qNZijvSLcTjimQaXkSf2PDafed4v85++gi7uYGWJG1t07XBlD4aNYvJiqvSCAiqsHk2DivJJLTD5RbPtofbsu6dAVFSefagFldHSwAGdO43Te+BHgFEBgrn8P3qmRpTCBomQGlEnElq4UwKRlNnuaN53oZlv1g5/e2H8jGofTN2PkUCsuPgDSMZ3E X-Microsoft-Exchange-Diagnostics: 1; AM5PR0501MB2036; 6:9TXO0WebcwFg6FhvKoKDuvGOcxYU7yh8Nhvt+yqKlNDOxvcguBamjEm01sraq9bZ5cy6DhD1UDPa2t6in+2gVZOpTKWp+Nem+sX+S4eBW5oo1YJj6AH2l+qNOIQOBBxt75j1XC3k9GVd36jxVPBFpWuNkBgIq81+0d3ppgLdJ2QNjQextAkAY9lO/x5oISxWTzZNEnRzvOJ1P2YOcwI9XQjtewqBGsU0UAmBc0l5fPqziKSKSkpK6i2Q4xcQW2L+J35P9myEuMwNZN3nPFkWjoN7bDE3dJOCV78xPu9TR+2lwJBPiTQ4QDA3CZjfdpXwWQPCA1I9yVz5uPM4cZYhSA==; 5:aJTGZZ4IUhc+DiTYYFOLvuhX1eo2sKAeqVEFvWKQlaDIoei6Qa7peo3L4o5/pLHkZ7PiylJLnQsE9o9gIyxn9bR+hsV1QWbwS/i7giW67qUG9wSmtAG3Qna+ji/GzWFpuHc60DBIBbElqyqvScuAGg==; 24:5c3NIS2fn6VxdjrTM694mHI2uXELjBr7FkYLTqWRk691O5gn6fc3CPhWm5hoU7ooYaQtfYicE3dJmhCcTFWjWJb1vAgs2EygJuP/lRRs7Hw=; 7:NlT/uotN6z3LDw/YMGHbnLNN2pQEuBdWKqU05WIFqJuMmBwPFPTYOgWqOMkIuEe3u3PZyxJ/qTbbv+9q2pV8AP98x1AzPMxn2lJmgli1Kl3Hz/umRDbF4CuNBRlyLHUC+x5pWV+SnyiFA/AILzQt/SjDkN5zyOWg593+60McEphmCaiPwyhgLUJP9XfP7oSPcoyCRoepnB01b+6gRuZFnFgBUFXL1ol7Mw8R/ngYfM4= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Oct 2017 22:10:48.5492 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0501MB2036 Subject: [dpdk-dev] [PATCH v2] net/mlx5: fix deadlock due to buffered slots in Rx SW ring X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When replenishing Rx ring, there're always buffered slots reserved between consumed entries and HW owned entries. These have to be filled with fake mbufs to protect from possible overflow rather than optimistically expecting successful replenishment which can cause deadlock with small-sized queue. Fixes: 951649d21edf ("net/mlx5: fix overflow of Rx SW ring") Cc: stable@dpdk.org Reported-by: Martin Weiser Signed-off-by: Yongseok Koh --- v2: * Replace vector st/ld with regular assignment as performance gain is negligible for the short loop. This is to make the function shareable with ARM NEON. This patch actually reverts "net/mlx5: fix overflow of Rx SW ring". If possible, these two can be squashed by disregrading the old one. drivers/net/mlx5/mlx5_rxtx_vec_sse.c | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/drivers/net/mlx5/mlx5_rxtx_vec_sse.c b/drivers/net/mlx5/mlx5_rxtx_vec_sse.c index 075dce908..4de8b7333 100644 --- a/drivers/net/mlx5/mlx5_rxtx_vec_sse.c +++ b/drivers/net/mlx5/mlx5_rxtx_vec_sse.c @@ -548,7 +548,7 @@ rxq_replenish_bulk_mbuf(struct rxq *rxq, uint16_t n) { const uint16_t q_n = 1 << rxq->elts_n; const uint16_t q_mask = q_n - 1; - const uint16_t elts_idx = rxq->rq_ci & q_mask; + uint16_t elts_idx = rxq->rq_ci & q_mask; struct rte_mbuf **elts = &(*rxq->elts)[elts_idx]; volatile struct mlx5_wqe_data_seg *wq = &(*rxq->wqes)[elts_idx]; unsigned int i; @@ -566,6 +566,10 @@ rxq_replenish_bulk_mbuf(struct rxq *rxq, uint16_t n) wq[i].addr = rte_cpu_to_be_64((uintptr_t)elts[i]->buf_addr + RTE_PKTMBUF_HEADROOM); rxq->rq_ci += n; + /* Prevent overflowing into consumed mbufs. */ + elts_idx = rxq->rq_ci & q_mask; + for (i = 0; i < MLX5_VPMD_DESCS_PER_LOOP; ++i) + (*rxq->elts)[elts_idx + i] = &rxq->fake_mbuf; rte_wmb(); *rxq->rq_db = rte_cpu_to_be_32(rxq->rq_ci); } @@ -639,13 +643,6 @@ rxq_cq_decompress_v(struct rxq *rxq, RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, hash) != offsetof(struct rte_mbuf, rx_descriptor_fields1) + 12); /* - * Not to overflow elts array. Decompress next time after mbuf - * replenishment. - */ - if (unlikely(mcqe_n + MLX5_VPMD_DESCS_PER_LOOP > - (uint16_t)(rxq->rq_ci - rxq->cq_ci))) - return; - /* * A. load mCQEs into a 128bit register. * B. store rearm data to mbuf. * C. combine data from mCQEs with rx_descriptor_fields1. @@ -1035,10 +1032,8 @@ rxq_burst_v(struct rxq *rxq, struct rte_mbuf **pkts, uint16_t pkts_n) } elts_idx = rxq->rq_pi & q_mask; elts = &(*rxq->elts)[elts_idx]; - pkts_n = RTE_MIN(pkts_n - rcvd_pkt, - (uint16_t)(rxq->rq_ci - rxq->cq_ci)); - /* Not to overflow pkts/elts array. */ - pkts_n = RTE_ALIGN_FLOOR(pkts_n, MLX5_VPMD_DESCS_PER_LOOP); + /* Not to overflow pkts array. */ + pkts_n = RTE_ALIGN_FLOOR(pkts_n - rcvd_pkt, MLX5_VPMD_DESCS_PER_LOOP); /* Not to cross queue end. */ pkts_n = RTE_MIN(pkts_n, q_n - elts_idx); if (!pkts_n)