From patchwork Tue Jan 14 00:39:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rasesh Mody X-Patchwork-Id: 64593 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id B498BA04FD; Tue, 14 Jan 2020 01:39:40 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3401B1C1D2; Tue, 14 Jan 2020 01:39:40 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 0FC5D1C1CB; Tue, 14 Jan 2020 01:39:38 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00E0Qg7l020730; Mon, 13 Jan 2020 16:39:38 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : mime-version : content-type; s=pfpt0818; bh=A1DkO320kLCFA+ZW59RmSQXxO0ih0ORnNbuJdSW+uvM=; b=ljcTaoDSbufTPQ415uTnl7maSK3pmDwkDYaST5RgGtyMl8vXllhk8NdMBX9ZJw/d1hC9 JRFrikjVIsi1FCJmRCYrHZVWv3ZwLSEy15Gbwy/pY8cE3trp19gN3pS3rWuDuVYCMSYj Hpk1hXdZdZ0l22Lj8mySP2Qu4nQB+mG4p/RVgPNJS+mZfVDsOLV9S6bGPuQUKNnaapye rYMho+dwODTygq2VH4noDgnzd8zclh0wqVUNLplptCk3ZqTE8iqN+6ko7NxwX+Y7SrAy W6mxz4pD4L7+qz+D7zvegZPj0JAEACys0kGkHXQy1j3T0z4qCDQXMlNz0kktMg0/akjx yA== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0a-0016f401.pphosted.com with ESMTP id 2xfckur0e1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 13 Jan 2020 16:39:38 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 13 Jan 2020 16:39:36 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 13 Jan 2020 16:39:36 -0800 Received: from irv1user08.caveonetworks.com (unknown [10.104.116.105]) by maili.marvell.com (Postfix) with ESMTP id 9A1943F703F; Mon, 13 Jan 2020 16:39:36 -0800 (PST) Received: (from rmody@localhost) by irv1user08.caveonetworks.com (8.14.4/8.14.4/Submit) id 00E0daPa017774; Mon, 13 Jan 2020 16:39:36 -0800 X-Authentication-Warning: irv1user08.caveonetworks.com: rmody set sender to rmody@marvell.com using -f From: Rasesh Mody To: , , CC: Rasesh Mody , , Date: Mon, 13 Jan 2020 16:39:18 -0800 Message-ID: <20200114003920.17705-1-rmody@marvell.com> X-Mailer: git-send-email 2.18.0 MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-13_08:2020-01-13, 2020-01-13 signatures=0 Subject: [dpdk-dev] [PATCH 1/3] net/bnx2x: fix to use required mem barriers in Rx path X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" When handling RX completion queue PMD is not using required read/write barriers before reading completion queue element (CQE) indices, updating/writing hardware consumer and producer. This patch adds appropriate read/write memory barriers in places which are required by driver and adapter to read or update indices. Fixes: 540a211084a7 ("bnx2x: driver core") Cc: stable@dpdk.org Signed-off-by: Rasesh Mody --- drivers/net/bnx2x/bnx2x.c | 5 +++++ drivers/net/bnx2x/bnx2x_rxtx.c | 22 ++++++++++++++++++++++ 2 files changed, 27 insertions(+) diff --git a/drivers/net/bnx2x/bnx2x.c b/drivers/net/bnx2x/bnx2x.c index ed31335ac..9c5e7995d 100644 --- a/drivers/net/bnx2x/bnx2x.c +++ b/drivers/net/bnx2x/bnx2x.c @@ -1255,6 +1255,11 @@ static uint8_t bnx2x_rxeof(struct bnx2x_softc *sc, struct bnx2x_fastpath *fp) return 0; } + /* Add memory barrier as status block fields can change. This memory + * barrier will flush out all the read/write operations to status block + * generated before the barrier. It will ensure stale data is not read + */ + mb(); /* CQ "next element" is of the size of the regular element */ hw_cq_cons = le16toh(*fp->rx_cq_cons_sb); if (unlikely((hw_cq_cons & USABLE_RCQ_ENTRIES_PER_PAGE) == diff --git a/drivers/net/bnx2x/bnx2x_rxtx.c b/drivers/net/bnx2x/bnx2x_rxtx.c index ae97dfee3..b52f023ea 100644 --- a/drivers/net/bnx2x/bnx2x_rxtx.c +++ b/drivers/net/bnx2x/bnx2x_rxtx.c @@ -324,11 +324,22 @@ bnx2x_upd_rx_prod_fast(struct bnx2x_softc *sc, struct bnx2x_fastpath *fp, struct ustorm_eth_rx_producers rx_prods = { 0 }; uint32_t *val = NULL; + /* Update producers */ rx_prods.bd_prod = rx_bd_prod; rx_prods.cqe_prod = rx_cq_prod; + /* + * Make sure that the BD and SGE data is updated before updating the + * producers since FW might read the BD/SGE right after the producer + * is updated. + * The following barrier is also mandatory since FW will assumes BDs + * must have buffers. + */ + wmb(); val = (uint32_t *)&rx_prods; REG_WR(sc, fp->ustorm_rx_prods_offset, val[0]); + + wmb(); /* keep prod updates ordered */ } static uint16_t @@ -346,6 +357,11 @@ bnx2x_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) uint16_t len, pad; struct rte_mbuf *rx_mb = NULL; + /* Add memory barrier as status block fields can change. This memory + * barrier will flush out all the read/write operations to status block + * generated before the barrier. It will ensure stale data is not read + */ + mb(); hw_cq_cons = le16toh(*fp->rx_cq_cons_sb); if ((hw_cq_cons & USABLE_RCQ_ENTRIES_PER_PAGE) == USABLE_RCQ_ENTRIES_PER_PAGE) { @@ -357,6 +373,12 @@ bnx2x_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) sw_cq_cons = rxq->rx_cq_head; sw_cq_prod = rxq->rx_cq_tail; + /* + * Memory barrier necessary as speculative reads of the Rx + * buffer can be ahead of the index in the status block + */ + rmb(); + if (sw_cq_cons == hw_cq_cons) return 0; From patchwork Tue Jan 14 00:39:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rasesh Mody X-Patchwork-Id: 64594 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id E0169A04FD; Tue, 14 Jan 2020 01:39:48 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EA4C91C204; Tue, 14 Jan 2020 01:39:46 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 660611C1F1; Tue, 14 Jan 2020 01:39:45 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00E0RDvR021193; Mon, 13 Jan 2020 16:39:44 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0818; bh=TZzh2EsIcbKURHd635pfUNS6XV2Wq5/7TXvT2TNLtJw=; b=PRiw6CslCP/76kn97TEkqbU08z6NwtEVK30n6Gv4gylfUKcK0Qx8gizb5HoXf47lBd8A qRp0Sm2cf6SIhPRsrXkvBhfTlm+KfQROInUrS0avLy53bQI9RWweVti1gH2TE1paKPFZ mce3nVG7bXkzN60aJYzCAZ5cH275WDW8+jfmbH1rDy//7QhDUnSOIO2Xd2J7WiKeojNd JPE5pol0YN/0nO8BKQjgKFSuFepHJTMSP9Ox7EdcZx6tsg49Y+T01Y6Tzny8zt8EEt+i 8MdofGpf89ZmkAHgeECWbxcLnz0cE8Fh+JFAEsCzuOd08Jm2+zrtgHm+kXCLBseBqN+5 jw== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0a-0016f401.pphosted.com with ESMTP id 2xfckur0e9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 13 Jan 2020 16:39:44 -0800 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 13 Jan 2020 16:39:43 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 13 Jan 2020 16:39:43 -0800 Received: from irv1user08.caveonetworks.com (unknown [10.104.116.105]) by maili.marvell.com (Postfix) with ESMTP id 2A0FC3F703F; Mon, 13 Jan 2020 16:39:43 -0800 (PST) Received: (from rmody@localhost) by irv1user08.caveonetworks.com (8.14.4/8.14.4/Submit) id 00E0dgWC017784; Mon, 13 Jan 2020 16:39:42 -0800 X-Authentication-Warning: irv1user08.caveonetworks.com: rmody set sender to rmody@marvell.com using -f From: Rasesh Mody To: , , CC: Rasesh Mody , , Date: Mon, 13 Jan 2020 16:39:19 -0800 Message-ID: <20200114003920.17705-2-rmody@marvell.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20200114003920.17705-1-rmody@marvell.com> References: <20200114003920.17705-1-rmody@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-13_08:2020-01-13, 2020-01-13 signatures=0 Subject: [dpdk-dev] [PATCH 2/3] net/bnx2x: fix reset of scan FP flag X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The fastpath task queue handler resets the fastpath scan flag unconditionally, this patch changes that to reset the flag only if it was set. Fixes: 08a6e472c3d7 ("net/bnx2x: fix packet drop") Cc: stable@dpdk.org Signed-off-by: Rasesh Mody --- drivers/net/bnx2x/bnx2x.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/net/bnx2x/bnx2x.c b/drivers/net/bnx2x/bnx2x.c index 9c5e7995d..d38da4f60 100644 --- a/drivers/net/bnx2x/bnx2x.c +++ b/drivers/net/bnx2x/bnx2x.c @@ -4582,10 +4582,10 @@ static void bnx2x_handle_fp_tq(struct bnx2x_fastpath *fp) bnx2x_handle_fp_tq(fp); return; } + /* We have completed slow path completion, clear the flag */ + rte_atomic32_set(&sc->scan_fp, 0); } - /* Assuming we have completed slow path completion, clear the flag */ - rte_atomic32_set(&sc->scan_fp, 0); bnx2x_ack_sb(sc, fp->igu_sb_id, USTORM_ID, le16toh(fp->fp_hc_idx), IGU_INT_ENABLE, 1); } From patchwork Tue Jan 14 00:39:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rasesh Mody X-Patchwork-Id: 64595 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C3EEDA04FD; Tue, 14 Jan 2020 01:39:55 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id BCFE11C1F8; Tue, 14 Jan 2020 01:39:49 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 5579F1D160; Tue, 14 Jan 2020 01:39:48 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00E0QdhM020719; Mon, 13 Jan 2020 16:39:47 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0818; bh=UclS+hZNQ0UG7qbCBf7P1AULPLln4zL672WgpUt+zog=; b=b464/FXcwd+B1CcuT+4DvZMbXjFKzmLLMa0MGIf7/gLG5V+6p4ekfe0qehzadYpdP/a4 0YEes/LrQrojuWqc+LbT/n+8UJmy5IyONgtvRmDSMMysYksSVehQP19WynY/JnVOyyEx orZwToGYVZ1Eg5GDB2/iWUjHbpmpw7Ib4GfyAmL6P0I9G+WCfKSSt1epjnTMOWaPY0EO K/oJvwsIkb7qmusxUg2uEZf7cybYHRkKg8GaWMpZvKw2YXoksJHjb8sn9N4p3KCAkj4z d6eOhYGsqmj5Gfq79IYodCJFXq2EBf0FMfM5HjDGd464ONFfGrJ1piJMyfjWFcB/VMsX HA== Received: from sc-exch02.marvell.com ([199.233.58.182]) by mx0a-0016f401.pphosted.com with ESMTP id 2xfckur0ec-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 13 Jan 2020 16:39:47 -0800 Received: from SC-EXCH03.marvell.com (10.93.176.83) by SC-EXCH02.marvell.com (10.93.176.82) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 13 Jan 2020 16:39:45 -0800 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 13 Jan 2020 16:39:45 -0800 Received: from irv1user08.caveonetworks.com (unknown [10.104.116.105]) by maili.marvell.com (Postfix) with ESMTP id C92553F703F; Mon, 13 Jan 2020 16:39:45 -0800 (PST) Received: (from rmody@localhost) by irv1user08.caveonetworks.com (8.14.4/8.14.4/Submit) id 00E0djfT017787; Mon, 13 Jan 2020 16:39:45 -0800 X-Authentication-Warning: irv1user08.caveonetworks.com: rmody set sender to rmody@marvell.com using -f From: Rasesh Mody To: , , CC: Rasesh Mody , , Date: Mon, 13 Jan 2020 16:39:20 -0800 Message-ID: <20200114003920.17705-3-rmody@marvell.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20200114003920.17705-1-rmody@marvell.com> References: <20200114003920.17705-1-rmody@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138, 18.0.572 definitions=2020-01-13_08:2020-01-13, 2020-01-13 signatures=0 Subject: [dpdk-dev] [PATCH 3/3] net/bnx2x: fix to sync fastpath Rx queue access X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" PMD handles fast path completions in the Rx handler and control path completions in the interrupt handler. They both are processing completions from the same fastpath completion queue. There is a potential for race condition when these two paths are processing the completions from the same queue and trying to updating Rx Producer. Add a fastpath Rx lock between these two paths to close this race. Fixes: 540a211084a7 ("bnx2x: driver core") Cc: stable@dpdk.org Signed-off-by: Rasesh Mody --- drivers/net/bnx2x/bnx2x.c | 12 ++++++++++++ drivers/net/bnx2x/bnx2x.h | 3 +++ drivers/net/bnx2x/bnx2x_rxtx.c | 8 +++++++- 3 files changed, 22 insertions(+), 1 deletion(-) diff --git a/drivers/net/bnx2x/bnx2x.c b/drivers/net/bnx2x/bnx2x.c index d38da4f60..7ea98b936 100644 --- a/drivers/net/bnx2x/bnx2x.c +++ b/drivers/net/bnx2x/bnx2x.c @@ -1167,6 +1167,10 @@ static int bnx2x_has_rx_work(struct bnx2x_fastpath *fp) if (unlikely((rx_cq_cons_sb & MAX_RCQ_ENTRIES(rxq)) == MAX_RCQ_ENTRIES(rxq))) rx_cq_cons_sb++; + + PMD_RX_LOG(DEBUG, "hw CQ cons = %d, sw CQ cons = %d", + rx_cq_cons_sb, rxq->rx_cq_head); + return rxq->rx_cq_head != rx_cq_cons_sb; } @@ -1249,9 +1253,12 @@ static uint8_t bnx2x_rxeof(struct bnx2x_softc *sc, struct bnx2x_fastpath *fp) uint16_t bd_cons, bd_prod, bd_prod_fw, comp_ring_cons; uint16_t hw_cq_cons, sw_cq_cons, sw_cq_prod; + rte_spinlock_lock(&(fp)->rx_mtx); + rxq = sc->rx_queues[fp->index]; if (!rxq) { PMD_RX_LOG(ERR, "RX queue %d is NULL", fp->index); + rte_spinlock_unlock(&(fp)->rx_mtx); return 0; } @@ -1326,9 +1333,14 @@ static uint8_t bnx2x_rxeof(struct bnx2x_softc *sc, struct bnx2x_fastpath *fp) rxq->rx_cq_head = sw_cq_cons; rxq->rx_cq_tail = sw_cq_prod; + PMD_RX_LOG(DEBUG, "BD prod = %d, sw CQ prod = %d", + bd_prod_fw, sw_cq_prod); + /* Update producers */ bnx2x_update_rx_prod(sc, fp, bd_prod_fw, sw_cq_prod); + rte_spinlock_unlock(&(fp)->rx_mtx); + return sw_cq_cons != hw_cq_cons; } diff --git a/drivers/net/bnx2x/bnx2x.h b/drivers/net/bnx2x/bnx2x.h index 3383c7675..1dbc98197 100644 --- a/drivers/net/bnx2x/bnx2x.h +++ b/drivers/net/bnx2x/bnx2x.h @@ -360,6 +360,9 @@ struct bnx2x_fastpath { /* pointer back to parent structure */ struct bnx2x_softc *sc; + /* Used to synchronize fastpath Rx access */ + rte_spinlock_t rx_mtx; + /* status block */ struct bnx2x_dma sb_dma; union bnx2x_host_hc_status_block status_block; diff --git a/drivers/net/bnx2x/bnx2x_rxtx.c b/drivers/net/bnx2x/bnx2x_rxtx.c index b52f023ea..c8bb202d6 100644 --- a/drivers/net/bnx2x/bnx2x_rxtx.c +++ b/drivers/net/bnx2x/bnx2x_rxtx.c @@ -357,6 +357,8 @@ bnx2x_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) uint16_t len, pad; struct rte_mbuf *rx_mb = NULL; + rte_spinlock_lock(&(fp)->rx_mtx); + /* Add memory barrier as status block fields can change. This memory * barrier will flush out all the read/write operations to status block * generated before the barrier. It will ensure stale data is not read @@ -379,8 +381,10 @@ bnx2x_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) */ rmb(); - if (sw_cq_cons == hw_cq_cons) + if (sw_cq_cons == hw_cq_cons) { + rte_spinlock_unlock(&(fp)->rx_mtx); return 0; + } while (nb_rx < nb_pkts && sw_cq_cons != hw_cq_cons) { @@ -461,6 +465,8 @@ bnx2x_recv_pkts(void *p_rxq, struct rte_mbuf **rx_pkts, uint16_t nb_pkts) bnx2x_upd_rx_prod_fast(sc, fp, bd_prod, sw_cq_prod); + rte_spinlock_unlock(&(fp)->rx_mtx); + return nb_rx; }