From patchwork Thu Oct 5 06:25:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harman Kalra X-Patchwork-Id: 132330 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 15EDC426BE; Thu, 5 Oct 2023 08:25:35 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 16458402B3; Thu, 5 Oct 2023 08:25:33 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id C5305402A9 for ; Thu, 5 Oct 2023 08:25:31 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 39543DH5016272 for ; Wed, 4 Oct 2023 23:25:31 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : mime-version : content-type; s=pfpt0220; bh=/ThXy0kb8HxiBnLa7ORl/fuRQSNm/LinJ+1uvEKHlU8=; b=FhZn4+rakxKJ02CA0q2LBlCCyjUH0A29+hu7eIpVYd2oLnVNfha0zCwklCXigzBbLXUx Z6DMBTWItPWTU/EFASDNhY2ZdiflhbiLlfazWlGeprTv/1NCyichr8g8srlWWWVK176i hNiMBNVrP3r1M2YXlEeXpCEhk2vadFVZkFewLV/JywwQwIZh3PSFdGo5Ypa9OkQR5J2t qJsC6OVaioF55PJXp9cbgueEF8Yza8uY50cQVHrlwfbXtBI37ZYPOl/eZJmkxBV3+5ZO gGBpooZK5MiQg7P0+4i/sIbD8IEnJW183NpdEt9IkUjeyjl3Km7wd5B88pFvkd8wDxGC zQ== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3th2b6mw9t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Wed, 04 Oct 2023 23:25:30 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Wed, 4 Oct 2023 23:25:29 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Wed, 4 Oct 2023 23:25:28 -0700 Received: from localhost.localdomain (unknown [10.29.52.211]) by maili.marvell.com (Postfix) with ESMTP id 247DD3F7043; Wed, 4 Oct 2023 23:25:21 -0700 (PDT) From: Harman Kalra To: , Nithin Dabilpuram , "Kiran Kumar K" , Sunil Kumar Kori , Satha Rao CC: , Harman Kalra Subject: [PATCH 1/2] common/cnxk: fix handling up and down interrupts Date: Thu, 5 Oct 2023 11:55:12 +0530 Message-ID: <20231005062513.29467-1-hkalra@marvell.com> X-Mailer: git-send-email 2.18.0 MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: T01sBXN4k8wdFZpYPNMJb5qPpwPDAJio X-Proofpoint-GUID: T01sBXN4k8wdFZpYPNMJb5qPpwPDAJio X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.980,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-10-05_03,2023-10-02_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Down messages should be processed only when down interrupt is received while UP messages should be processed when up interrupt is received. A scenario has been observed where processing down message on UP message interrupt causes processing of old message response while response of latest message is not received. Fixes: fa4ee2d43188 ("common/cnxk: sync between mbox up and down messages") Signed-off-by: Harman Kalra --- drivers/common/cnxk/roc_dev.c | 40 ++++++++++++++++++++++++----------- 1 file changed, 28 insertions(+), 12 deletions(-) diff --git a/drivers/common/cnxk/roc_dev.c b/drivers/common/cnxk/roc_dev.c index e41235ea8a..c948caba63 100644 --- a/drivers/common/cnxk/roc_dev.c +++ b/drivers/common/cnxk/roc_dev.c @@ -770,14 +770,22 @@ roc_pf_vf_mbox_irq(void *param) * by 1ms until this region is zeroed mbox_wait_for_zero() */ mbox_data = plt_read64(dev->bar2 + RVU_VF_VFPF_MBOX0); - if (mbox_data) - plt_write64(!mbox_data, dev->bar2 + RVU_VF_VFPF_MBOX0); + /* If interrupt occurred for down message */ + if (mbox_data & MBOX_DOWN_MSG) { + mbox_data &= ~MBOX_DOWN_MSG; + plt_write64(mbox_data, dev->bar2 + RVU_VF_VFPF_MBOX0); - /* First process all configuration messages */ - process_msgs(dev, dev->mbox); + /* First process all configuration messages */ + process_msgs(dev, dev->mbox); + } + /* If interrupt occurred for UP message */ + if (mbox_data & MBOX_UP_MSG) { + mbox_data &= ~MBOX_UP_MSG; + plt_write64(mbox_data, dev->bar2 + RVU_VF_VFPF_MBOX0); - /* Process Uplink messages */ - process_msgs_up(dev, &dev->mbox_up); + /* Process Uplink messages */ + process_msgs_up(dev, &dev->mbox_up); + } } /* IRQ to PF from AF - PF context (interrupt thread) */ @@ -799,14 +807,22 @@ roc_af_pf_mbox_irq(void *param) * by 1ms until this region is zeroed mbox_wait_for_zero() */ mbox_data = plt_read64(dev->bar2 + RVU_PF_PFAF_MBOX0); - if (mbox_data) - plt_write64(!mbox_data, dev->bar2 + RVU_PF_PFAF_MBOX0); + /* If interrupt occurred for down message */ + if (mbox_data & MBOX_DOWN_MSG) { + mbox_data &= ~MBOX_DOWN_MSG; + plt_write64(mbox_data, dev->bar2 + RVU_PF_PFAF_MBOX0); - /* First process all configuration messages */ - process_msgs(dev, dev->mbox); + /* First process all configuration messages */ + process_msgs(dev, dev->mbox); + } + /* If interrupt occurred for up message */ + if (mbox_data & MBOX_UP_MSG) { + mbox_data &= ~MBOX_UP_MSG; + plt_write64(mbox_data, dev->bar2 + RVU_PF_PFAF_MBOX0); - /* Process Uplink messages */ - process_msgs_up(dev, &dev->mbox_up); + /* Process Uplink messages */ + process_msgs_up(dev, &dev->mbox_up); + } } static int From patchwork Thu Oct 5 06:25:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harman Kalra X-Patchwork-Id: 132331 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2B86D426BE; Thu, 5 Oct 2023 08:25:39 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 39EDA402D1; Thu, 5 Oct 2023 08:25:38 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 06F7F402A9 for ; Thu, 5 Oct 2023 08:25:36 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 3953xPT1019149 for ; Wed, 4 Oct 2023 23:25:36 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0220; bh=q4YZciaMiS0r2+vS9ZEigMTn8YtY/r9q1xON3ygMOtA=; b=PuVt4JUBEGjITaDMAmrIsEM4Reb/Lz2FGjOsMxVPEBNoRjOPxgU+bk8yCb9+fTgW/tus H2rLUO0fkYsmX/MazmLdAX6ygnkVQkTZ/9v3HA5TW7hHnGTsXGYlcYyPUesl6xPuyTEH ZYjFhESBeA/CY3cnvH4VS/qTW3Xl1zLUUvgj5FhjTHDWlwKHmERjePgnqoOL0K92PRnI pzuOYu4hxQ2XBsPUKL5RkuktRaXwIE3YPVTvqU5lWptpb5k3VJ/m8lnh7/er4qkUGntm hSszjWP7xS0AFxkkiX1Dmw3Mp+/+3HRpPf66B1ZfVVfFWoPZwrOsUO5K/EGvXKCr+r2B 1A== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3th29umsv3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Wed, 04 Oct 2023 23:25:36 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Wed, 4 Oct 2023 23:25:33 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Wed, 4 Oct 2023 23:25:33 -0700 Received: from localhost.localdomain (unknown [10.29.52.211]) by maili.marvell.com (Postfix) with ESMTP id 3B9F23F7070; Wed, 4 Oct 2023 23:25:26 -0700 (PDT) From: Harman Kalra To: , Nithin Dabilpuram , "Kiran Kumar K" , Sunil Kumar Kori , Satha Rao CC: , Harman Kalra Subject: [PATCH 2/2] common/cnxk: fix race condition between up and down mbox Date: Thu, 5 Oct 2023 11:55:13 +0530 Message-ID: <20231005062513.29467-2-hkalra@marvell.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20231005062513.29467-1-hkalra@marvell.com> References: <20231005062513.29467-1-hkalra@marvell.com> MIME-Version: 1.0 X-Proofpoint-GUID: tsMwI6VPtFAxd9aoP_wGUXhEL-PLd8Sd X-Proofpoint-ORIG-GUID: tsMwI6VPtFAxd9aoP_wGUXhEL-PLd8Sd X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.980,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-10-05_03,2023-10-02_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Fixing a possible case for race condition where an up mbox interrupt over writes the down mbox message. Although mbox_wait_for_zero() makes sure no up/down message is pending before raising an up mbox interrupt. But there is a small window were a VF may send a down mbox request to PF after mbox_wait_for_zero() and before PF attempts to send a up message to same VF. In such scenario interrupt register which has down message bit set will get overwritten by up message bit. As a solution, read interrupt register and OR the status with required up/down bit before writing to the interrupt register. Fixes: fa4ee2d43188 ("common/cnxk: sync between mbox up and down messages") Signed-off-by: Harman Kalra --- drivers/common/cnxk/roc_mbox.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/drivers/common/cnxk/roc_mbox.c b/drivers/common/cnxk/roc_mbox.c index c91fa63e83..7b734fcd24 100644 --- a/drivers/common/cnxk/roc_mbox.c +++ b/drivers/common/cnxk/roc_mbox.c @@ -209,10 +209,9 @@ static void mbox_msg_send_data(struct mbox *mbox, int devid, uint8_t data) { struct mbox_dev *mdev = &mbox->dev[devid]; - struct mbox_hdr *tx_hdr = - (struct mbox_hdr *)((uintptr_t)mdev->mbase + mbox->tx_start); - struct mbox_hdr *rx_hdr = - (struct mbox_hdr *)((uintptr_t)mdev->mbase + mbox->rx_start); + struct mbox_hdr *tx_hdr = (struct mbox_hdr *)((uintptr_t)mdev->mbase + mbox->tx_start); + struct mbox_hdr *rx_hdr = (struct mbox_hdr *)((uintptr_t)mdev->mbase + mbox->rx_start); + uint64_t intr_val; /* Reset header for next messages */ tx_hdr->msg_size = mdev->msg_size; @@ -229,11 +228,16 @@ mbox_msg_send_data(struct mbox *mbox, int devid, uint8_t data) /* Sync mbox data into memory */ plt_wmb(); + /* Check for any pending interrupt */ + intr_val = plt_read64( + (volatile void *)(mbox->reg_base + (mbox->trigger | (devid << mbox->tr_shift)))); + + intr_val |= (uint64_t)data; /* The interrupt should be fired after num_msgs is written * to the shared memory */ - plt_write64(data, (volatile void *)(mbox->reg_base + - (mbox->trigger | (devid << mbox->tr_shift)))); + plt_write64(intr_val, (volatile void *)(mbox->reg_base + + (mbox->trigger | (devid << mbox->tr_shift)))); } /**