From patchwork Tue Apr 11 09:11:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 125922 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5C6874291B; Tue, 11 Apr 2023 11:13:55 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A3DD042D61; Tue, 11 Apr 2023 11:12:54 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 1C35542D50 for ; Tue, 11 Apr 2023 11:12:53 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 33B8g4Cc014870 for ; Tue, 11 Apr 2023 02:12:52 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=pMe2IPo8T3ph4jGX2J317OQQDJEx08PoxAbJzmuXDno=; b=YOVBWoDynY7OEmLxb/Ch9hYF2EeimECxrWWTHCl9btXdKI7frJ9JQdikySuVOfrMcauw adjHS/Oi3Gjil+OfIoM0MgRd1M71j89KUVrx3W43dXbjHYZeFYXxtY31ReaMZVMetC3M f1tC1NayVvmLfWZQX5PsgBnltD/cJHjpUrZWMo7nlRrUmWNEXX/lQZzZBPkQUPXaZlMO KQ3SOdO1YdRjFyJ+3Lvn7cBffXchdx8cDWNHhInwxEzDy5zwZy+TeUaaEM8WjeVG/PrP HrcN84PcQk9o4KEYZWbNoFZKIF8NR1rNwTXC1jIAUBMdeEJiUpQ4CwJBzyE45fJhZog5 UA== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3purfs94jw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Tue, 11 Apr 2023 02:12:52 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Tue, 11 Apr 2023 02:12:50 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Tue, 11 Apr 2023 02:12:50 -0700 Received: from hyd1588t430.caveonetworks.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id E2E393F706F; Tue, 11 Apr 2023 02:12:47 -0700 (PDT) From: Nithin Dabilpuram To: Nithin Kumar Dabilpuram , Kiran Kumar K , Sunil Kumar Kori , Satha Rao CC: , , Harman Kalra Subject: [PATCH 17/21] common/cnxk: add more comments to mbox code Date: Tue, 11 Apr 2023 14:41:40 +0530 Message-ID: <20230411091144.1087887-17-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230411091144.1087887-1-ndabilpuram@marvell.com> References: <20230411091144.1087887-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-ORIG-GUID: XhU2jSK7dM5depUA0z1hS2GImRLpFgw8 X-Proofpoint-GUID: XhU2jSK7dM5depUA0z1hS2GImRLpFgw8 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-11_05,2023-04-06_03,2023-02-09_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Harman Kalra Adding more comments to the mbox routines to understand the flow well. Signed-off-by: Harman Kalra --- drivers/common/cnxk/roc_dev.c | 20 +++++++++++++++++--- drivers/common/cnxk/roc_mbox.c | 5 +++++ 2 files changed, 22 insertions(+), 3 deletions(-) diff --git a/drivers/common/cnxk/roc_dev.c b/drivers/common/cnxk/roc_dev.c index e5a5cd7c10..3125f9dda2 100644 --- a/drivers/common/cnxk/roc_dev.c +++ b/drivers/common/cnxk/roc_dev.c @@ -98,6 +98,9 @@ pf_af_sync_msg(struct dev *dev, struct mbox_msghdr **rsp) return rc; } +/* PF will send the messages to AF and wait for responses and forward the + * responses to VF. + */ static int af_pf_wait_msg(struct dev *dev, uint16_t vf, int num_msg) { @@ -115,9 +118,10 @@ af_pf_wait_msg(struct dev *dev, uint16_t vf, int num_msg) /* We need to disable PF interrupts. We are in timer interrupt */ plt_write64(~0ull, dev->bar2 + RVU_PF_INT_ENA_W1C); - /* Send message */ + /* Send message to AF */ mbox_msg_send(mbox, 0); + /* Wait for AF response */ do { plt_delay_ms(sleep); timeout++; @@ -206,6 +210,7 @@ af_pf_wait_msg(struct dev *dev, uint16_t vf, int num_msg) return req_hdr->num_msgs; } +/* PF receives mbox DOWN messages from VF and forwards to AF */ static int vf_pf_process_msgs(struct dev *dev, uint16_t vf) { @@ -274,6 +279,7 @@ vf_pf_process_msgs(struct dev *dev, uint16_t vf) if (routed > 0) { plt_base_dbg("pf:%d routed %d messages from vf:%d to AF", dev->pf, routed, vf); + /* PF will send the messages to AF and wait for responses */ af_pf_wait_msg(dev, vf, routed); mbox_reset(dev->mbox, 0); } @@ -289,6 +295,7 @@ vf_pf_process_msgs(struct dev *dev, uint16_t vf) return i; } +/* VF sends Ack to PF's UP messages */ static int vf_pf_process_up_msgs(struct dev *dev, uint16_t vf) { @@ -339,6 +346,7 @@ vf_pf_process_up_msgs(struct dev *dev, uint16_t vf) return i; } +/* PF handling messages from VF */ static void roc_vf_pf_mbox_handle_msg(void *param) { @@ -352,8 +360,9 @@ roc_vf_pf_mbox_handle_msg(void *param) if (dev->intr.bits[vf / max_bits] & BIT_ULL(vf % max_bits)) { plt_base_dbg("Process vf:%d request (pf:%d, vf:%d)", vf, dev->pf, dev->vf); + /* VF initiated down messages */ vf_pf_process_msgs(dev, vf); - /* UP messages */ + /* VF replies to PF's UP messages */ vf_pf_process_up_msgs(dev, vf); dev->intr.bits[vf / max_bits] &= ~(BIT_ULL(vf % max_bits)); @@ -362,6 +371,7 @@ roc_vf_pf_mbox_handle_msg(void *param) dev->timer_set = 0; } +/* IRQ to PF from VF - PF context (interrupt thread) */ static void roc_vf_pf_mbox_irq(void *param) { @@ -392,6 +402,7 @@ roc_vf_pf_mbox_irq(void *param) } } +/* Received response from AF (PF context) / PF (VF context) */ static void process_msgs(struct dev *dev, struct mbox *mbox) { @@ -451,7 +462,7 @@ process_msgs(struct dev *dev, struct mbox *mbox) } mbox_reset(mbox, 0); - /* Update acked if someone is waiting a message */ + /* Update acked if someone is waiting a message - mbox_wait is waiting */ mdev->msgs_acked = msgs_acked; plt_wmb(); } @@ -597,6 +608,7 @@ mbox_process_msgs_up(struct dev *dev, struct mbox_msghdr *req) return -ENODEV; } +/* Received up messages from AF (PF context) / PF (in context) */ static void process_msgs_up(struct dev *dev, struct mbox *mbox) { @@ -629,6 +641,7 @@ process_msgs_up(struct dev *dev, struct mbox *mbox) } } +/* IRQ to VF from PF - VF context (interrupt thread) */ static void roc_pf_vf_mbox_irq(void *param) { @@ -657,6 +670,7 @@ roc_pf_vf_mbox_irq(void *param) process_msgs_up(dev, &dev->mbox_up); } +/* IRQ to PF from AF - PF context (interrupt thread) */ static void roc_af_pf_mbox_irq(void *param) { diff --git a/drivers/common/cnxk/roc_mbox.c b/drivers/common/cnxk/roc_mbox.c index 5338a960d9..c91fa63e83 100644 --- a/drivers/common/cnxk/roc_mbox.c +++ b/drivers/common/cnxk/roc_mbox.c @@ -350,6 +350,11 @@ mbox_wait(struct mbox *mbox, int devid, uint32_t rst_timo) uint32_t timeout = 0, sleep = 1; rst_timo = rst_timo * 1000; /* Milli seconds to micro seconds */ + + /* Waiting for mdev->msgs_acked tp become equal to mdev->num_msgs, + * mdev->msgs_acked are incremented at process_msgs() in interrupt + * thread context. + */ while (mdev->num_msgs > mdev->msgs_acked) { plt_delay_us(sleep); timeout += sleep;