From patchwork Thu Jul 18 08:46:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 56707 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id A5CA24C90; Thu, 18 Jul 2019 10:46:47 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 4C90D4C90 for ; Thu, 18 Jul 2019 10:46:45 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x6I8iJ34010801; Thu, 18 Jul 2019 01:46:44 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0818; bh=T3BnGHX7flSoi9BibXHqinV6zRXW5lixKM1UEo/samQ=; b=YHEFhla2klsiVdgEc2rOYEDrU5yvLYuZ0lvOisNKcNVs47Cm+v/YVVD8j03jFtliQ/c2 /SN3gXs+jGp4X7kpvdKyDsiMiaaxu1dRfTJkvIUMpFrtwJp0DURnxb29G/hPfX/Y+00L kgDbdf9XNP8dHWVUsQ5Baf+qtQqlATPnEqpNBQ61rzV4XyZ614gbRvJZIfdgFUD6CfD0 95a+qCDc+Kqg+IR3Mpd69WV47xCmX7clLEmrfW4cmFW3kKAAPEAeGcx//E8rgykdbEpe A4VQZiLQR/udh+8iqqAPAhYq0ELLv+z66nxtC3k0aixsEOaqC2kwqbA2fjGyjLcRSmsA 3g== Received: from sc-exch03.marvell.com ([199.233.58.183]) by mx0b-0016f401.pphosted.com with ESMTP id 2ts0a2b8jg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 18 Jul 2019 01:46:44 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH03.marvell.com (10.93.176.83) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Thu, 18 Jul 2019 01:46:42 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Thu, 18 Jul 2019 01:46:42 -0700 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id B09E23F703F; Thu, 18 Jul 2019 01:46:38 -0700 (PDT) From: Nithin Dabilpuram To: Hyong Youb Kim , David Marchand , Thomas Monjalon , "Ferruh Yigit" , Bruce Richardson CC: , John Daley , Shahed Shaikh , , Nithin Dabilpuram Date: Thu, 18 Jul 2019 14:16:26 +0530 Message-ID: <20190718084628.134290-3-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20190718084628.134290-1-ndabilpuram@marvell.com> References: <20190717115852.171416-1-ndabilpuram@marvell.com> <20190718084628.134290-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:5.22.84,1.0.8 definitions=2019-07-18_04:2019-07-18,2019-07-18 signatures=0 Subject: [dpdk-dev] [PATCH v3 2/3] eal: add ack interrupt API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add new ack interrupt API to avoid using VFIO_IRQ_SET_ACTION_TRIGGER(rte_intr_enable()) for acking interrupt purpose for VFIO based interrupt handlers. This implementation is specific to Linux. Using rte_intr_enable() for acking interrupt has below issues * Time consuming to do for every interrupt received as it will free_irq() followed by request_irq() and all other initializations * A race condition because of a window between free_irq() and request_irq() with packet reception still on and device still enabled and would throw warning messages like below. [158764.159833] do_IRQ: 9.34 No irq handler for vector In this patch, rte_intr_ack() is a no-op for VFIO_MSIX/VFIO_MSI interrupts as they are edge triggered and kernel would not mask the interrupt before delivering the event to userspace and we don't need to ack. Signed-off-by: Nithin Dabilpuram Signed-off-by: Jerin Jacob Tested-by: Shahed Shaikh --- v3: * Update note on new api v2: * No change lib/librte_eal/common/include/rte_interrupts.h | 24 ++++++++ lib/librte_eal/freebsd/eal/eal_interrupts.c | 9 +++ lib/librte_eal/linux/eal/eal_interrupts.c | 81 ++++++++++++++++++++++++++ lib/librte_eal/rte_eal_version.map | 1 + 4 files changed, 115 insertions(+) diff --git a/lib/librte_eal/common/include/rte_interrupts.h b/lib/librte_eal/common/include/rte_interrupts.h index c1e912c..5811b78 100644 --- a/lib/librte_eal/common/include/rte_interrupts.h +++ b/lib/librte_eal/common/include/rte_interrupts.h @@ -118,6 +118,30 @@ int rte_intr_enable(const struct rte_intr_handle *intr_handle); */ int rte_intr_disable(const struct rte_intr_handle *intr_handle); +/** + * It acks an interrupt raised for the specified handle. + * + * Call this function to ack an interrupt from interrupt + * handler either from application or driver, so that + * new interrupts are raised. + * + * @note PMD generally calls this function at the end of its IRQ callback. + * Internally, it unmasks the interrupt if possible. For INTx, unmasking + * is required as the interrupt is auto-masked prior to invoking + * callback. For MSI/MSI-X, unmasking is typically not needed as the + * interrupt is not auto-masked. In fact, for interrupt handle types + * VFIO_MSIX and VFIO_MSI, this function is no-op. + * + * @param intr_handle + * pointer to the interrupt handle. + * + * @return + * - On success, zero. + * - On failure, a negative value. + */ +__rte_experimental +int rte_intr_ack(const struct rte_intr_handle *intr_handle); + #ifdef __cplusplus } #endif diff --git a/lib/librte_eal/freebsd/eal/eal_interrupts.c b/lib/librte_eal/freebsd/eal/eal_interrupts.c index 10375bd..f6831b7 100644 --- a/lib/librte_eal/freebsd/eal/eal_interrupts.c +++ b/lib/librte_eal/freebsd/eal/eal_interrupts.c @@ -387,6 +387,15 @@ rte_intr_disable(const struct rte_intr_handle *intr_handle) return 0; } +int +rte_intr_ack(const struct rte_intr_handle *intr_handle) +{ + if (intr_handle && intr_handle->type == RTE_INTR_HANDLE_VDEV) + return 0; + + return -1; +} + static void eal_intr_process_interrupts(struct kevent *events, int nfds) { diff --git a/lib/librte_eal/linux/eal/eal_interrupts.c b/lib/librte_eal/linux/eal/eal_interrupts.c index 79ad5e8..91e220c 100644 --- a/lib/librte_eal/linux/eal/eal_interrupts.c +++ b/lib/librte_eal/linux/eal/eal_interrupts.c @@ -197,6 +197,35 @@ vfio_disable_intx(const struct rte_intr_handle *intr_handle) { return 0; } +/* unmask/ack legacy (INTx) interrupts */ +static int +vfio_ack_intx(const struct rte_intr_handle *intr_handle) +{ + struct vfio_irq_set *irq_set; + char irq_set_buf[IRQ_SET_BUF_LEN]; + int len, ret; + + len = sizeof(struct vfio_irq_set); + + /* unmask INTx */ + irq_set = (struct vfio_irq_set *) irq_set_buf; + memset(irq_set, 0, len); + irq_set->argsz = len; + irq_set->count = 1; + irq_set->flags = VFIO_IRQ_SET_DATA_NONE | VFIO_IRQ_SET_ACTION_UNMASK; + irq_set->index = VFIO_PCI_INTX_IRQ_INDEX; + irq_set->start = 0; + + ret = ioctl(intr_handle->vfio_dev_fd, VFIO_DEVICE_SET_IRQS, irq_set); + + if (ret) { + RTE_LOG(ERR, EAL, "Error unmasking INTx interrupts for fd %d\n", + intr_handle->fd); + return -1; + } + return 0; +} + /* enable MSI interrupts */ static int vfio_enable_msi(const struct rte_intr_handle *intr_handle) { @@ -694,6 +723,58 @@ rte_intr_enable(const struct rte_intr_handle *intr_handle) } int +rte_intr_ack(const struct rte_intr_handle *intr_handle) +{ + if (intr_handle && intr_handle->type == RTE_INTR_HANDLE_VDEV) + return 0; + + if (!intr_handle || intr_handle->fd < 0 || intr_handle->uio_cfg_fd < 0) + return -1; + + switch (intr_handle->type) { + /* Both acking and disabling are same for UIO */ + case RTE_INTR_HANDLE_UIO: + if (uio_intr_enable(intr_handle)) + return -1; + break; + case RTE_INTR_HANDLE_UIO_INTX: + if (uio_intx_intr_enable(intr_handle)) + return -1; + break; + /* not used at this moment */ + case RTE_INTR_HANDLE_ALARM: + return -1; +#ifdef VFIO_PRESENT + /* Since VFIO_MSIX is implicitly acked + * unlike INTx, we report success + */ + case RTE_INTR_HANDLE_VFIO_MSIX: + case RTE_INTR_HANDLE_VFIO_MSI: + return 0; + case RTE_INTR_HANDLE_VFIO_LEGACY: + if (vfio_ack_intx(intr_handle)) + return -1; + break; +#ifdef HAVE_VFIO_DEV_REQ_INTERFACE + case RTE_INTR_HANDLE_VFIO_REQ: + return -1; +#endif +#endif + /* not used at this moment */ + case RTE_INTR_HANDLE_DEV_EVENT: + return -1; + /* unknown handle type */ + default: + RTE_LOG(ERR, EAL, + "Unknown handle type of fd %d\n", + intr_handle->fd); + return -1; + } + + return 0; +} + +int rte_intr_disable(const struct rte_intr_handle *intr_handle) { if (intr_handle && intr_handle->type == RTE_INTR_HANDLE_VDEV) diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 1892d9e..0537a6d 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -407,4 +407,5 @@ EXPERIMENTAL { rte_lcore_to_cpu_id; rte_mcfg_timer_lock; rte_mcfg_timer_unlock; + rte_intr_ack; };