From patchwork Wed Jul 17 12:43:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nithin Dabilpuram X-Patchwork-Id: 56621 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id F3DE41BC20; Wed, 17 Jul 2019 14:44:17 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by dpdk.org (Postfix) with ESMTP id 4F1BC1BC20 for ; Wed, 17 Jul 2019 14:44:16 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id x6HCeMhV028818; Wed, 17 Jul 2019 05:44:14 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=pfpt0818; bh=b/nixBUhiYwa3p7+Q1fVDjvqho9HVfDlxSEajxqXTg4=; b=CTeqpeYcmjl2BPN1LIQAR53tkmyLWt4W6UNpYtGkvw7eC5qb6W0+3glGNqdMQ7blCqU+ gw6j0WIUhS1izacbn6XgX5pubBSNl1sKypJMvS/Smn29Hf0n/eKPGKIv1TobKwJsjekI HCDZ9Bojid58siXU3E/LDPhoVOHGQa/YmrZpL3hUEgDSchaQdWidNRjy+it32y+xLJRU c0fkmBFvyRQia3thSWICaFzOPtOLLlbVBSK+J/Q59tKPPgvyV/bgUSZSG7O3ScvlVdkS 0FrQtYdtP3RkkQGX3ESbVnOz3LMgw/jPqmAzeJBCqmsISY6AAx5XsSMZwegnRh4rWdy0 DQ== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0a-0016f401.pphosted.com with ESMTP id 2ts07vg683-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 17 Jul 2019 05:44:14 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Wed, 17 Jul 2019 05:44:13 -0700 Received: from maili.marvell.com (10.93.176.43) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Wed, 17 Jul 2019 05:44:13 -0700 Received: from hyd1588t430.marvell.com (unknown [10.29.52.204]) by maili.marvell.com (Postfix) with ESMTP id 2506F3F703F; Wed, 17 Jul 2019 05:44:09 -0700 (PDT) From: Nithin Dabilpuram To: Hyong Youb Kim , David Marchand , Thomas Monjalon , "Ferruh Yigit" , Bruce Richardson CC: , John Daley , Shahed Shaikh , , Nithin Dabilpuram Date: Wed, 17 Jul 2019 18:13:53 +0530 Message-ID: <20190717124354.142668-3-ndabilpuram@marvell.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20190717124354.142668-1-ndabilpuram@marvell.com> References: <20190717115852.171416-1-ndabilpuram@marvell.com> <20190717124354.142668-1-ndabilpuram@marvell.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:5.22.84,1.0.8 definitions=2019-07-17_05:2019-07-17,2019-07-17 signatures=0 Subject: [dpdk-dev] [PATCH v2 2/3] eal: add ack interrupt API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add new ack interrupt API to avoid using VFIO_IRQ_SET_ACTION_TRIGGER(rte_intr_enable()) for acking interrupt purpose for VFIO based interrupt handlers. This implementation is specific to Linux. Using rte_intr_enable() for acking interrupt has below issues * Time consuming to do for every interrupt received as it will free_irq() followed by request_irq() and all other initializations * A race condition because of a window between free_irq() and request_irq() with packet reception still on and device still enabled and would throw warning messages like below. [158764.159833] do_IRQ: 9.34 No irq handler for vector In this patch, rte_intr_ack() is a no-op for VFIO_MSIX/VFIO_MSI interrupts as they are edge triggered and kernel would not mask the interrupt before delivering the event to userspace and we don't need to ack. Signed-off-by: Nithin Dabilpuram Signed-off-by: Jerin Jacob --- v2: * No change lib/librte_eal/common/include/rte_interrupts.h | 22 +++++++ lib/librte_eal/freebsd/eal/eal_interrupts.c | 9 +++ lib/librte_eal/linux/eal/eal_interrupts.c | 81 ++++++++++++++++++++++++++ lib/librte_eal/rte_eal_version.map | 1 + 4 files changed, 113 insertions(+) diff --git a/lib/librte_eal/common/include/rte_interrupts.h b/lib/librte_eal/common/include/rte_interrupts.h index c1e912c..93b31cd 100644 --- a/lib/librte_eal/common/include/rte_interrupts.h +++ b/lib/librte_eal/common/include/rte_interrupts.h @@ -118,6 +118,28 @@ int rte_intr_enable(const struct rte_intr_handle *intr_handle); */ int rte_intr_disable(const struct rte_intr_handle *intr_handle); +/** + * It acks an interrupt raised for the specified handle. + * + * Call this function to ack an interrupt from interrupt + * handler either from application or driver, so that + * new interrupts are raised. + * + * @note For interrupt handle types VFIO_MSIX and VFIO_MSI, + * this function is a no-op and returns success without + * changing anything as kernel doesn't expect + * them to be acked. + * + * @param intr_handle + * pointer to the interrupt handle. + * + * @return + * - On success, zero. + * - On failure, a negative value. + */ +__rte_experimental +int rte_intr_ack(const struct rte_intr_handle *intr_handle); + #ifdef __cplusplus } #endif diff --git a/lib/librte_eal/freebsd/eal/eal_interrupts.c b/lib/librte_eal/freebsd/eal/eal_interrupts.c index 10375bd..f6831b7 100644 --- a/lib/librte_eal/freebsd/eal/eal_interrupts.c +++ b/lib/librte_eal/freebsd/eal/eal_interrupts.c @@ -387,6 +387,15 @@ rte_intr_disable(const struct rte_intr_handle *intr_handle) return 0; } +int +rte_intr_ack(const struct rte_intr_handle *intr_handle) +{ + if (intr_handle && intr_handle->type == RTE_INTR_HANDLE_VDEV) + return 0; + + return -1; +} + static void eal_intr_process_interrupts(struct kevent *events, int nfds) { diff --git a/lib/librte_eal/linux/eal/eal_interrupts.c b/lib/librte_eal/linux/eal/eal_interrupts.c index 79ad5e8..91e220c 100644 --- a/lib/librte_eal/linux/eal/eal_interrupts.c +++ b/lib/librte_eal/linux/eal/eal_interrupts.c @@ -197,6 +197,35 @@ vfio_disable_intx(const struct rte_intr_handle *intr_handle) { return 0; } +/* unmask/ack legacy (INTx) interrupts */ +static int +vfio_ack_intx(const struct rte_intr_handle *intr_handle) +{ + struct vfio_irq_set *irq_set; + char irq_set_buf[IRQ_SET_BUF_LEN]; + int len, ret; + + len = sizeof(struct vfio_irq_set); + + /* unmask INTx */ + irq_set = (struct vfio_irq_set *) irq_set_buf; + memset(irq_set, 0, len); + irq_set->argsz = len; + irq_set->count = 1; + irq_set->flags = VFIO_IRQ_SET_DATA_NONE | VFIO_IRQ_SET_ACTION_UNMASK; + irq_set->index = VFIO_PCI_INTX_IRQ_INDEX; + irq_set->start = 0; + + ret = ioctl(intr_handle->vfio_dev_fd, VFIO_DEVICE_SET_IRQS, irq_set); + + if (ret) { + RTE_LOG(ERR, EAL, "Error unmasking INTx interrupts for fd %d\n", + intr_handle->fd); + return -1; + } + return 0; +} + /* enable MSI interrupts */ static int vfio_enable_msi(const struct rte_intr_handle *intr_handle) { @@ -694,6 +723,58 @@ rte_intr_enable(const struct rte_intr_handle *intr_handle) } int +rte_intr_ack(const struct rte_intr_handle *intr_handle) +{ + if (intr_handle && intr_handle->type == RTE_INTR_HANDLE_VDEV) + return 0; + + if (!intr_handle || intr_handle->fd < 0 || intr_handle->uio_cfg_fd < 0) + return -1; + + switch (intr_handle->type) { + /* Both acking and disabling are same for UIO */ + case RTE_INTR_HANDLE_UIO: + if (uio_intr_enable(intr_handle)) + return -1; + break; + case RTE_INTR_HANDLE_UIO_INTX: + if (uio_intx_intr_enable(intr_handle)) + return -1; + break; + /* not used at this moment */ + case RTE_INTR_HANDLE_ALARM: + return -1; +#ifdef VFIO_PRESENT + /* Since VFIO_MSIX is implicitly acked + * unlike INTx, we report success + */ + case RTE_INTR_HANDLE_VFIO_MSIX: + case RTE_INTR_HANDLE_VFIO_MSI: + return 0; + case RTE_INTR_HANDLE_VFIO_LEGACY: + if (vfio_ack_intx(intr_handle)) + return -1; + break; +#ifdef HAVE_VFIO_DEV_REQ_INTERFACE + case RTE_INTR_HANDLE_VFIO_REQ: + return -1; +#endif +#endif + /* not used at this moment */ + case RTE_INTR_HANDLE_DEV_EVENT: + return -1; + /* unknown handle type */ + default: + RTE_LOG(ERR, EAL, + "Unknown handle type of fd %d\n", + intr_handle->fd); + return -1; + } + + return 0; +} + +int rte_intr_disable(const struct rte_intr_handle *intr_handle) { if (intr_handle && intr_handle->type == RTE_INTR_HANDLE_VDEV) diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map index 1892d9e..0537a6d 100644 --- a/lib/librte_eal/rte_eal_version.map +++ b/lib/librte_eal/rte_eal_version.map @@ -407,4 +407,5 @@ EXPERIMENTAL { rte_lcore_to_cpu_id; rte_mcfg_timer_lock; rte_mcfg_timer_unlock; + rte_intr_ack; };