From patchwork Sat Aug 20 00:59:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timothy McDaniel X-Patchwork-Id: 115306 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id ABE8CA034C; Sat, 20 Aug 2022 03:00:20 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 32E3A427ED; Sat, 20 Aug 2022 03:00:11 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id E534340693 for ; Sat, 20 Aug 2022 03:00:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1660957207; x=1692493207; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Y9Msl/LenELzXgKg21Q9rwwtt0mWceZBSnHoiANnpvA=; b=VQpy7eWtExqLwWGvapjizhDFzk7exL3U2lZqUV74cd+KIi6aoe0IvlgF 4F0kR3jJSnc9rR4BwhWpoCTVPesvV7/eHDseu9bV7GW7ofRDocVTdmyIu 5AW1FZJBszI2oSknOUfDM7bMui2bZfwly5D+vPqQnmCs0ey0/nPITeawZ E0u3LqPbXUaiIAXbgTffWBivA/P1awEffCIUq19ycFWZDxA90x+f+6+9G 9EWNIu5tPc7DPuGxkGyvfJRO6Eq806sBo2BmmAPQbY8F/CdCpeJ8eawbU MHrga6kohlqiZWzaOLGzaNM2ZrrS6bxzIDhtT3YfXylA5LdWgluBUSlLp A==; X-IronPort-AV: E=McAfee;i="6500,9779,10444"; a="357119058" X-IronPort-AV: E=Sophos;i="5.93,249,1654585200"; d="scan'208";a="357119058" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Aug 2022 18:00:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,249,1654585200"; d="scan'208";a="668803541" Received: from txanpdk03.an.intel.com ([10.123.117.78]) by fmsmga008.fm.intel.com with ESMTP; 19 Aug 2022 18:00:04 -0700 From: Timothy McDaniel To: jerinj@marvell.com Cc: dev@dpdk.org Subject: [PATCH 2/3] event/dlb2: add fence bypass option for producer ports Date: Fri, 19 Aug 2022 19:59:56 -0500 Message-Id: <20220820005957.2986689-3-timothy.mcdaniel@intel.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20220820005957.2986689-1-timothy.mcdaniel@intel.com> References: <20220820005957.2986689-1-timothy.mcdaniel@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org If producer thread is only acting as a bridge between NIC and DLB, then performance can be greatly improved by bypassing the fence instrucntion. DLB enqueue API calls memory fence once per enqueue burst. If prodcuer thread is just reading from NIC and sending to DLB without updating the read buffers or buffer headers OR producer is not writing to data structures with dependencies on the enqueue write order, then fencing can be safely disabled. Signed-off-by: Timothy McDaniel --- drivers/event/dlb2/dlb2.c | 31 +++++++++++++++++++++++-------- 1 file changed, 23 insertions(+), 8 deletions(-) diff --git a/drivers/event/dlb2/dlb2.c b/drivers/event/dlb2/dlb2.c index a9a174e136..76f51736c4 100644 --- a/drivers/event/dlb2/dlb2.c +++ b/drivers/event/dlb2/dlb2.c @@ -35,6 +35,16 @@ #include "dlb2_iface.h" #include "dlb2_inline_fns.h" +/* + * Bypass memory fencing instructions when port is of Producer type. + * This should be enabled very carefully with understanding that producer + * is not doing any writes which need fencing. The movdir64 instruction used to + * enqueue events to DLB is a weakly-ordered instruction and movdir64 write + * to DLB can go ahead of relevant application writes like updates to buffers + * being sent with event + */ +#define DLB2_BYPASS_FENCE_ON_PP 0 /* 1 == Bypass fence, 0 == do not bypass */ + /* * Resources exposed to eventdev. Some values overridden at runtime using * values returned by the DLB kernel driver. @@ -1965,21 +1975,15 @@ dlb2_eventdev_port_setup(struct rte_eventdev *dev, sw_credit_quanta = dlb2->sw_credit_quanta; hw_credit_quanta = dlb2->hw_credit_quanta; + ev_port->qm_port.is_producer = false; ev_port->qm_port.is_directed = port_conf->event_port_cfg & RTE_EVENT_PORT_CFG_SINGLE_LINK; - /* - * Validate credit config before creating port - */ - - /* Default for worker ports */ - sw_credit_quanta = dlb2->sw_credit_quanta; - hw_credit_quanta = dlb2->hw_credit_quanta; - if (port_conf->event_port_cfg & RTE_EVENT_PORT_CFG_HINT_PRODUCER) { /* Producer type ports. Mostly enqueue */ sw_credit_quanta = DLB2_SW_CREDIT_P_QUANTA_DEFAULT; hw_credit_quanta = DLB2_SW_CREDIT_P_BATCH_SZ; + ev_port->qm_port.is_producer = true; } if (port_conf->event_port_cfg & RTE_EVENT_PORT_CFG_HINT_CONSUMER) { /* Consumer type ports. Mostly dequeue */ @@ -1989,6 +1993,10 @@ dlb2_eventdev_port_setup(struct rte_eventdev *dev, ev_port->credit_update_quanta = sw_credit_quanta; ev_port->qm_port.hw_credit_quanta = hw_credit_quanta; + /* + * Validate credit config before creating port + */ + if (port_conf->enqueue_depth > sw_credit_quanta || port_conf->enqueue_depth > hw_credit_quanta) { DLB2_LOG_ERR("Invalid port config. Enqueue depth %d must be <= credit quanta %d and batch size %d\n", @@ -3055,6 +3063,13 @@ __dlb2_event_enqueue_burst(void *event_port, dlb2_hw_do_enqueue(qm_port, i == 0, port_data); +#if DLB2_BYPASS_FENCE_ON_PP == 1 + /* Bypass fence instruction for producer ports */ + dlb2_hw_do_enqueue(qm_port, i == 0 && !qm_port->is_producer, port_data); +#else + dlb2_hw_do_enqueue(qm_port, i == 0, port_data); +#endif + /* Don't include the token pop QE in the enqueue count */ i += j - pop_offs;