From patchwork Tue Oct 19 10:56:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Krawczyk X-Patchwork-Id: 102193 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id ED7AFA0C43; Tue, 19 Oct 2021 12:57:22 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A3A4341172; Tue, 19 Oct 2021 12:56:59 +0200 (CEST) Received: from mail-lf1-f52.google.com (mail-lf1-f52.google.com [209.85.167.52]) by mails.dpdk.org (Postfix) with ESMTP id 374874115F for ; Tue, 19 Oct 2021 12:56:55 +0200 (CEST) Received: by mail-lf1-f52.google.com with SMTP id i24so6435126lfj.13 for ; Tue, 19 Oct 2021 03:56:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=semihalf-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AVjNj9C3sFtARDCgr8vuZQakTxLnAsKnsDeQGvIaoQ8=; b=amwLVjOM2oqDKxC80luTS2POhNzNv2YIcTdTHzgVLNJnL6VZM4gIjpArEO2Q2S9Dv5 SMFoMJAfSPwRC4thBipzt1fY+w6TG2l5Fqt/sPT4/HfFoC3c23P7S6TFsDnrhHu5y2Qu 6wuJtSkscyX9Msy3RzE8x9szX1G4FUFRbLDmOcAqk6GVOl2jIhmHuwiZiKtwo61zlcNT 77rq8Ps/98+/EpeGoCfK7psKHnjNu3GaQ/h2sORDhEeaMGLRWEsKn6OzTZYHqqKfITGp gBcNciKmAZVUEKtRfc+b4YbhJCyijuLgCQVyrQ7STKPnXDcE6VMYhjWHIfOrZg1uhmXO e9Aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AVjNj9C3sFtARDCgr8vuZQakTxLnAsKnsDeQGvIaoQ8=; b=Q6jaap1fOEhrpZKa/lVZPbnjdM2Z0UNRPzQeAsM2SPV64yqbkvgY6Wb10Z7BcSwyYK VRD/z+yAzZsGZ+pwoistyfpotqKfVLaVMeOn8+vbFndG75enM8qSl6v1q9c0ILcVnCFW hzbyWSwgdcY2UtRbaSTvVEpv5O7rYJ3c4H72PBnu0mbSKkH1qv0gr2RC09K2tEnf1Aiv RD5DCX7PobSbRwKAvHNggNVYNnpZ+HxGRZGslgfqCzJ+XyUo+hbtxtisJb0HSvSY0GLI qNUqxn3P2SaKr1CJwLAeOmyGic3VPUcFfmSnbwoghz2BxMb83seZJFE+3I9nX1TteziV Q4sQ== X-Gm-Message-State: AOAM533icjV3+izJqqcYGzA1UVuzGkfi9win7pMjocUAZjkvsx+W6747 /6ZEUOoI+Kd7XUH1seqs1/JSGA== X-Google-Smtp-Source: ABdhPJz1rxRG5P6Q48F52JjL22MqUrxRvi5rmqnG5hsDK5dzUoJa4txRkkohVJRgl3TDbBZAKwvQlg== X-Received: by 2002:a05:6512:907:: with SMTP id e7mr5563623lft.671.1634641014778; Tue, 19 Oct 2021 03:56:54 -0700 (PDT) Received: from DESKTOP-U5LNN3J.localdomain (89-79-181-52.dynamic.chello.pl. [89.79.181.52]) by smtp.gmail.com with ESMTPSA id bn17sm1496058ljb.4.2021.10.19.03.56.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Oct 2021 03:56:54 -0700 (PDT) From: Michal Krawczyk To: ferruh.yigit@intel.com Cc: dev@dpdk.org, upstream@semihalf.com, shaibran@amazon.com, ndagan@amazon.com, igorch@amazon.com, Michal Krawczyk Date: Tue, 19 Oct 2021 12:56:28 +0200 Message-Id: <20211019105629.11731-7-mk@semihalf.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211019105629.11731-1-mk@semihalf.com> References: <20211015162701.16324-1-mk@semihalf.com> <20211019105629.11731-1-mk@semihalf.com> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v3 6/7] net/ena: add check for missing Tx completions X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" In some cases Tx descriptors may be uncompleted by the HW and as a result they will never be released. This patch adds checking for the missing Tx completions to the ENA timer service, so in order to use this feature, the application must call the function rte_timer_manage(). Missing Tx completion reset threshold is determined dynamically, by taking into consideration ring size and the default value. Tx cleanup is associated with the Tx burst function. As DPDK applications can call Tx burst function dynamically, time when last cleanup was called must be traced to avoid false detection of the missing Tx completion. Signed-off-by: Michal Krawczyk Reviewed-by: Igor Chauskin Reviewed-by: Shai Brandes --- doc/guides/rel_notes/release_21_11.rst | 1 + drivers/net/ena/ena_ethdev.c | 118 +++++++++++++++++++++++++ drivers/net/ena/ena_ethdev.h | 15 ++++ 3 files changed, 134 insertions(+) diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index 6ac867321b..c5f76081e5 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -109,6 +109,7 @@ New Features * Support for the tx_free_thresh and rx_free_thresh configuration parameters. * NUMA aware allocations for the queue helper structures. + * Watchdog's feature which is checking for missing Tx completions. * **Updated Broadcom bnxt PMD.** diff --git a/drivers/net/ena/ena_ethdev.c b/drivers/net/ena/ena_ethdev.c index 4e9925b6be..1a70cd781c 100644 --- a/drivers/net/ena/ena_ethdev.c +++ b/drivers/net/ena/ena_ethdev.c @@ -99,6 +99,7 @@ static const struct ena_stats ena_stats_tx_strings[] = { ENA_STAT_TX_ENTRY(doorbells), ENA_STAT_TX_ENTRY(bad_req_id), ENA_STAT_TX_ENTRY(available_desc), + ENA_STAT_TX_ENTRY(missed_tx), }; static const struct ena_stats ena_stats_rx_strings[] = { @@ -1164,6 +1165,7 @@ static int ena_tx_queue_setup(struct rte_eth_dev *dev, txq->size_mask = nb_desc - 1; txq->numa_socket_id = socket_id; txq->pkts_without_db = false; + txq->last_cleanup_ticks = 0; txq->tx_buffer_info = rte_zmalloc_socket("txq->tx_buffer_info", sizeof(struct ena_tx_buffer) * txq->ring_size, @@ -1213,6 +1215,9 @@ static int ena_tx_queue_setup(struct rte_eth_dev *dev, txq->ring_size - ENA_REFILL_THRESH_PACKET); } + txq->missing_tx_completion_threshold = + RTE_MIN(txq->ring_size / 2, ENA_DEFAULT_MISSING_COMP); + /* Store pointer to this queue in upper layer */ txq->configured = 1; dev->data->tx_queues[queue_idx] = txq; @@ -1539,6 +1544,85 @@ static void check_for_admin_com_state(struct ena_adapter *adapter) } } +static int check_for_tx_completion_in_queue(struct ena_adapter *adapter, + struct ena_ring *tx_ring) +{ + struct ena_tx_buffer *tx_buf; + uint64_t timestamp; + uint64_t completion_delay; + uint32_t missed_tx = 0; + unsigned int i; + int rc = 0; + + for (i = 0; i < tx_ring->ring_size; ++i) { + tx_buf = &tx_ring->tx_buffer_info[i]; + timestamp = tx_buf->timestamp; + + if (timestamp == 0) + continue; + + completion_delay = rte_get_timer_cycles() - timestamp; + if (completion_delay > adapter->missing_tx_completion_to) { + if (unlikely(!tx_buf->print_once)) { + PMD_TX_LOG(WARNING, + "Found a Tx that wasn't completed on time, qid %d, index %d. Missing Tx outstanding for %" PRIu64 " msecs.\n", + tx_ring->id, i, completion_delay / + rte_get_timer_hz() * 1000); + tx_buf->print_once = true; + } + ++missed_tx; + } + } + + if (unlikely(missed_tx > tx_ring->missing_tx_completion_threshold)) { + PMD_DRV_LOG(ERR, + "The number of lost Tx completions is above the threshold (%d > %d). Trigger the device reset.\n", + missed_tx, + tx_ring->missing_tx_completion_threshold); + adapter->reset_reason = ENA_REGS_RESET_MISS_TX_CMPL; + adapter->trigger_reset = true; + rc = -EIO; + } + + tx_ring->tx_stats.missed_tx += missed_tx; + + return rc; +} + +static void check_for_tx_completions(struct ena_adapter *adapter) +{ + struct ena_ring *tx_ring; + uint64_t tx_cleanup_delay; + size_t qid; + int budget; + uint16_t nb_tx_queues = adapter->edev_data->nb_tx_queues; + + if (adapter->missing_tx_completion_to == ENA_HW_HINTS_NO_TIMEOUT) + return; + + nb_tx_queues = adapter->edev_data->nb_tx_queues; + budget = adapter->missing_tx_completion_budget; + + qid = adapter->last_tx_comp_qid; + while (budget-- > 0) { + tx_ring = &adapter->tx_ring[qid]; + + /* Tx cleanup is called only by the burst function and can be + * called dynamically by the application. Also cleanup is + * limited by the threshold. To avoid false detection of the + * missing HW Tx completion, get the delay since last cleanup + * function was called. + */ + tx_cleanup_delay = rte_get_timer_cycles() - + tx_ring->last_cleanup_ticks; + if (tx_cleanup_delay < adapter->tx_cleanup_stall_delay) + check_for_tx_completion_in_queue(adapter, tx_ring); + qid = (qid + 1) % nb_tx_queues; + } + + adapter->last_tx_comp_qid = qid; +} + static void ena_timer_wd_callback(__rte_unused struct rte_timer *timer, void *arg) { @@ -1547,6 +1631,7 @@ static void ena_timer_wd_callback(__rte_unused struct rte_timer *timer, check_for_missing_keep_alive(adapter); check_for_admin_com_state(adapter); + check_for_tx_completions(adapter); if (unlikely(adapter->trigger_reset)) { PMD_DRV_LOG(ERR, "Trigger reset is on\n"); @@ -1926,6 +2011,20 @@ static int ena_dev_configure(struct rte_eth_dev *dev) */ dev->data->scattered_rx = 1; + adapter->last_tx_comp_qid = 0; + + adapter->missing_tx_completion_budget = + RTE_MIN(ENA_MONITORED_TX_QUEUES, dev->data->nb_tx_queues); + + adapter->missing_tx_completion_to = ENA_TX_TIMEOUT; + /* To avoid detection of the spurious Tx completion timeout due to + * application not calling the Tx cleanup function, set timeout for the + * Tx queue which should be half of the missing completion timeout for a + * safety. If there will be a lot of missing Tx completions in the + * queue, they will be detected sooner or later. + */ + adapter->tx_cleanup_stall_delay = adapter->missing_tx_completion_to / 2; + adapter->tx_selected_offloads = dev->data->dev_conf.txmode.offloads; adapter->rx_selected_offloads = dev->data->dev_conf.rxmode.offloads; @@ -2433,6 +2532,20 @@ static void ena_update_hints(struct ena_adapter *adapter, adapter->ena_dev.mmio_read.reg_read_to = hints->mmio_read_timeout * 1000; + if (hints->missing_tx_completion_timeout) { + if (hints->missing_tx_completion_timeout == + ENA_HW_HINTS_NO_TIMEOUT) { + adapter->missing_tx_completion_to = + ENA_HW_HINTS_NO_TIMEOUT; + } else { + /* Convert from msecs to ticks */ + adapter->missing_tx_completion_to = rte_get_timer_hz() * + hints->missing_tx_completion_timeout / 1000; + adapter->tx_cleanup_stall_delay = + adapter->missing_tx_completion_to / 2; + } + } + if (hints->driver_watchdog_timeout) { if (hints->driver_watchdog_timeout == ENA_HW_HINTS_NO_TIMEOUT) adapter->keep_alive_timeout = ENA_HW_HINTS_NO_TIMEOUT; @@ -2623,6 +2736,7 @@ static int ena_xmit_mbuf(struct ena_ring *tx_ring, struct rte_mbuf *mbuf) } tx_info->tx_descs = nb_hw_desc; + tx_info->timestamp = rte_get_timer_cycles(); tx_ring->tx_stats.cnt++; tx_ring->tx_stats.bytes += mbuf->pkt_len; @@ -2655,6 +2769,7 @@ static void ena_tx_cleanup(struct ena_ring *tx_ring) /* Get Tx info & store how many descs were processed */ tx_info = &tx_ring->tx_buffer_info[req_id]; + tx_info->timestamp = 0; mbuf = tx_info->mbuf; rte_pktmbuf_free(mbuf); @@ -2675,6 +2790,9 @@ static void ena_tx_cleanup(struct ena_ring *tx_ring) ena_com_comp_ack(tx_ring->ena_com_io_sq, total_tx_descs); ena_com_update_dev_comp_head(tx_ring->ena_com_io_cq); } + + /* Notify completion handler that the cleanup was just called */ + tx_ring->last_cleanup_ticks = rte_get_timer_cycles(); } static uint16_t eth_ena_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, diff --git a/drivers/net/ena/ena_ethdev.h b/drivers/net/ena/ena_ethdev.h index 176d713dff..4f4142ed12 100644 --- a/drivers/net/ena/ena_ethdev.h +++ b/drivers/net/ena/ena_ethdev.h @@ -36,6 +36,10 @@ #define ENA_WD_TIMEOUT_SEC 3 #define ENA_DEVICE_KALIVE_TIMEOUT (ENA_WD_TIMEOUT_SEC * rte_get_timer_hz()) +#define ENA_TX_TIMEOUT (5 * rte_get_timer_hz()) +#define ENA_MONITORED_TX_QUEUES 3 +#define ENA_DEFAULT_MISSING_COMP 256U + /* While processing submitted and completed descriptors (rx and tx path * respectively) in a loop it is desired to: * - perform batch submissions while populating sumbissmion queue @@ -75,6 +79,8 @@ struct ena_tx_buffer { struct rte_mbuf *mbuf; unsigned int tx_descs; unsigned int num_of_bufs; + uint64_t timestamp; + bool print_once; struct ena_com_buf bufs[ENA_PKT_MAX_BUFS]; }; @@ -103,6 +109,7 @@ struct ena_stats_tx { u64 doorbells; u64 bad_req_id; u64 available_desc; + u64 missed_tx; }; struct ena_stats_rx { @@ -118,6 +125,7 @@ struct ena_stats_rx { struct ena_ring { u16 next_to_use; u16 next_to_clean; + uint64_t last_cleanup_ticks; enum ena_ring_type type; enum ena_admin_placement_policy_type tx_mem_queue_type; @@ -171,6 +179,8 @@ struct ena_ring { }; unsigned int numa_socket_id; + + uint32_t missing_tx_completion_threshold; } __rte_cache_aligned; enum ena_adapter_state { @@ -291,6 +301,11 @@ struct ena_adapter { bool wd_state; bool use_large_llq_hdr; + + uint32_t last_tx_comp_qid; + uint64_t missing_tx_completion_to; + uint64_t missing_tx_completion_budget; + uint64_t tx_cleanup_stall_delay; }; int ena_rss_reta_update(struct rte_eth_dev *dev,