From patchwork Thu Sep 5 09:05:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mingjin Ye X-Patchwork-Id: 143638 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5267B4590C; Thu, 5 Sep 2024 11:29:20 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0642342E48; Thu, 5 Sep 2024 11:29:18 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) by mails.dpdk.org (Postfix) with ESMTP id 4C7EF42E24 for ; Thu, 5 Sep 2024 11:29:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1725528557; x=1757064557; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bSOxDd/UEBT4AEIQdzIasUN5o/X/DWFJNVSo0R77JT8=; b=dpduDIyUDCwRpdDt060xKdVR48GBkhj0AnzsPvkG9SLmpSMv1u6bPQSp XZuj4vrzkkJuhZgJ8QOVd/0sRsEQ6VjcZnPtuVXn8kFnNQo2ufthIfmE0 M55h60asn1nAieIgzIeNP3c9Sbs+BxvnSP4mj/St5bMWIIlg0Nd6E896Y idhNo0lApH5Q/7J258YbgVED0gaMEQiqdJPNx6NjZZ4j2rBEWpjCP2i23 RDMrFfx1z7iE5EaRsum9N1WtrV2j8rE0VndJCP79mXuoQtyegQao7h4QM J0v9L+YkAg7ajCG3Rpi5uWXSa0O3lbg1t1OX92gQGQcvZz4v9+FYTRdxv w==; X-CSE-ConnectionGUID: GIv8NxcpR4uUbZv+5Mhr/g== X-CSE-MsgGUID: /SkyXu8jS+akXNcXXYS3Eg== X-IronPort-AV: E=McAfee;i="6700,10204,11185"; a="28023264" X-IronPort-AV: E=Sophos;i="6.10,204,1719903600"; d="scan'208";a="28023264" Received: from fmviesa010.fm.intel.com ([10.60.135.150]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Sep 2024 02:29:16 -0700 X-CSE-ConnectionGUID: 4LhsKuLfSuqdcJZ+cnGGRA== X-CSE-MsgGUID: 1g59dgveTT24ov4Mj/Ao7A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,204,1719903600"; d="scan'208";a="65807921" Received: from unknown (HELO localhost.localdomain) ([10.239.252.253]) by fmviesa010-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Sep 2024 02:29:14 -0700 From: Mingjin Ye To: dev@dpdk.org Cc: Mingjin Ye , Simei Su , Thomas Monjalon , Ferruh Yigit , Andrew Rybchenko Subject: [PATCH 1/3] ethdev: add frequency adjustment API Date: Thu, 5 Sep 2024 09:05:43 +0000 Message-Id: <20240905090545.853730-2-mingjinx.ye@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240905090545.853730-1-mingjinx.ye@intel.com> References: <20240905090545.853730-1-mingjinx.ye@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds freq adjustment API for PTP high accuracy. Signed-off-by: Simei Su Signed-off-by: Mingjin Ye --- doc/guides/nics/features.rst | 4 +++- lib/ethdev/ethdev_driver.h | 5 +++++ lib/ethdev/ethdev_trace.h | 9 +++++++++ lib/ethdev/ethdev_trace_points.c | 3 +++ lib/ethdev/rte_ethdev.c | 18 ++++++++++++++++++ lib/ethdev/rte_ethdev.h | 19 +++++++++++++++++++ lib/ethdev/version.map | 1 + 7 files changed, 58 insertions(+), 1 deletion(-) diff --git a/doc/guides/nics/features.rst b/doc/guides/nics/features.rst index cd0115ffb3..0508f118fe 100644 --- a/doc/guides/nics/features.rst +++ b/doc/guides/nics/features.rst @@ -677,10 +677,12 @@ Supports IEEE1588/802.1AS timestamping. * **[implements] eth_dev_ops**: ``timesync_enable``, ``timesync_disable`` ``timesync_read_rx_timestamp``, ``timesync_read_tx_timestamp``, - ``timesync_adjust_time``, ``timesync_read_time``, ``timesync_write_time``. + ``timesync_adjust_time``, ``timesync_adjust_freq``, + ``timesync_read_time``, ``timesync_write_time``. * **[related] API**: ``rte_eth_timesync_enable()``, ``rte_eth_timesync_disable()``, ``rte_eth_timesync_read_rx_timestamp()``, ``rte_eth_timesync_read_tx_timestamp``, ``rte_eth_timesync_adjust_time()``, + ``rte_eth_timesync_adjust_freq()``, ``rte_eth_timesync_read_time()``, ``rte_eth_timesync_write_time()``. diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 883e59a927..68cadc14a5 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -664,6 +664,9 @@ typedef int (*eth_timesync_read_tx_timestamp_t)(struct rte_eth_dev *dev, /** @internal Function used to adjust the device clock. */ typedef int (*eth_timesync_adjust_time)(struct rte_eth_dev *dev, int64_t); +/** @internal Function used to adjust the clock frequency. */ +typedef int (*eth_timesync_adjust_freq)(struct rte_eth_dev *dev, int64_t); + /** @internal Function used to get time from the device clock. */ typedef int (*eth_timesync_read_time)(struct rte_eth_dev *dev, struct timespec *timestamp); @@ -1378,6 +1381,8 @@ struct eth_dev_ops { eth_timesync_read_tx_timestamp_t timesync_read_tx_timestamp; /** Adjust the device clock */ eth_timesync_adjust_time timesync_adjust_time; + /** Adjust the clock frequency */ + eth_timesync_adjust_freq timesync_adjust_freq; /** Get the device clock time */ eth_timesync_read_time timesync_read_time; /** Set the device clock time */ diff --git a/lib/ethdev/ethdev_trace.h b/lib/ethdev/ethdev_trace.h index 3bec87bfdb..e273d5853e 100644 --- a/lib/ethdev/ethdev_trace.h +++ b/lib/ethdev/ethdev_trace.h @@ -2183,6 +2183,15 @@ RTE_TRACE_POINT_FP( rte_trace_point_emit_int(ret); ) +/* Called in loop in examples/ptpclient */ +RTE_TRACE_POINT_FP( + rte_eth_trace_timesync_adjust_freq, + RTE_TRACE_POINT_ARGS(uint16_t port_id, int64_t ppm, int ret), + rte_trace_point_emit_u16(port_id); + rte_trace_point_emit_i64(ppm); + rte_trace_point_emit_int(ret); +) + /* Called in loop in app/test-flow-perf */ RTE_TRACE_POINT_FP( rte_flow_trace_create, diff --git a/lib/ethdev/ethdev_trace_points.c b/lib/ethdev/ethdev_trace_points.c index 99e04f5893..a99fec0c1e 100644 --- a/lib/ethdev/ethdev_trace_points.c +++ b/lib/ethdev/ethdev_trace_points.c @@ -409,6 +409,9 @@ RTE_TRACE_POINT_REGISTER(rte_eth_trace_timesync_read_tx_timestamp, RTE_TRACE_POINT_REGISTER(rte_eth_trace_timesync_adjust_time, lib.ethdev.timesync_adjust_time) +RTE_TRACE_POINT_REGISTER(rte_eth_trace_timesync_adjust_freq, + lib.ethdev.timesync_adjust_freq) + RTE_TRACE_POINT_REGISTER(rte_eth_trace_timesync_read_time, lib.ethdev.timesync_read_time) diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index f1c658f49e..660eab2f1e 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -6310,6 +6310,24 @@ rte_eth_timesync_adjust_time(uint16_t port_id, int64_t delta) return ret; } +int +rte_eth_timesync_adjust_freq(uint16_t port_id, int64_t ppm) +{ + struct rte_eth_dev *dev; + int ret; + + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); + dev = &rte_eth_devices[port_id]; + + if (*dev->dev_ops->timesync_adjust_freq == NULL) + return -ENOTSUP; + ret = eth_err(port_id, (*dev->dev_ops->timesync_adjust_freq)(dev, ppm)); + + rte_eth_trace_timesync_adjust_freq(port_id, ppm, ret); + + return ret; +} + int rte_eth_timesync_read_time(uint16_t port_id, struct timespec *timestamp) { diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index 548fada1c7..42fa2ca39c 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -5297,6 +5297,25 @@ int rte_eth_timesync_read_tx_timestamp(uint16_t port_id, */ int rte_eth_timesync_adjust_time(uint16_t port_id, int64_t delta); +/** + * Adjust the clock increment rate on an Ethernet device. + * + * This is usually used in conjunction with other Ethdev timesync functions to + * synchronize the device time using the IEEE1588/802.1AS protocol. + * + * @param port_id + * The port identifier of the Ethernet device. + * @param ppm + * Parts per million with 16-bit fractional field + * + * @return + * - 0: Success. + * - -ENODEV: The port ID is invalid. + * - -EIO: if device is removed. + * - -ENOTSUP: The function is not supported by the Ethernet driver. + */ +int rte_eth_timesync_adjust_freq(uint16_t port_id, int64_t ppm); + /** * Read the time from the timesync clock on an Ethernet device. * diff --git a/lib/ethdev/version.map b/lib/ethdev/version.map index 1669055ca5..7f751b5685 100644 --- a/lib/ethdev/version.map +++ b/lib/ethdev/version.map @@ -106,6 +106,7 @@ DPDK_25 { rte_eth_stats_get; rte_eth_stats_reset; rte_eth_timesync_adjust_time; + rte_eth_timesync_adjust_freq; rte_eth_timesync_disable; rte_eth_timesync_enable; rte_eth_timesync_read_rx_timestamp; From patchwork Thu Sep 5 09:05:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mingjin Ye X-Patchwork-Id: 143639 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 442724590C; Thu, 5 Sep 2024 11:29:26 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 55B1842E51; Thu, 5 Sep 2024 11:29:20 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) by mails.dpdk.org (Postfix) with ESMTP id 2403342E4C for ; Thu, 5 Sep 2024 11:29:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1725528558; x=1757064558; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Y+IN6HyUEVsxJEu2VCwTXtVvIUQdftvB40ulLdySPKk=; b=I1B1fN310MUMf/WprrrpkUmrrmalPI7p3n7ztMHEC/kGjNXObOOgC2ll ok5YgGZiZWwEsslHbXiT/iNLpywADaHpJDEU/EdYrAAjSNOqAbocjjOIE 9pxhavIcrRygHcgz5BGC7k8MHklViLZW4B5RYhQf32AMgUKhN4hGfFbrV cYw2Ppk+UtKt10Vzg+ebtuAaQ6O5K/pUDjb3G6cwb9mglgq4l12zdDySF /xqu2uQ/Avdgpa6+ofU7Jmq57L1jZ7swFo32wij7v+T6elE7iu/tRhK/Z ZUkgnZXHwVP4fBpF17rwCkjZOhtQFz0UsREKB0YL0Y7c6Qz2HmsibGzKa Q==; X-CSE-ConnectionGUID: aqdia42wSdGwc1JPPaODSw== X-CSE-MsgGUID: Ljmqsaw7QIeba7/HIdyArw== X-IronPort-AV: E=McAfee;i="6700,10204,11185"; a="28023274" X-IronPort-AV: E=Sophos;i="6.10,204,1719903600"; d="scan'208";a="28023274" Received: from fmviesa010.fm.intel.com ([10.60.135.150]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Sep 2024 02:29:18 -0700 X-CSE-ConnectionGUID: 8WaSNdiMQOWD6Dc+ECUnbw== X-CSE-MsgGUID: /0Qp24nTSSm2Cau1s0JAeA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,204,1719903600"; d="scan'208";a="65807932" Received: from unknown (HELO localhost.localdomain) ([10.239.252.253]) by fmviesa010-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Sep 2024 02:29:16 -0700 From: Mingjin Ye To: dev@dpdk.org Cc: Mingjin Ye , Simei Su Subject: [PATCH 2/3] net/ice: add frequency adjustment support for PTP Date: Thu, 5 Sep 2024 09:05:44 +0000 Message-Id: <20240905090545.853730-3-mingjinx.ye@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240905090545.853730-1-mingjinx.ye@intel.com> References: <20240905090545.853730-1-mingjinx.ye@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add ice support for new ethdev API to adjust frequency for IEEE1588 PTP. Also, this patch reworks code for converting software update to hardware update. Signed-off-by: Simei Su Signed-off-by: Mingjin Ye --- doc/guides/nics/ice.rst | 15 +++ drivers/net/ice/ice_ethdev.c | 177 ++++++++++++++++++++++++++--------- drivers/net/ice/ice_ethdev.h | 2 + drivers/net/ice/ice_rxtx.c | 4 +- 4 files changed, 153 insertions(+), 45 deletions(-) diff --git a/doc/guides/nics/ice.rst b/doc/guides/nics/ice.rst index ae975d19ad..dec4cc2434 100644 --- a/doc/guides/nics/ice.rst +++ b/doc/guides/nics/ice.rst @@ -328,6 +328,21 @@ Forward Error Correction (FEC) Supports get/set FEC mode and get FEC capability. +Time Synchronisation +~~~~~~~~~~~~~~~~~~~~ + +The system operator can run a PTP (Precision Time Protocol) client application +to synchronise the time on the network card (and optionally the time on the +system) to the PTP master. + +ICE PMD supports PTP client applications that use the DPDK IEEE1588 API to +communicate with the PTP master clock. Note that the PTP client application +needs to run on the PF and vector mode needs to be disabled. + +.. code-block:: console + + examples/dpdk-ptpclient -c f -n 3 -a 0000:ec:00.1 -- -T 1 -p 0x1 --controller=pi + Generic Flow Support ~~~~~~~~~~~~~~~~~~~~ diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c index 304f959b7e..33293a917f 100644 --- a/drivers/net/ice/ice_ethdev.c +++ b/drivers/net/ice/ice_ethdev.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -176,6 +177,7 @@ static int ice_timesync_read_rx_timestamp(struct rte_eth_dev *dev, static int ice_timesync_read_tx_timestamp(struct rte_eth_dev *dev, struct timespec *timestamp); static int ice_timesync_adjust_time(struct rte_eth_dev *dev, int64_t delta); +static int ice_timesync_adjust_freq(struct rte_eth_dev *dev, int64_t ppm); static int ice_timesync_read_time(struct rte_eth_dev *dev, struct timespec *timestamp); static int ice_timesync_write_time(struct rte_eth_dev *dev, @@ -307,6 +309,7 @@ static const struct eth_dev_ops ice_eth_dev_ops = { .timesync_read_rx_timestamp = ice_timesync_read_rx_timestamp, .timesync_read_tx_timestamp = ice_timesync_read_tx_timestamp, .timesync_adjust_time = ice_timesync_adjust_time, + .timesync_adjust_freq = ice_timesync_adjust_freq, .timesync_read_time = ice_timesync_read_time, .timesync_write_time = ice_timesync_write_time, .timesync_disable = ice_timesync_disable, @@ -2332,6 +2335,34 @@ ice_get_supported_rxdid(struct ice_hw *hw) return supported_rxdid; } +static void ice_ptp_init_info(struct rte_eth_dev *dev) +{ + struct ice_hw *hw = ICE_DEV_PRIVATE_TO_HW(dev->data->dev_private); + struct ice_adapter *ad = + ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private); + + switch (hw->phy_model) { + case ICE_PHY_ETH56G: + ad->ptp_tx_block = hw->pf_id; + ad->ptp_tx_index = 0; + break; + case ICE_PHY_E810: + /* fallthrough */ + case ICE_PHY_E830: + ad->ptp_tx_block = hw->port_info->lport; + ad->ptp_tx_index = 0; + break; + case ICE_PHY_E822: + ad->ptp_tx_block = hw->pf_id / ICE_PORTS_PER_QUAD; + ad->ptp_tx_index = (hw->pf_id % ICE_PORTS_PER_QUAD) * + ICE_PORTS_PER_PHY_E822 * ICE_QUADS_PER_PHY_E822; + break; + default: + PMD_DRV_LOG(WARNING, "Unsupported PHY model"); + break; + } +} + static int ice_dev_init(struct rte_eth_dev *dev) { @@ -2499,6 +2530,9 @@ ice_dev_init(struct rte_eth_dev *dev) /* Initialize PHY model */ ice_ptp_init_phy_model(hw); + /* Initialize PTP info */ + ice_ptp_init_info(dev); + if (hw->phy_model == ICE_PHY_E822) { ret = ice_start_phy_timer_e822(hw, hw->pf_id); if (ret) @@ -6466,23 +6500,6 @@ ice_timesync_enable(struct rte_eth_dev *dev) } } - /* Initialize cycle counters for system time/RX/TX timestamp */ - memset(&ad->systime_tc, 0, sizeof(struct rte_timecounter)); - memset(&ad->rx_tstamp_tc, 0, sizeof(struct rte_timecounter)); - memset(&ad->tx_tstamp_tc, 0, sizeof(struct rte_timecounter)); - - ad->systime_tc.cc_mask = ICE_CYCLECOUNTER_MASK; - ad->systime_tc.cc_shift = 0; - ad->systime_tc.nsec_mask = 0; - - ad->rx_tstamp_tc.cc_mask = ICE_CYCLECOUNTER_MASK; - ad->rx_tstamp_tc.cc_shift = 0; - ad->rx_tstamp_tc.nsec_mask = 0; - - ad->tx_tstamp_tc.cc_mask = ICE_CYCLECOUNTER_MASK; - ad->tx_tstamp_tc.cc_shift = 0; - ad->tx_tstamp_tc.nsec_mask = 0; - ad->ptp_ena = 1; return 0; @@ -6497,14 +6514,13 @@ ice_timesync_read_rx_timestamp(struct rte_eth_dev *dev, ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private); struct ice_rx_queue *rxq; uint32_t ts_high; - uint64_t ts_ns, ns; + uint64_t ts_ns; rxq = dev->data->rx_queues[flags]; ts_high = rxq->time_high; ts_ns = ice_tstamp_convert_32b_64b(hw, ad, 1, ts_high); - ns = rte_timecounter_update(&ad->rx_tstamp_tc, ts_ns); - *timestamp = rte_ns_to_timespec(ns); + *timestamp = rte_ns_to_timespec(ts_ns); return 0; } @@ -6516,22 +6532,18 @@ ice_timesync_read_tx_timestamp(struct rte_eth_dev *dev, struct ice_hw *hw = ICE_DEV_PRIVATE_TO_HW(dev->data->dev_private); struct ice_adapter *ad = ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private); - uint8_t lport; - uint64_t ts_ns, ns, tstamp; + uint64_t ts_ns, tstamp; const uint64_t mask = 0xFFFFFFFF; int ret; - lport = hw->port_info->lport; - - ret = ice_read_phy_tstamp(hw, lport, 0, &tstamp); - if (ret) { + ret = ice_read_phy_tstamp(hw, ad->ptp_tx_block, ad->ptp_tx_index, &tstamp); + if (ret || tstamp == 0) { PMD_DRV_LOG(ERR, "Failed to read phy timestamp"); return -1; } ts_ns = ice_tstamp_convert_32b_64b(hw, ad, 1, (tstamp >> 8) & mask); - ns = rte_timecounter_update(&ad->tx_tstamp_tc, ts_ns); - *timestamp = rte_ns_to_timespec(ns); + *timestamp = rte_ns_to_timespec(ts_ns); return 0; } @@ -6539,28 +6551,108 @@ ice_timesync_read_tx_timestamp(struct rte_eth_dev *dev, static int ice_timesync_adjust_time(struct rte_eth_dev *dev, int64_t delta) { - struct ice_adapter *ad = - ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private); + struct ice_hw *hw = ICE_DEV_PRIVATE_TO_HW(dev->data->dev_private); + uint8_t tmr_idx = hw->func_caps.ts_func_info.tmr_index_assoc; + uint32_t lo, lo2, hi; + uint64_t time, ns; + int ret; + + if (delta > INT32_MAX || delta < INT32_MIN) { + lo = ICE_READ_REG(hw, GLTSYN_TIME_L(tmr_idx)); + hi = ICE_READ_REG(hw, GLTSYN_TIME_H(tmr_idx)); + lo2 = ICE_READ_REG(hw, GLTSYN_TIME_L(tmr_idx)); + + if (lo2 < lo) { + lo = ICE_READ_REG(hw, GLTSYN_TIME_L(tmr_idx)); + hi = ICE_READ_REG(hw, GLTSYN_TIME_H(tmr_idx)); + } + + time = ((uint64_t)hi << 32) | lo; + ns = time + delta; + + wr32(hw, GLTSYN_SHTIME_L(tmr_idx), ICE_LO_DWORD(ns)); + wr32(hw, GLTSYN_SHTIME_H(tmr_idx), ICE_HI_DWORD(ns)); + wr32(hw, GLTSYN_SHTIME_0(tmr_idx), 0); + + ret = ice_ptp_init_time(hw, ns, true); + if (ret) { + PMD_DRV_LOG(ERR, "PTP init time failed, err %d", ret); + return -1; + } + return 0; + } + + ret = ice_ptp_adj_clock(hw, delta, true); + if (ret) { + PMD_DRV_LOG(ERR, "PTP adj clock failed, err %d", ret); + return -1; + } + + return 0; +} + +static int +ice_timesync_adjust_freq(struct rte_eth_dev *dev, int64_t ppm) +{ + struct ice_hw *hw = ICE_DEV_PRIVATE_TO_HW(dev->data->dev_private); + int64_t incval, diff = 0; + bool negative = false; + uint64_t div, rem; + uint64_t divisor = 1000000ULL << 16; + int shift; + int ret; + + incval = ice_get_base_incval(hw, ICE_SRC_TMR_MODE_NANOSECONDS); + + if (ppm < 0) { + negative = true; + ppm = -ppm; + } + + /* can incval * ppm overflow ? */ + if (log2(incval) + log2(ppm) > 62) { + rem = ppm % divisor; + div = ppm / divisor; + diff = div * incval; + ppm = rem; - ad->systime_tc.nsec += delta; - ad->rx_tstamp_tc.nsec += delta; - ad->tx_tstamp_tc.nsec += delta; + shift = log2(incval) + log2(ppm) - 62; + if (shift > 0) { + /* drop precision */ + ppm >>= shift; + divisor >>= shift; + } + } + + if (divisor) + diff = diff + incval * ppm / divisor; + + if (negative) + incval -= diff; + else + incval += diff; + ret = ice_ptp_write_incval_locked(hw, incval, true); + if (ret) { + PMD_DRV_LOG(ERR, "PTP failed to set incval, err %d", ret); + return -1; + } return 0; } static int ice_timesync_write_time(struct rte_eth_dev *dev, const struct timespec *ts) { - struct ice_adapter *ad = - ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private); + struct ice_hw *hw = ICE_DEV_PRIVATE_TO_HW(dev->data->dev_private); uint64_t ns; + int ret; ns = rte_timespec_to_ns(ts); - - ad->systime_tc.nsec = ns; - ad->rx_tstamp_tc.nsec = ns; - ad->tx_tstamp_tc.nsec = ns; + ret = ice_ptp_init_time(hw, ns, true); + if (ret) { + PMD_DRV_LOG(ERR, "PTP init time failed, err %d", ret); + return -1; + } return 0; } @@ -6569,11 +6661,9 @@ static int ice_timesync_read_time(struct rte_eth_dev *dev, struct timespec *ts) { struct ice_hw *hw = ICE_DEV_PRIVATE_TO_HW(dev->data->dev_private); - struct ice_adapter *ad = - ICE_DEV_PRIVATE_TO_ADAPTER(dev->data->dev_private); uint8_t tmr_idx = hw->func_caps.ts_func_info.tmr_index_assoc; uint32_t hi, lo, lo2; - uint64_t time, ns; + uint64_t time; lo = ICE_READ_REG(hw, GLTSYN_TIME_L(tmr_idx)); hi = ICE_READ_REG(hw, GLTSYN_TIME_H(tmr_idx)); @@ -6585,8 +6675,7 @@ ice_timesync_read_time(struct rte_eth_dev *dev, struct timespec *ts) } time = ((uint64_t)hi << 32) | lo; - ns = rte_timecounter_update(&ad->systime_tc, time); - *ts = rte_ns_to_timespec(ns); + *ts = rte_ns_to_timespec(time); return 0; } diff --git a/drivers/net/ice/ice_ethdev.h b/drivers/net/ice/ice_ethdev.h index 3ea9f37dc8..db192e76fe 100644 --- a/drivers/net/ice/ice_ethdev.h +++ b/drivers/net/ice/ice_ethdev.h @@ -614,6 +614,8 @@ struct ice_adapter { struct rte_timecounter systime_tc; struct rte_timecounter rx_tstamp_tc; struct rte_timecounter tx_tstamp_tc; + uint8_t ptp_tx_block; + uint8_t ptp_tx_index; bool ptp_ena; uint64_t time_hw; struct ice_fdir_prof_info fdir_prof_info[ICE_MAX_PTGS]; diff --git a/drivers/net/ice/ice_rxtx.c b/drivers/net/ice/ice_rxtx.c index f270498ed1..5f399e8aae 100644 --- a/drivers/net/ice/ice_rxtx.c +++ b/drivers/net/ice/ice_rxtx.c @@ -3066,7 +3066,9 @@ ice_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts) else if (ol_flags & RTE_MBUF_F_TX_IEEE1588_TMST) cd_type_cmd_tso_mss |= ((uint64_t)ICE_TX_CTX_DESC_TSYN << - ICE_TXD_CTX_QW1_CMD_S); + ICE_TXD_CTX_QW1_CMD_S) | + (((uint64_t)txq->vsi->adapter->ptp_tx_index << + ICE_TXD_CTX_QW1_TSYN_S) & ICE_TXD_CTX_QW1_TSYN_M); ctx_txd->tunneling_params = rte_cpu_to_le_32(cd_tunneling_params); From patchwork Thu Sep 5 09:05:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mingjin Ye X-Patchwork-Id: 143640 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4153C4590C; Thu, 5 Sep 2024 11:29:35 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 314FE42E61; Thu, 5 Sep 2024 11:29:22 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.14]) by mails.dpdk.org (Postfix) with ESMTP id 2707A42E45 for ; Thu, 5 Sep 2024 11:29:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1725528560; x=1757064560; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=aWHMWwJX/4fxfYXlDWKBQSFvsmrNl16PjHpvefPTP6w=; b=ZfXkEzAiDU7O9avSP3JYAf8HPcrmcdvnxFOpaykASgdH+dfv1xb99ZNU jTczEEFv0rgfvO2ynsGi+6pGRN5WZF/GBVMco+znoNpA2LD4RJc7eRpxE XOjBHsLJrWwK/3Oa4jhMootNgUy8GdFvsm3yWBX1agpads8jrBssaCivd E5kxySi0HVh+0+v9FtxdRkougOQ1lQedeEhhsd8STsXlaaz2Igvb6WbR8 80zdkJBMv9uSLMxsl7czK4ZjQPxLZZOjcOcReqUqrJrt+mPCJO8exix5y SmeXBFCUiKTZ0xAUQpXgkjwDsmrlD6x1NfbvccNepF60iTnlxmimwqE2I A==; X-CSE-ConnectionGUID: TYAzhWWgTRabYbdB7k6/AQ== X-CSE-MsgGUID: kVQpdTSGTdWrr9Bo4ve0yQ== X-IronPort-AV: E=McAfee;i="6700,10204,11185"; a="28023279" X-IronPort-AV: E=Sophos;i="6.10,204,1719903600"; d="scan'208";a="28023279" Received: from fmviesa010.fm.intel.com ([10.60.135.150]) by orvoesa106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Sep 2024 02:29:20 -0700 X-CSE-ConnectionGUID: Ocz7K//+Ss6ZnSKVWkV7xA== X-CSE-MsgGUID: dAjqqPnyT5iljVra056D/w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,204,1719903600"; d="scan'208";a="65807947" Received: from unknown (HELO localhost.localdomain) ([10.239.252.253]) by fmviesa010-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Sep 2024 02:29:18 -0700 From: Mingjin Ye To: dev@dpdk.org Cc: Mingjin Ye , Simei Su , Wenjun Wu , Kirill Rybalchenko Subject: [PATCH 3/3] examples/ptpclient: add frequency adjustment support Date: Thu, 5 Sep 2024 09:05:45 +0000 Message-Id: <20240905090545.853730-4-mingjinx.ye@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240905090545.853730-1-mingjinx.ye@intel.com> References: <20240905090545.853730-1-mingjinx.ye@intel.com> MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch adds PI servo algorithm to support frequency adjustment API for IEEE1588 PTP. For example, the command for starting ptpclient with PI algorithm is: ./build/examples/dpdk-ptpclient -a 0000:81:00.0 -c 1 -n 3 -- -T 0 -p 0x1 --controller=pi Signed-off-by: Simei Su Signed-off-by: Wenjun Wu Signed-off-by: Mingjin Ye --- examples/ptpclient/ptpclient.c | 300 +++++++++++++++++++++++++++++---- 1 file changed, 265 insertions(+), 35 deletions(-) diff --git a/examples/ptpclient/ptpclient.c b/examples/ptpclient/ptpclient.c index afb61bba51..b97b858310 100644 --- a/examples/ptpclient/ptpclient.c +++ b/examples/ptpclient/ptpclient.c @@ -46,6 +46,35 @@ static volatile bool force_quit; #define KERNEL_TIME_ADJUST_LIMIT 20000 #define PTP_PROTOCOL 0x88F7 +#define KP 0.7 +#define KI 0.3 +#define FREQ_EST_MARGIN 0.001 + +enum servo_state { + SERVO_UNLOCKED, + SERVO_JUMP, + SERVO_LOCKED, +}; + +struct pi_servo { + double offset[2]; + double local[2]; + double drift; + double last_freq; + int count; + + double max_frequency; + double step_threshold; + double first_step_threshold; + int first_update; +}; + +enum controller_mode { + MODE_NONE, + MODE_PI, + MAX_ALL +} mode = MODE_PI; + struct rte_mempool *mbuf_pool; uint32_t ptp_enabled_port_mask; uint8_t ptp_enabled_port_nb; @@ -135,6 +164,9 @@ struct ptpv2_data_slave_ordinary { uint8_t ptpset; uint8_t kernel_time_set; uint16_t current_ptp_port; + int64_t master_offset; + int64_t path_delay; + struct pi_servo *servo; }; static struct ptpv2_data_slave_ordinary ptp_data; @@ -293,36 +325,44 @@ print_clock_info(struct ptpv2_data_slave_ordinary *ptp_data) ptp_data->tstamp3.tv_sec, (ptp_data->tstamp3.tv_nsec)); - printf("\nT4 - Master Clock. %lds %ldns ", + printf("\nT4 - Master Clock. %lds %ldns\n", ptp_data->tstamp4.tv_sec, (ptp_data->tstamp4.tv_nsec)); - printf("\nDelta between master and slave clocks:%"PRId64"ns\n", + if (mode == MODE_NONE) { + printf("\nDelta between master and slave clocks:%"PRId64"ns\n", ptp_data->delta); - clock_gettime(CLOCK_REALTIME, &sys_time); - rte_eth_timesync_read_time(ptp_data->current_ptp_port, &net_time); + clock_gettime(CLOCK_REALTIME, &sys_time); + rte_eth_timesync_read_time(ptp_data->current_ptp_port, + &net_time); - time_t ts = net_time.tv_sec; + time_t ts = net_time.tv_sec; - printf("\n\nComparison between Linux kernel Time and PTP:"); + printf("\n\nComparison between Linux kernel Time and PTP:"); - printf("\nCurrent PTP Time: %.24s %.9ld ns", + printf("\nCurrent PTP Time: %.24s %.9ld ns", ctime(&ts), net_time.tv_nsec); - nsec = (int64_t)timespec64_to_ns(&net_time) - + nsec = (int64_t)timespec64_to_ns(&net_time) - (int64_t)timespec64_to_ns(&sys_time); - ptp_data->new_adj = ns_to_timeval(nsec); + ptp_data->new_adj = ns_to_timeval(nsec); - gettimeofday(&ptp_data->new_adj, NULL); + gettimeofday(&ptp_data->new_adj, NULL); - time_t tp = ptp_data->new_adj.tv_sec; + time_t tp = ptp_data->new_adj.tv_sec; - printf("\nCurrent SYS Time: %.24s %.6ld ns", - ctime(&tp), ptp_data->new_adj.tv_usec); + printf("\nCurrent SYS Time: %.24s %.6ld ns", + ctime(&tp), ptp_data->new_adj.tv_usec); - printf("\nDelta between PTP and Linux Kernel time:%"PRId64"ns\n", - nsec); + printf("\nDelta between PTP and Linux Kernel time:%"PRId64"ns\n", + nsec); + } + + if (mode == MODE_PI) { + printf("path delay: %"PRId64"ns\n", ptp_data->path_delay); + printf("master offset: %"PRId64"ns\n", ptp_data->master_offset); + } printf("[Ctrl+C to quit]\n"); @@ -385,21 +425,11 @@ parse_sync(struct ptpv2_data_slave_ordinary *ptp_data, uint16_t rx_tstamp_idx) static void parse_fup(struct ptpv2_data_slave_ordinary *ptp_data) { - struct rte_ether_hdr *eth_hdr; - struct rte_ether_addr eth_addr; struct ptp_header *ptp_hdr; - struct clock_id *client_clkid; struct ptp_message *ptp_msg; - struct delay_req_msg *req_msg; - struct rte_mbuf *created_pkt; struct tstamp *origin_tstamp; - struct rte_ether_addr eth_multicast = ether_multicast; - size_t pkt_size; - int wait_us; struct rte_mbuf *m = ptp_data->m; - int ret; - eth_hdr = rte_pktmbuf_mtod(m, struct rte_ether_hdr *); ptp_hdr = rte_pktmbuf_mtod_offset(m, struct ptp_header *, sizeof(struct rte_ether_hdr)); if (memcmp(&ptp_data->master_clock_id, @@ -416,6 +446,150 @@ parse_fup(struct ptpv2_data_slave_ordinary *ptp_data) ptp_data->tstamp1.tv_sec = ((uint64_t)ntohl(origin_tstamp->sec_lsb)) | (((uint64_t)ntohs(origin_tstamp->sec_msb)) << 32); +} + +static double +pi_sample(struct pi_servo *s, int64_t offset, double local_ts, + enum servo_state *state) +{ + double ki_term, ppb = s->last_freq; + double freq_est_interval, localdiff; + + switch (s->count) { + case 0: + s->offset[0] = offset; + s->local[0] = local_ts; + *state = SERVO_UNLOCKED; + s->count = 1; + break; + case 1: + s->offset[1] = offset; + s->local[1] = local_ts; + + /* Make sure the first sample is older than the second. */ + if (s->local[0] >= s->local[1]) { + *state = SERVO_UNLOCKED; + s->count = 0; + break; + } + + /* Wait long enough before estimating the frequency offset. */ + localdiff = (s->local[1] - s->local[0]) / 1e9; + localdiff += localdiff * FREQ_EST_MARGIN; + freq_est_interval = 0.016 / KI; + if (freq_est_interval > 1000.0) + freq_est_interval = 1000.0; + + if (localdiff < freq_est_interval) { + *state = SERVO_UNLOCKED; + break; + } + + /* Adjust drift by the measured frequency offset. */ + s->drift += (1e9 - s->drift) * (s->offset[1] - s->offset[0]) / + (s->local[1] - s->local[0]); + + if (s->drift < -s->max_frequency) + s->drift = -s->max_frequency; + else if (s->drift > s->max_frequency) + s->drift = s->max_frequency; + + if ((s->first_update && + s->first_step_threshold && + s->first_step_threshold < llabs(offset)) || + (s->step_threshold && + s->step_threshold < llabs(offset))) + *state = SERVO_JUMP; + else + *state = SERVO_LOCKED; + + ppb = s->drift; + s->count = 2; + break; + case 2: + /* + * reset the clock servo when offset is greater than the max + * offset value. Note that the clock jump will be performed in + * step 1, so it is not necessary to have clock jump + * immediately. This allows re-calculating drift as in initial + * clock startup. + */ + if (s->step_threshold && + s->step_threshold < llabs(offset)) { + *state = SERVO_UNLOCKED; + s->count = 0; + break; + } + + ki_term = KI * offset; + ppb = KP * offset + s->drift + ki_term; + if (ppb < -s->max_frequency) + ppb = -s->max_frequency; + else if (ppb > s->max_frequency) + ppb = s->max_frequency; + else + s->drift += ki_term; + + *state = SERVO_LOCKED; + break; + } + + s->last_freq = ppb; + return ppb; +} + +static void +ptp_adjust_freq(struct ptpv2_data_slave_ordinary *ptp_data) +{ + uint64_t t1_ns, t2_ns; + double adj_freq; + enum servo_state state = SERVO_UNLOCKED; + + t1_ns = timespec64_to_ns(&ptp_data->tstamp1); + t2_ns = timespec64_to_ns(&ptp_data->tstamp2); + ptp_data->master_offset = t2_ns - t1_ns - ptp_data->path_delay; + if (!ptp_data->path_delay) + return; + + adj_freq = pi_sample(ptp_data->servo, ptp_data->master_offset, t2_ns, + &state); + + switch (state) { + case SERVO_UNLOCKED: + break; + case SERVO_JUMP: + ptp_data->servo->first_update = 0; + rte_eth_timesync_adjust_freq(ptp_data->portid, + -(long)(adj_freq * 65.536)); + rte_eth_timesync_adjust_time(ptp_data->portid, + -ptp_data->master_offset); + break; + case SERVO_LOCKED: + ptp_data->servo->first_update = 0; + rte_eth_timesync_adjust_freq(ptp_data->portid, + -(long)(adj_freq * 65.536)); + break; + } +} + +static void +send_delay_request(struct ptpv2_data_slave_ordinary *ptp_data) +{ + struct rte_ether_hdr *eth_hdr; + struct rte_ether_addr eth_addr; + struct ptp_header *ptp_hdr; + struct clock_id *client_clkid; + struct delay_req_msg *req_msg; + struct rte_mbuf *created_pkt; + struct rte_ether_addr eth_multicast = ether_multicast; + size_t pkt_size; + int wait_us; + struct rte_mbuf *m = ptp_data->m; + int ret; + + eth_hdr = rte_pktmbuf_mtod(m, struct rte_ether_hdr *); + ptp_hdr = (struct ptp_header *)(rte_pktmbuf_mtod(m, char *) + + sizeof(struct rte_ether_hdr)); if (ptp_data->seqID_FOLLOWUP == ptp_data->seqID_SYNC) { ret = rte_eth_macaddr_get(ptp_data->portid, ð_addr); @@ -488,11 +662,10 @@ parse_fup(struct ptpv2_data_slave_ordinary *ptp_data) ptp_data->tstamp3.tv_sec = 0; /* Wait at least 1 us to read TX timestamp. */ - while ((rte_eth_timesync_read_tx_timestamp(ptp_data->portid, - &ptp_data->tstamp3) < 0) && (wait_us < 1000)) { + do { rte_delay_us(1); - wait_us++; - } + } while ((rte_eth_timesync_read_tx_timestamp(ptp_data->portid, + &ptp_data->tstamp3) < 0) && (wait_us++ < 10)); } } @@ -529,6 +702,25 @@ update_kernel_time(void) } +static void +clock_path_delay(struct ptpv2_data_slave_ordinary *ptp_data) +{ + uint64_t t1_ns, t2_ns, t3_ns, t4_ns; + int64_t pd, diff; + + t1_ns = timespec64_to_ns(&ptp_data->tstamp1); + t2_ns = timespec64_to_ns(&ptp_data->tstamp2); + t3_ns = timespec64_to_ns(&ptp_data->tstamp3); + t4_ns = timespec64_to_ns(&ptp_data->tstamp4); + + pd = (t2_ns - t3_ns) + (t4_ns - t1_ns); + diff = t3_ns - t2_ns; + if (diff <= INT32_MAX && diff >= INT32_MIN) + ptp_data->path_delay = pd / 2; + else + ptp_data->path_delay = 0; +} + /* * Parse the DELAY_RESP message. */ @@ -541,7 +733,7 @@ parse_drsp(struct ptpv2_data_slave_ordinary *ptp_data) uint16_t seq_id; ptp_msg = rte_pktmbuf_mtod_offset(m, struct ptp_message *, - sizeof(struct rte_ether_hdr)); + sizeof(struct rte_ether_hdr)); seq_id = rte_be_to_cpu_16(ptp_msg->delay_resp.hdr.seq_id); if (memcmp(&ptp_data->client_clock_id, &ptp_msg->delay_resp.req_port_id.clock_id, @@ -553,11 +745,15 @@ parse_drsp(struct ptpv2_data_slave_ordinary *ptp_data) ((uint64_t)ntohl(rx_tstamp->sec_lsb)) | (((uint64_t)ntohs(rx_tstamp->sec_msb)) << 32); - /* Evaluate the delta for adjustment. */ - ptp_data->delta = delta_eval(ptp_data); + if (mode == MODE_PI) { + clock_path_delay(ptp_data); + } else { + /* Evaluate the delta for adjustment. */ + ptp_data->delta = delta_eval(ptp_data); - rte_eth_timesync_adjust_time(ptp_data->portid, - ptp_data->delta); + rte_eth_timesync_adjust_time(ptp_data->portid, + ptp_data->delta); + } ptp_data->current_ptp_port = ptp_data->portid; @@ -597,6 +793,9 @@ parse_ptp_frames(uint16_t portid, struct rte_mbuf *m) { break; case FOLLOW_UP: parse_fup(&ptp_data); + if (mode == MODE_PI) + ptp_adjust_freq(&ptp_data); + send_delay_request(&ptp_data); break; case DELAY_RESP: parse_drsp(&ptp_data); @@ -688,6 +887,18 @@ parse_ptp_kernel(const char *param) return 1; } +static void +servo_init(struct pi_servo *servo) +{ + memset(servo, 0x00, sizeof(*servo)); + + servo->drift = 100000000; + servo->last_freq = 100000000; + servo->count = 0; + servo->max_frequency = 100000000; + servo->step_threshold = 0.1 * NSEC_PER_SEC; +} + /* Parse the commandline arguments. */ static int ptp_parse_args(int argc, char **argv) @@ -696,7 +907,10 @@ ptp_parse_args(int argc, char **argv) char **argvopt; int option_index; char *prgname = argv[0]; - static struct option lgopts[] = { {NULL, 0, 0, 0} }; + static struct option lgopts[] = { + {"controller", 1, 0, 0}, + {NULL, 0, 0, 0} + }; argvopt = argv; @@ -724,6 +938,11 @@ ptp_parse_args(int argc, char **argv) ptp_data.kernel_time_set = ret; break; + case 0: + if (!strcmp(lgopts[option_index].name, "controller")) + if (!strcmp(optarg, "pi")) + mode = MODE_PI; + break; default: print_usage(prgname); @@ -778,6 +997,14 @@ main(int argc, char *argv[]) rte_exit(EXIT_FAILURE, "Error with PTP initialization\n"); /* >8 End of parsing specific arguments. */ + if (mode == MODE_PI) { + ptp_data.servo = malloc(sizeof(*(ptp_data.servo))); + if (!ptp_data.servo) + rte_exit(EXIT_FAILURE, "no memory for servo\n"); + + servo_init(ptp_data.servo); + } + /* Check that there is an even number of ports to send/receive on. */ nb_ports = rte_eth_dev_count_avail(); @@ -831,6 +1058,9 @@ main(int argc, char *argv[]) rte_eth_dev_close(portid); } + if (mode == MODE_PI) + free(ptp_data.servo); + /* clean up the EAL */ rte_eal_cleanup();