net/ice: fix memzone leak when firmware is missing
Checks
Commit Message
Caught by our QE.
When the firmware is missing, memzones were not released.
$ dpdk-testpmd -c 0x1f -a 0:0:0.0 -- -i
...
testpmd> dump_memzone
...
Zone 6: name:<RTE_METRICS>, len:0x15040, virt:0x1661b24c0, socket_id:0,
flags:0
physical segments used:
addr: 0x140000000 iova: 0x140000000 len: 0x40000000 pagesz: 0x40000000
testpmd> port attach 0000:5e:00.0
Attaching a new port...
EAL: Using IOMMU type 1 (Type 1)
EAL: Probe PCI driver: net_ice (8086:159b) device: 0000:5e:00.0 (socket 0)
ice_load_pkg(): failed to open file: /lib/firmware/intel/ice/ddp/ice.pkg
ice_dev_init(): Failed to load the DDP package,Use safe-mode-support=1 to
enter Safe Mode
EAL: Releasing PCI mapped resource for 0000:5e:00.0
EAL: Calling pci_unmap_resource for 0000:5e:00.0 at 0x2200000000
EAL: Calling pci_unmap_resource for 0000:5e:00.0 at 0x2202000000
EAL: Driver cannot attach the device (0000:5e:00.0)
EAL: Failed to attach device on primary process
testpmd: Failed to attach port 0000:5e:00.0
testpmd> dump_memzone
...
Zone 139: name:<ice_dma_17168374657430093156>, len:0x1000,
virt:0x1660ed800, socket_id:0, flags:0 physical segments used:
addr: 0x140000000 iova: 0x140000000 len: 0x40000000 pagesz: 0x40000000
With 20 tries attaching a net/ice port, we would end up with:
EAL: Probe PCI driver: net_ice (8086:159b) device: 0000:5e:00.0 (socket 0)
EAL: memzone_reserve_aligned_thread_unsafe(): Number of requested memzone
segments exceeds RTE_MAX_MEMZONE
ice_dev_init(): Failed to initialize HW
Fixes: a4c8c48fe3f4 ("net/ice: load OS default package")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
---
drivers/net/ice/ice_ethdev.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
Comments
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Tuesday, July 6, 2021 16:57
> To: dev@dpdk.org
> Cc: Wang, Haiyue <haiyue.wang@intel.com>; stable@dpdk.org; Yang, Qiming <qiming.yang@intel.com>; Zhang,
> Qi Z <qi.z.zhang@intel.com>; Stillwell Jr, Paul M <paul.m.stillwell.jr@intel.com>
> Subject: [PATCH] net/ice: fix memzone leak when firmware is missing
>
> Caught by our QE.
> When the firmware is missing, memzones were not released.
>
> $ dpdk-testpmd -c 0x1f -a 0:0:0.0 -- -i
> ...
>
> testpmd> dump_memzone
> ...
> Zone 6: name:<RTE_METRICS>, len:0x15040, virt:0x1661b24c0, socket_id:0,
> flags:0
> physical segments used:
> addr: 0x140000000 iova: 0x140000000 len: 0x40000000 pagesz: 0x40000000
>
> testpmd> port attach 0000:5e:00.0
> Attaching a new port...
> EAL: Using IOMMU type 1 (Type 1)
> EAL: Probe PCI driver: net_ice (8086:159b) device: 0000:5e:00.0 (socket 0)
> ice_load_pkg(): failed to open file: /lib/firmware/intel/ice/ddp/ice.pkg
>
> ice_dev_init(): Failed to load the DDP package,Use safe-mode-support=1 to
> enter Safe Mode
> EAL: Releasing PCI mapped resource for 0000:5e:00.0
> EAL: Calling pci_unmap_resource for 0000:5e:00.0 at 0x2200000000
> EAL: Calling pci_unmap_resource for 0000:5e:00.0 at 0x2202000000
> EAL: Driver cannot attach the device (0000:5e:00.0)
> EAL: Failed to attach device on primary process
> testpmd: Failed to attach port 0000:5e:00.0
>
> testpmd> dump_memzone
> ...
> Zone 139: name:<ice_dma_17168374657430093156>, len:0x1000,
> virt:0x1660ed800, socket_id:0, flags:0 physical segments used:
> addr: 0x140000000 iova: 0x140000000 len: 0x40000000 pagesz: 0x40000000
>
> With 20 tries attaching a net/ice port, we would end up with:
>
> EAL: Probe PCI driver: net_ice (8086:159b) device: 0000:5e:00.0 (socket 0)
> EAL: memzone_reserve_aligned_thread_unsafe(): Number of requested memzone
> segments exceeds RTE_MAX_MEMZONE
> ice_dev_init(): Failed to initialize HW
>
> Fixes: a4c8c48fe3f4 ("net/ice: load OS default package")
> Cc: stable@dpdk.org
>
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> drivers/net/ice/ice_ethdev.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
> index 09e38590e5..f85c7cc8da 100644
> --- a/drivers/net/ice/ice_ethdev.c
> +++ b/drivers/net/ice/ice_ethdev.c
> @@ -2085,7 +2085,7 @@ ice_dev_init(struct rte_eth_dev *dev)
> if (ad->devargs.safe_mode_support == 0) {
> PMD_INIT_LOG(ERR, "Failed to load the DDP package,"
> "Use safe-mode-support=1 to enter Safe Mode");
> - return ret;
> + goto err_init_fw;
> }
>
> PMD_INIT_LOG(WARNING, "Failed to load the DDP package,"
> @@ -2175,6 +2175,8 @@ ice_dev_init(struct rte_eth_dev *dev)
> rte_free(dev->data->mac_addrs);
> dev->data->mac_addrs = NULL;
> err_init_mac:
> + ice_free_hw_tbls(hw);
> +err_init_fw:
> ice_sched_cleanup_all(hw);
> rte_free(hw->port_info);
> ice_shutdown_all_ctrlq(hw);
The function 'ice_deinit_hw' should be used. ;-)
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/net/ethernet/intel/ice/ice_main.c#n4508
diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
index 09e38590e5..c777474468 100644
--- a/drivers/net/ice/ice_ethdev.c
+++ b/drivers/net/ice/ice_ethdev.c
@@ -2085,7 +2085,7 @@ ice_dev_init(struct rte_eth_dev *dev)
if (ad->devargs.safe_mode_support == 0) {
PMD_INIT_LOG(ERR, "Failed to load the DDP package,"
"Use safe-mode-support=1 to enter Safe Mode");
- return ret;
+ goto err_init_fw;
}
PMD_INIT_LOG(WARNING, "Failed to load the DDP package,"
@@ -2175,10 +2175,9 @@ ice_dev_init(struct rte_eth_dev *dev)
rte_free(dev->data->mac_addrs);
dev->data->mac_addrs = NULL;
err_init_mac:
- ice_sched_cleanup_all(hw);
- rte_free(hw->port_info);
- ice_shutdown_all_ctrlq(hw);
rte_free(pf->proto_xtr);
+err_init_fw:
+ ice_deinit_hw(hw);
return ret;
}
====
Before attach:
testpmd> dump_memzone
Zone 0: name:<MP_mb_pool_1>, len:0x182100, virt:0x11ff954ec0, socket_id:1, flags:0
physical segments used:
addr: 0x11c0000000 iova: 0x11c0000000 len: 0x40000000 pagesz: 0x40000000
Zone 1: name:<RG_MP_mb_pool_1>, len:0x200180, virt:0x11ff754c00, socket_id:1, flags:0
physical segments used:
addr: 0x11c0000000 iova: 0x11c0000000 len: 0x40000000 pagesz: 0x40000000
Zone 2: name:<MP_mb_pool_1_0>, len:0x18333940, virt:0x11e7421240, socket_id:1, flags:0
physical segments used:
addr: 0x11c0000000 iova: 0x11c0000000 len: 0x40000000 pagesz: 0x40000000
Zone 3: name:<RTE_METRICS>, len:0x15040, virt:0x11e739f340, socket_id:1, flags:0
physical segments used:
addr: 0x11c0000000 iova: 0x11c0000000 len: 0x40000000 pagesz: 0x40000000
testpmd>
After attach:
testpmd> dump_memzone
Zone 0: name:<MP_mb_pool_1>, len:0x182100, virt:0x11ff954ec0, socket_id:1, flags:0
physical segments used:
addr: 0x11c0000000 iova: 0x11c0000000 len: 0x40000000 pagesz: 0x40000000
Zone 1: name:<RG_MP_mb_pool_1>, len:0x200180, virt:0x11ff754c00, socket_id:1, flags:0
physical segments used:
addr: 0x11c0000000 iova: 0x11c0000000 len: 0x40000000 pagesz: 0x40000000
Zone 2: name:<MP_mb_pool_1_0>, len:0x18333940, virt:0x11e7421240, socket_id:1, flags:0
physical segments used:
addr: 0x11c0000000 iova: 0x11c0000000 len: 0x40000000 pagesz: 0x40000000
Zone 3: name:<RTE_METRICS>, len:0x15040, virt:0x11e739f340, socket_id:1, flags:0
physical segments used:
addr: 0x11c0000000 iova: 0x11c0000000 len: 0x40000000 pagesz: 0x40000000
Zone 4: name:<rte_eth_dev_data>, len:0x37040, virt:0x11e73670c0, socket_id:1, flags:0
physical segments used:
addr: 0x11c0000000 iova: 0x11c0000000 len: 0x40000000 pagesz: 0x40000000
Only the 'rte_eth_dev_data' is added.
> --
> 2.23.0
On Tue, Jul 6, 2021 at 2:41 PM Wang, Haiyue <haiyue.wang@intel.com> wrote:
> > diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
> > index 09e38590e5..f85c7cc8da 100644
> > --- a/drivers/net/ice/ice_ethdev.c
> > +++ b/drivers/net/ice/ice_ethdev.c
> > @@ -2085,7 +2085,7 @@ ice_dev_init(struct rte_eth_dev *dev)
> > if (ad->devargs.safe_mode_support == 0) {
> > PMD_INIT_LOG(ERR, "Failed to load the DDP package,"
> > "Use safe-mode-support=1 to enter Safe Mode");
> > - return ret;
> > + goto err_init_fw;
> > }
> >
> > PMD_INIT_LOG(WARNING, "Failed to load the DDP package,"
> > @@ -2175,6 +2175,8 @@ ice_dev_init(struct rte_eth_dev *dev)
> > rte_free(dev->data->mac_addrs);
> > dev->data->mac_addrs = NULL;
> > err_init_mac:
> > + ice_free_hw_tbls(hw);
> > +err_init_fw:
> > ice_sched_cleanup_all(hw);
> > rte_free(hw->port_info);
> > ice_shutdown_all_ctrlq(hw);
>
> The function 'ice_deinit_hw' should be used. ;-)
>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/net/ethernet/intel/ice/ice_main.c#n4508
Well I was inspired by ice_dev_close(), so I guess this code too could
use some cleanup but I don't have the hw anymore to test this change.
I'll trust you and simply update ice_dev_init.
>
> diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
> index 09e38590e5..c777474468 100644
> --- a/drivers/net/ice/ice_ethdev.c
> +++ b/drivers/net/ice/ice_ethdev.c
> @@ -2085,7 +2085,7 @@ ice_dev_init(struct rte_eth_dev *dev)
> if (ad->devargs.safe_mode_support == 0) {
> PMD_INIT_LOG(ERR, "Failed to load the DDP package,"
> "Use safe-mode-support=1 to enter Safe Mode");
> - return ret;
> + goto err_init_fw;
> }
>
> PMD_INIT_LOG(WARNING, "Failed to load the DDP package,"
> @@ -2175,10 +2175,9 @@ ice_dev_init(struct rte_eth_dev *dev)
> rte_free(dev->data->mac_addrs);
> dev->data->mac_addrs = NULL;
> err_init_mac:
> - ice_sched_cleanup_all(hw);
> - rte_free(hw->port_info);
> - ice_shutdown_all_ctrlq(hw);
> rte_free(pf->proto_xtr);
> +err_init_fw:
In such a case a single label should be enough, right?
> + ice_deinit_hw(hw);
>
> return ret;
> }
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Tuesday, July 6, 2021 20:47
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev@dpdk.org; stable@dpdk.org; Yang, Qiming <qiming.yang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Stillwell Jr, Paul M <paul.m.stillwell.jr@intel.com>
> Subject: Re: [PATCH] net/ice: fix memzone leak when firmware is missing
>
> On Tue, Jul 6, 2021 at 2:41 PM Wang, Haiyue <haiyue.wang@intel.com> wrote:
> > > diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
> > > index 09e38590e5..f85c7cc8da 100644
> > > --- a/drivers/net/ice/ice_ethdev.c
> > > +++ b/drivers/net/ice/ice_ethdev.c
> > > @@ -2085,7 +2085,7 @@ ice_dev_init(struct rte_eth_dev *dev)
> > > if (ad->devargs.safe_mode_support == 0) {
> > > PMD_INIT_LOG(ERR, "Failed to load the DDP package,"
> > > "Use safe-mode-support=1 to enter Safe Mode");
> > > - return ret;
> > > + goto err_init_fw;
> > > }
> > >
> > > PMD_INIT_LOG(WARNING, "Failed to load the DDP package,"
> > > @@ -2175,6 +2175,8 @@ ice_dev_init(struct rte_eth_dev *dev)
> > > rte_free(dev->data->mac_addrs);
> > > dev->data->mac_addrs = NULL;
> > > err_init_mac:
> > > + ice_free_hw_tbls(hw);
> > > +err_init_fw:
> > > ice_sched_cleanup_all(hw);
> > > rte_free(hw->port_info);
> > > ice_shutdown_all_ctrlq(hw);
> >
> > The function 'ice_deinit_hw' should be used. ;-)
> >
> >
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/net/ethernet/intel/ice
> /ice_main.c#n4508
>
> Well I was inspired by ice_dev_close(), so I guess this code too could
> use some cleanup but I don't have the hw anymore to test this change.
> I'll trust you and simply update ice_dev_init.
>
>
> >
> > diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c
> > index 09e38590e5..c777474468 100644
> > --- a/drivers/net/ice/ice_ethdev.c
> > +++ b/drivers/net/ice/ice_ethdev.c
> > @@ -2085,7 +2085,7 @@ ice_dev_init(struct rte_eth_dev *dev)
> > if (ad->devargs.safe_mode_support == 0) {
> > PMD_INIT_LOG(ERR, "Failed to load the DDP package,"
> > "Use safe-mode-support=1 to enter Safe Mode");
> > - return ret;
> > + goto err_init_fw;
> > }
> >
> > PMD_INIT_LOG(WARNING, "Failed to load the DDP package,"
> > @@ -2175,10 +2175,9 @@ ice_dev_init(struct rte_eth_dev *dev)
> > rte_free(dev->data->mac_addrs);
> > dev->data->mac_addrs = NULL;
> > err_init_mac:
> > - ice_sched_cleanup_all(hw);
> > - rte_free(hw->port_info);
> > - ice_shutdown_all_ctrlq(hw);
> > rte_free(pf->proto_xtr);
> > +err_init_fw:
>
> In such a case a single label should be enough, right?
Yes, since pf->proto_xtr = NULL can be 'freed' safely.
>
>
> > + ice_deinit_hw(hw);
> >
> > return ret;
> > }
>
>
> --
> David Marchand
On Tue, Jul 6, 2021 at 2:53 PM Wang, Haiyue <haiyue.wang@intel.com> wrote:
> > > The function 'ice_deinit_hw' should be used. ;-)
> > >
> > >
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/net/ethernet/intel/ice
> > /ice_main.c#n4508
> >
> > Well I was inspired by ice_dev_close(), so I guess this code too could
> > use some cleanup but I don't have the hw anymore to test this change.
> > I'll trust you and simply update ice_dev_init.
Adding to this, there are other potential leaks if ice_flow_init() or
ice_reset_fxp_resource() fails.
Probably worth opening a bz for later processing.
WDYT?
> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Tuesday, July 6, 2021 21:28
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev@dpdk.org; stable@dpdk.org; Yang, Qiming <qiming.yang@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Stillwell Jr, Paul M <paul.m.stillwell.jr@intel.com>
> Subject: Re: [PATCH] net/ice: fix memzone leak when firmware is missing
>
> On Tue, Jul 6, 2021 at 2:53 PM Wang, Haiyue <haiyue.wang@intel.com> wrote:
> > > > The function 'ice_deinit_hw' should be used. ;-)
> > > >
> > > >
> > >
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/net/ethernet/intel/ice
> > > /ice_main.c#n4508
> > >
> > > Well I was inspired by ice_dev_close(), so I guess this code too could
> > > use some cleanup but I don't have the hw anymore to test this change.
> > > I'll trust you and simply update ice_dev_init.
>
> Adding to this, there are other potential leaks if ice_flow_init() or
> ice_reset_fxp_resource() fails.
> Probably worth opening a bz for later processing.
> WDYT?
Yeah, make sense. ;-)
>
>
> --
> David Marchand
On Tue, Jul 6, 2021 at 2:53 PM Wang, Haiyue <haiyue.wang@intel.com> wrote:
> > In such a case a single label should be enough, right?
>
> Yes, since pf->proto_xtr = NULL can be 'freed' safely.
Ah ok, I thought proto_xtr was initialised earlier and I had missed
the ice_pf_sw_init() call.
I'll go with your suggestion.
Thanks.
@@ -2085,7 +2085,7 @@ ice_dev_init(struct rte_eth_dev *dev)
if (ad->devargs.safe_mode_support == 0) {
PMD_INIT_LOG(ERR, "Failed to load the DDP package,"
"Use safe-mode-support=1 to enter Safe Mode");
- return ret;
+ goto err_init_fw;
}
PMD_INIT_LOG(WARNING, "Failed to load the DDP package,"
@@ -2175,6 +2175,8 @@ ice_dev_init(struct rte_eth_dev *dev)
rte_free(dev->data->mac_addrs);
dev->data->mac_addrs = NULL;
err_init_mac:
+ ice_free_hw_tbls(hw);
+err_init_fw:
ice_sched_cleanup_all(hw);
rte_free(hw->port_info);
ice_shutdown_all_ctrlq(hw);