eal/alarm_cancel: Fix thread starvation
Checks
Commit Message
Issue:
Two threads:
- A, executing rte_eal_alarm_cancel,
- B, executing eal_alarm_callback.
Such case can cause starvation of thread B. Please see that there is a
small time window between lock and unlock in thread A, so thread B must
be switched to within a very small time window, so that it can obtain
the lock.
Solution to this problem is use sched_yield(), which puts current thread
(A) at the end of thread execution priority queue and allows thread B to
execute.
The issue can be observed e.g. on hot-pluggable device detach path.
On such path, rte_alarm can used to check if DPDK has completed
the detachment. Waiting for completion, rte_eal_alarm_cancel
is called, while another thread periodically calls eal_alarm_callback
causing the issue to occur.
Signed-off-by: Wojciech Panfil <wojciech.panfil@intel.com>
---
lib/eal/freebsd/eal_alarm.c | 6 ++++++
lib/eal/linux/eal_alarm.c | 6 ++++++
lib/eal/windows/eal_alarm.c | 5 +++++
3 files changed, 17 insertions(+)
Comments
On Wed, 18 Sep 2024 13:39:06 +0200
Wojciech Panfil <wojciech.panfil@intel.com> wrote:
> Issue:
> Two threads:
>
> - A, executing rte_eal_alarm_cancel,
> - B, executing eal_alarm_callback.
>
> Such case can cause starvation of thread B. Please see that there is a
> small time window between lock and unlock in thread A, so thread B must
> be switched to within a very small time window, so that it can obtain
> the lock.
>
> Solution to this problem is use sched_yield(), which puts current thread
> (A) at the end of thread execution priority queue and allows thread B to
> execute.
>
> The issue can be observed e.g. on hot-pluggable device detach path.
> On such path, rte_alarm can used to check if DPDK has completed
> the detachment. Waiting for completion, rte_eal_alarm_cancel
> is called, while another thread periodically calls eal_alarm_callback
> causing the issue to occur.
>
> Signed-off-by: Wojciech Panfil <wojciech.panfil@intel.com>
Make sense. Alarm is non-EAL thread, and so is hotplug.
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Does the timer_stop code have similar issues?
Probably only if users do unexpected things like
map multiple logical lcores to same CPU.
@@ -318,7 +318,13 @@ rte_eal_alarm_cancel(rte_eal_alarm_callback cb_fn, void *cb_arg)
}
ap_prev = ap;
}
+
rte_spinlock_unlock(&alarm_list_lk);
+
+ /* Yield control to a second thread executing eal_alarm_callback to avoid
+ * its starvation, as it is waiting for the lock we have just released.
+ */
+ sched_yield();
} while (executing != 0);
if (count == 0 && err == 0)
@@ -248,7 +248,13 @@ rte_eal_alarm_cancel(rte_eal_alarm_callback cb_fn, void *cb_arg)
}
ap_prev = ap;
}
+
rte_spinlock_unlock(&alarm_list_lk);
+
+ /* Yield control to a second thread executing eal_alarm_callback to avoid
+ * its starvation, as it is waiting for the lock we have just released.
+ */
+ sched_yield();
} while (executing != 0);
if (count == 0 && err == 0)
@@ -211,6 +211,11 @@ rte_eal_alarm_cancel(rte_eal_alarm_callback cb_fn, void *cb_arg)
}
rte_spinlock_unlock(&alarm_lock);
+
+ /* Yield control to a second thread executing eal_alarm_callback to avoid
+ * its starvation, as it is waiting for the lock we have just released.
+ */
+ SwitchToThread();
} while (executing);
rte_eal_trace_alarm_cancel(cb_fn, cb_arg, removed);