[v5] eal: add manual probing option
diff mbox series

Message ID c9c190e95ab17b7786c8b597546bceb670e137ce.1572018254.git.gaetan.rivet@6wind.com
State Superseded, archived
Delegated to: David Marchand
Headers show
Series
  • [v5] eal: add manual probing option
Related show

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/iol-intel-Performance success Performance Testing PASS
ci/iol-compilation success Compile Testing PASS
ci/iol-mellanox-Performance success Performance Testing PASS
ci/travis-robot success Travis build: passed
ci/Intel-compilation fail apply issues

Commit Message

Gaëtan Rivet Oct. 25, 2019, 3:46 p.m. UTC
Add a new EAL option enabling manual probing in the EAL.
This command line option will configure the EAL so that buses
will not trigger their probe step on their own.

Applications are then expected to hotplug devices as they see fit.

Devices declared on the command line by the user (using -w and --vdev),
will be probed using the hotplug API, in the order they are declared.

This has the effect of offering a way for users to control probe order
of their devices, for drivers requiring it.

Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
---

I haven't heard many opinions on the matter, please shout if you see an issue
with this approach.

@Slava: I have tested rather quickly that it does not break anything,
        and that it works as intended for basic cases.
        Can you test it further for your use-case and tell me if it works fine?

Beyond the obvious difference between both probe mode, something to keep in mind:
while using -w on invalid devices would not block (PCI) bus probing, it will stop manual
probing in its track. All devices need to exist and be valid device IDs.

v2: fixed a few typos, map file (and used Travis to validate).

    Slava, are you able to test this patch?

v3: properly fixed the map file (inherited 19.08 instead of 19.05).

    Added a function to set the probe manual from the application,
    without having the user do it from the command line.

    Stopped spamming Slava about it, Vamsi was actually the one interested in it!

Standing issue worth chiming in:

  Currently manual-probe will cut off probing from all buses.
  It could be interesting to be able to only cut buses supporting hotplug,
  given that they are the one able to probe devices afterward.

  No real use-case for this right now, so leaving as-is. Might be worth
  considering in the future.

v4: Rebased on master,
    Moved implementation in common EAL,
    Used public API within the EAL to set the option,
    Made the API experimental

v5: added note in the Getting Started Guide.

 doc/guides/linux_gsg/eal_args.include.rst  | 13 +++++++
 doc/guides/rel_notes/release_19_11.rst     |  9 +++++
 lib/librte_eal/common/eal_common_bus.c     |  6 ++++
 lib/librte_eal/common/eal_common_dev.c     | 54 ++++++++++++++++++++++++++++++
 lib/librte_eal/common/eal_common_options.c |  8 +++++
 lib/librte_eal/common/eal_internal_cfg.h   |  1 +
 lib/librte_eal/common/eal_options.h        |  2 ++
 lib/librte_eal/common/eal_private.h        |  9 +++++
 lib/librte_eal/common/include/rte_eal.h    | 36 ++++++++++++++++++++
 lib/librte_eal/rte_eal_version.map         |  4 +++
 10 files changed, 142 insertions(+)

Comments

Jerin Jacob Oct. 25, 2019, 3:51 p.m. UTC | #1
On Fri, Oct 25, 2019 at 9:16 PM Gaetan Rivet <gaetan.rivet@6wind.com> wrote:
>
> Add a new EAL option enabling manual probing in the EAL.
> This command line option will configure the EAL so that buses
> will not trigger their probe step on their own.
>
> Applications are then expected to hotplug devices as they see fit.
>
> Devices declared on the command line by the user (using -w and --vdev),
> will be probed using the hotplug API, in the order they are declared.
>
> This has the effect of offering a way for users to control probe order
> of their devices, for drivers requiring it.
>
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>

Reviewed-by: Jerin Jacob <jerinj@marvell.com>
Pavan Nikhilesh Bhagavatula Jan. 22, 2020, 4:51 p.m. UTC | #2
Ping
@grive@u256.net

Since, you weren't reachable on regular email I rebased the patch on ToT and sent a v6
http://patches.dpdk.org/patch/64235/

>-----Original Message-----
>From: dev <dev-bounces@dpdk.org> On Behalf Of Gaetan Rivet
>Sent: Friday, October 25, 2019 9:17 PM
>To: dev@dpdk.org
>Cc: Gaetan Rivet <gaetan.rivet@6wind.com>; Vamsi Krishna Attunuru
><vattunuru@marvell.com>; David Marchand
><david.marchand@redhat.com>; Jerin Jacob Kollanukkaran
><jerinj@marvell.com>
>Subject: [dpdk-dev] [PATCH v5] eal: add manual probing option
>
>Add a new EAL option enabling manual probing in the EAL.
>This command line option will configure the EAL so that buses
>will not trigger their probe step on their own.
>
>Applications are then expected to hotplug devices as they see fit.
>
>Devices declared on the command line by the user (using -w and --
>vdev),
>will be probed using the hotplug API, in the order they are declared.
>
>This has the effect of offering a way for users to control probe order
>of their devices, for drivers requiring it.
>
>Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
>---
>
>I haven't heard many opinions on the matter, please shout if you see an
>issue
>with this approach.
>
>@Slava: I have tested rather quickly that it does not break anything,
>        and that it works as intended for basic cases.
>        Can you test it further for your use-case and tell me if it works fine?
>
>Beyond the obvious difference between both probe mode, something
>to keep in mind:
>while using -w on invalid devices would not block (PCI) bus probing, it
>will stop manual
>probing in its track. All devices need to exist and be valid device IDs.
>
>v2: fixed a few typos, map file (and used Travis to validate).
>
>    Slava, are you able to test this patch?
>
>v3: properly fixed the map file (inherited 19.08 instead of 19.05).
>
>    Added a function to set the probe manual from the application,
>    without having the user do it from the command line.
>
>    Stopped spamming Slava about it, Vamsi was actually the one
>interested in it!
>
>Standing issue worth chiming in:
>
>  Currently manual-probe will cut off probing from all buses.
>  It could be interesting to be able to only cut buses supporting hotplug,
>  given that they are the one able to probe devices afterward.
>
>  No real use-case for this right now, so leaving as-is. Might be worth
>  considering in the future.
>
>v4: Rebased on master,
>    Moved implementation in common EAL,
>    Used public API within the EAL to set the option,
>    Made the API experimental
>
>v5: added note in the Getting Started Guide.
>
> doc/guides/linux_gsg/eal_args.include.rst  | 13 +++++++
> doc/guides/rel_notes/release_19_11.rst     |  9 +++++
> lib/librte_eal/common/eal_common_bus.c     |  6 ++++
> lib/librte_eal/common/eal_common_dev.c     | 54
>++++++++++++++++++++++++++++++
> lib/librte_eal/common/eal_common_options.c |  8 +++++
> lib/librte_eal/common/eal_internal_cfg.h   |  1 +
> lib/librte_eal/common/eal_options.h        |  2 ++
> lib/librte_eal/common/eal_private.h        |  9 +++++
> lib/librte_eal/common/include/rte_eal.h    | 36
>++++++++++++++++++++
> lib/librte_eal/rte_eal_version.map         |  4 +++
> 10 files changed, 142 insertions(+)
>
>diff --git a/doc/guides/linux_gsg/eal_args.include.rst
>b/doc/guides/linux_gsg/eal_args.include.rst
>index ed8b0e35b..d0717d4a0 100644
>--- a/doc/guides/linux_gsg/eal_args.include.rst
>+++ b/doc/guides/linux_gsg/eal_args.include.rst
>@@ -69,6 +69,19 @@ Device-related options
>
>        --vdev 'net_pcap0,rx_pcap=input.pcap,tx_pcap=output.pcap'
>
>+*   ``--manual-probe``
>+
>+    Switch the ``EAL`` probe mode to manual. The main bus probe step
>+    is disabled and applications are expected to manually probe
>+    devices using ``rte_dev_probe()``.
>+
>+    Devices declared on the command-line using ``-w`` and ``-vdev``
>+    are interpreted as hotplug commands. They are thus probed in the
>+    order they are declared.
>+
>+    This makes this option useful to enforce a specific device probe
>+    order, instead of relying on each bus scan implementation details.
>+
> *   ``-d <path to shared object or directory>``
>
>     Load external drivers. An argument can be a single shared object file,
>or a
>diff --git a/doc/guides/rel_notes/release_19_11.rst
>b/doc/guides/rel_notes/release_19_11.rst
>index e77d226b5..7251a17b2 100644
>--- a/doc/guides/rel_notes/release_19_11.rst
>+++ b/doc/guides/rel_notes/release_19_11.rst
>@@ -56,6 +56,15 @@ New Features
>      Also, make sure to start the actual text at the margin.
>
>======================================================
>===
>
>+* **EAL will now allow probing devices manually.**
>+
>+  Previously, a user could not force an order when probing declared
>devices.
>+  This could cause issues for drivers depending on another device being
>present.
>+  A new option ``--manual-probe`` is now available to do just that.
>+  This new option relies on the device bus supporting hotplug. It can
>+  also be used to disable automatic probing from the ``PCI`` bus without
>+  having to disable the whole bus.
>+
> * **FreeBSD now supports `--base-virtaddr` EAL option.**
>
>   FreeBSD version now also supports setting base virtual address for
>mapping
>diff --git a/lib/librte_eal/common/eal_common_bus.c
>b/lib/librte_eal/common/eal_common_bus.c
>index baa5b532a..145a96812 100644
>--- a/lib/librte_eal/common/eal_common_bus.c
>+++ b/lib/librte_eal/common/eal_common_bus.c
>@@ -6,6 +6,7 @@
> #include <string.h>
> #include <sys/queue.h>
>
>+#include <rte_eal.h>
> #include <rte_bus.h>
> #include <rte_debug.h>
> #include <rte_string_fns.h>
>@@ -63,6 +64,11 @@ rte_bus_probe(void)
> 	int ret;
> 	struct rte_bus *bus, *vbus = NULL;
>
>+	if (rte_eal_manual_probe()) {
>+		RTE_LOG(DEBUG, EAL, "Manual probing enabled.\n");
>+		return rte_dev_probe_devargs_list();
>+	}
>+
> 	TAILQ_FOREACH(bus, &rte_bus_list, next) {
> 		if (!strcmp(bus->name, "vdev")) {
> 			vbus = bus;
>diff --git a/lib/librte_eal/common/eal_common_dev.c
>b/lib/librte_eal/common/eal_common_dev.c
>index 9e4f09d83..368afa273 100644
>--- a/lib/librte_eal/common/eal_common_dev.c
>+++ b/lib/librte_eal/common/eal_common_dev.c
>@@ -21,6 +21,7 @@
> #include <rte_malloc.h>
> #include <rte_string_fns.h>
>
>+#include "eal_internal_cfg.h"
> #include "eal_private.h"
> #include "hotplug_mp.h"
>
>@@ -83,6 +84,59 @@ rte_dev_is_probed(const struct rte_device *dev)
> 	return dev->driver != NULL;
> }
>
>+int
>+rte_eal_manual_probe(void)
>+{
>+	return internal_config.manual_probe;
>+}
>+
>+void
>+rte_eal_manual_probe_set(int enabled)
>+{
>+	internal_config.manual_probe = !!enabled;
>+}
>+
>+int
>+rte_dev_probe_devargs_list(void)
>+{
>+	struct rte_device *dev;
>+	struct rte_devargs *da;
>+	int ret;
>+
>+	RTE_EAL_DEVARGS_FOREACH(NULL, da) {
>+		dev = da->bus->find_device(NULL, cmp_dev_name,
>da->name);
>+		if (dev == NULL) {
>+			RTE_LOG(ERR, EAL, "Unable to find device %s
>on bus %s\n",
>+				da->name, da->bus->name);
>+			continue;
>+		}
>+
>+		if (rte_dev_is_probed(dev))
>+			continue;
>+
>+		if (dev->bus->plug == NULL) {
>+			RTE_LOG(ERR, EAL, "Manual probing (hotplug)
>not supported by bus %s, "
>+					  "required by device %s\n",
>+				dev->bus->name, dev->name);
>+			continue;
>+		}
>+
>+		ret = dev->bus->plug(dev);
>+		/* Ignore positive return values, they are possibly
>+		 * triggered by blacklisted devices on the PCI bus.
>Probing
>+		 * should then continue.
>+		 */
>+		if (ret < 0) {
>+			RTE_LOG(ERR, EAL, "Driver cannot attach device
>%s\n",
>+				dev->name);
>+			/* Fail on first real probe error. */
>+			return ret;
>+		}
>+	}
>+
>+	return 0;
>+}
>+
> /* helper function to build devargs, caller should free the memory */
> static int
> build_devargs(const char *busname, const char *devname,
>diff --git a/lib/librte_eal/common/eal_common_options.c
>b/lib/librte_eal/common/eal_common_options.c
>index a7f9c5f9b..de73970fe 100644
>--- a/lib/librte_eal/common/eal_common_options.c
>+++ b/lib/librte_eal/common/eal_common_options.c
>@@ -82,6 +82,7 @@ eal_long_options[] = {
> 	{OPT_LEGACY_MEM,        0, NULL, OPT_LEGACY_MEM_NUM
>},
> 	{OPT_SINGLE_FILE_SEGMENTS, 0, NULL,
>OPT_SINGLE_FILE_SEGMENTS_NUM},
> 	{OPT_MATCH_ALLOCATIONS, 0, NULL,
>OPT_MATCH_ALLOCATIONS_NUM},
>+	{OPT_MANUAL_PROBE,      0, NULL,
>OPT_MANUAL_PROBE_NUM     },
> 	{0,                     0, NULL, 0                        }
> };
>
>@@ -1446,6 +1447,9 @@ eal_parse_common_option(int opt, const
>char *optarg,
> 			return -1;
> 		}
> 		break;
>+	case OPT_MANUAL_PROBE_NUM:
>+		rte_eal_manual_probe_set(1);
>+		break;
>
> 	/* don't know what to do, leave this to caller */
> 	default:
>@@ -1672,6 +1676,10 @@ eal_common_usage(void)
> 	       "  --"OPT_VDEV"              Add a virtual device.\n"
> 	       "                      The argument format is
><driver><id>[,key=val,...]\n"
> 	       "                      (ex: --vdev=net_pcap0,iface=eth2).\n"
>+	       "  --"OPT_MANUAL_PROBE"      Enable manual probing.\n"
>+	       "                      Disable probe step for all buses.\n"
>+	       "                      Devices will need to be probed using the hotplug
>API.\n"
>+	       "                      PCI and vdev declarations will be treated in
>order as hotplug commands.\n"
> 	       "  --"OPT_IOVA_MODE"   Set IOVA mode. 'pa' for
>IOVA_PA\n"
> 	       "                      'va' for IOVA_VA\n"
> 	       "  -d LIB.so|DIR       Add a driver or driver directory\n"
>diff --git a/lib/librte_eal/common/eal_internal_cfg.h
>b/lib/librte_eal/common/eal_internal_cfg.h
>index a42f34923..0006f903f 100644
>--- a/lib/librte_eal/common/eal_internal_cfg.h
>+++ b/lib/librte_eal/common/eal_internal_cfg.h
>@@ -44,6 +44,7 @@ struct internal_config {
> 	unsigned hugepage_unlink;         /**< true to unlink backing files
>*/
> 	volatile unsigned no_pci;         /**< true to disable PCI */
> 	volatile unsigned no_hpet;        /**< true to disable HPET */
>+	volatile unsigned manual_probe;   /**< true to enable manual
>device probing. */
> 	volatile unsigned vmware_tsc_map; /**< true to use VMware
>TSC mapping
>
>	* instead of native TSC */
> 	volatile unsigned no_shconf;      /**< true if there is no shared
>config */
>diff --git a/lib/librte_eal/common/eal_options.h
>b/lib/librte_eal/common/eal_options.h
>index 9855429e5..588fa32a6 100644
>--- a/lib/librte_eal/common/eal_options.h
>+++ b/lib/librte_eal/common/eal_options.h
>@@ -69,6 +69,8 @@ enum {
> 	OPT_IOVA_MODE_NUM,
> #define OPT_MATCH_ALLOCATIONS  "match-allocations"
> 	OPT_MATCH_ALLOCATIONS_NUM,
>+#define OPT_MANUAL_PROBE "manual-probe"
>+	OPT_MANUAL_PROBE_NUM,
> 	OPT_LONG_MAX_NUM
> };
>
>diff --git a/lib/librte_eal/common/eal_private.h
>b/lib/librte_eal/common/eal_private.h
>index 798ede553..fd7ac8e37 100644
>--- a/lib/librte_eal/common/eal_private.h
>+++ b/lib/librte_eal/common/eal_private.h
>@@ -381,4 +381,13 @@ rte_option_init(void);
> void
> rte_option_usage(void);
>
>+/**
>+ * Go through the devargs list and probe everything in order.
>+ *
>+ * @return
>+ *   0 on success, negative on error.
>+ */
>+int
>+rte_dev_probe_devargs_list(void);
>+
> #endif /* _EAL_PRIVATE_H_ */
>diff --git a/lib/librte_eal/common/include/rte_eal.h
>b/lib/librte_eal/common/include/rte_eal.h
>index b7cf91214..71d9b4ca6 100644
>--- a/lib/librte_eal/common/include/rte_eal.h
>+++ b/lib/librte_eal/common/include/rte_eal.h
>@@ -465,6 +465,42 @@ int rte_eal_has_hugepages(void);
> int rte_eal_has_pci(void);
>
> /**
>+ * Whether EAL probe is manual.
>+ * Enabled by the --manual-probe option or by
>+ * using rte_eal_manual_probe_set().
>+ *
>+ * When manual probing is enabled, batched bus probe of
>+ * their devices is disabled. All devices need to be probed
>+ * using the proper rte_dev API.
>+ *
>+ * In this mode, devices declared on the command line will
>+ * be probed using the bus hotplug API. It is used to enforce
>+ * a specific probe order.
>+ *
>+ * @return
>+ *   Nonzero if manual device probing is enabled.
>+ *
>+ * @see rte_eal_manual_probe_set
>+ */
>+__rte_experimental
>+int rte_eal_manual_probe(void);
>+
>+/**
>+ * Configure EAL probe mode -- manual or automatic.
>+ *
>+ * Enable or disable manual probe mode in EAL.
>+ * This function can be called at any time, but must be used
>+ * before calling rte_eal_init() to have any effect.
>+ *
>+ * @param enabled
>+ *   zero to disable manual probe, non-zero to enable it.
>+ *
>+ * @see rte_eal_manual_probe
>+ */
>+__rte_experimental
>+void rte_eal_manual_probe_set(int enabled);
>+
>+/**
>  * Whether the EAL was asked to create UIO device.
>  *
>  * @return
>diff --git a/lib/librte_eal/rte_eal_version.map
>b/lib/librte_eal/rte_eal_version.map
>index 7cbf82d37..7d7ff6531 100644
>--- a/lib/librte_eal/rte_eal_version.map
>+++ b/lib/librte_eal/rte_eal_version.map
>@@ -419,4 +419,8 @@ EXPERIMENTAL {
> 	rte_mcfg_timer_lock;
> 	rte_mcfg_timer_unlock;
> 	rte_rand_max;
>+
>+	# added in 19.11
>+	rte_eal_manual_probe;
>+	rte_eal_manual_probe_set;
> };
>--
>2.11.0
Gaetan Rivet Jan. 23, 2020, 9:20 a.m. UTC | #3
On 22/01/2020 17:51, Pavan Nikhilesh Bhagavatula wrote:
> Ping
> @grive@u256.net
> 
> Since, you weren't reachable on regular email I rebased the patch on ToT and sent a v6
> http://patches.dpdk.org/patch/64235/
> 

Hello Pavan, indeed sorry about this.
I have not heard remarks about the patchset anyway in the meantime, not 
sure whether there is anything to change.

Thanks for the rebase, I will follow-up if there are remarks blocking 
the integration.

>> -----Original Message-----
>> From: dev <dev-bounces@dpdk.org> On Behalf Of Gaetan Rivet
>> Sent: Friday, October 25, 2019 9:17 PM
>> To: dev@dpdk.org
>> Cc: Gaetan Rivet <gaetan.rivet@6wind.com>; Vamsi Krishna Attunuru
>> <vattunuru@marvell.com>; David Marchand
>> <david.marchand@redhat.com>; Jerin Jacob Kollanukkaran
>> <jerinj@marvell.com>
>> Subject: [dpdk-dev] [PATCH v5] eal: add manual probing option
>>
>> Add a new EAL option enabling manual probing in the EAL.
>> This command line option will configure the EAL so that buses
>> will not trigger their probe step on their own.
>>
>> Applications are then expected to hotplug devices as they see fit.
>>
>> Devices declared on the command line by the user (using -w and --
>> vdev),
>> will be probed using the hotplug API, in the order they are declared.
>>
>> This has the effect of offering a way for users to control probe order
>> of their devices, for drivers requiring it.
>>
>> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
>> ---
>>
>> I haven't heard many opinions on the matter, please shout if you see an
>> issue
>> with this approach.
>>
>> @Slava: I have tested rather quickly that it does not break anything,
>>         and that it works as intended for basic cases.
>>         Can you test it further for your use-case and tell me if it works fine?
>>
>> Beyond the obvious difference between both probe mode, something
>> to keep in mind:
>> while using -w on invalid devices would not block (PCI) bus probing, it
>> will stop manual
>> probing in its track. All devices need to exist and be valid device IDs.
>>
>> v2: fixed a few typos, map file (and used Travis to validate).
>>
>>     Slava, are you able to test this patch?
>>
>> v3: properly fixed the map file (inherited 19.08 instead of 19.05).
>>
>>     Added a function to set the probe manual from the application,
>>     without having the user do it from the command line.
>>
>>     Stopped spamming Slava about it, Vamsi was actually the one
>> interested in it!
>>
>> Standing issue worth chiming in:
>>
>>   Currently manual-probe will cut off probing from all buses.
>>   It could be interesting to be able to only cut buses supporting hotplug,
>>   given that they are the one able to probe devices afterward.
>>
>>   No real use-case for this right now, so leaving as-is. Might be worth
>>   considering in the future.
>>
>> v4: Rebased on master,
>>     Moved implementation in common EAL,
>>     Used public API within the EAL to set the option,
>>     Made the API experimental
>>
>> v5: added note in the Getting Started Guide.
>>
>> doc/guides/linux_gsg/eal_args.include.rst  | 13 +++++++
>> doc/guides/rel_notes/release_19_11.rst     |  9 +++++
>> lib/librte_eal/common/eal_common_bus.c     |  6 ++++
>> lib/librte_eal/common/eal_common_dev.c     | 54
>> ++++++++++++++++++++++++++++++
>> lib/librte_eal/common/eal_common_options.c |  8 +++++
>> lib/librte_eal/common/eal_internal_cfg.h   |  1 +
>> lib/librte_eal/common/eal_options.h        |  2 ++
>> lib/librte_eal/common/eal_private.h        |  9 +++++
>> lib/librte_eal/common/include/rte_eal.h    | 36
>> ++++++++++++++++++++
>> lib/librte_eal/rte_eal_version.map         |  4 +++
>> 10 files changed, 142 insertions(+)
>>
>> diff --git a/doc/guides/linux_gsg/eal_args.include.rst
>> b/doc/guides/linux_gsg/eal_args.include.rst
>> index ed8b0e35b..d0717d4a0 100644
>> --- a/doc/guides/linux_gsg/eal_args.include.rst
>> +++ b/doc/guides/linux_gsg/eal_args.include.rst
>> @@ -69,6 +69,19 @@ Device-related options
>>
>>         --vdev 'net_pcap0,rx_pcap=input.pcap,tx_pcap=output.pcap'
>>
>> +*   ``--manual-probe``
>> +
>> +    Switch the ``EAL`` probe mode to manual. The main bus probe step
>> +    is disabled and applications are expected to manually probe
>> +    devices using ``rte_dev_probe()``.
>> +
>> +    Devices declared on the command-line using ``-w`` and ``-vdev``
>> +    are interpreted as hotplug commands. They are thus probed in the
>> +    order they are declared.
>> +
>> +    This makes this option useful to enforce a specific device probe
>> +    order, instead of relying on each bus scan implementation details.
>> +
>> *   ``-d <path to shared object or directory>``
>>
>>      Load external drivers. An argument can be a single shared object file,
>> or a
>> diff --git a/doc/guides/rel_notes/release_19_11.rst
>> b/doc/guides/rel_notes/release_19_11.rst
>> index e77d226b5..7251a17b2 100644
>> --- a/doc/guides/rel_notes/release_19_11.rst
>> +++ b/doc/guides/rel_notes/release_19_11.rst
>> @@ -56,6 +56,15 @@ New Features
>>       Also, make sure to start the actual text at the margin.
>>
>> ======================================================
>> ===
>>
>> +* **EAL will now allow probing devices manually.**
>> +
>> +  Previously, a user could not force an order when probing declared
>> devices.
>> +  This could cause issues for drivers depending on another device being
>> present.
>> +  A new option ``--manual-probe`` is now available to do just that.
>> +  This new option relies on the device bus supporting hotplug. It can
>> +  also be used to disable automatic probing from the ``PCI`` bus without
>> +  having to disable the whole bus.
>> +
>> * **FreeBSD now supports `--base-virtaddr` EAL option.**
>>
>>    FreeBSD version now also supports setting base virtual address for
>> mapping
>> diff --git a/lib/librte_eal/common/eal_common_bus.c
>> b/lib/librte_eal/common/eal_common_bus.c
>> index baa5b532a..145a96812 100644
>> --- a/lib/librte_eal/common/eal_common_bus.c
>> +++ b/lib/librte_eal/common/eal_common_bus.c
>> @@ -6,6 +6,7 @@
>> #include <string.h>
>> #include <sys/queue.h>
>>
>> +#include <rte_eal.h>
>> #include <rte_bus.h>
>> #include <rte_debug.h>
>> #include <rte_string_fns.h>
>> @@ -63,6 +64,11 @@ rte_bus_probe(void)
>> 	int ret;
>> 	struct rte_bus *bus, *vbus = NULL;
>>
>> +	if (rte_eal_manual_probe()) {
>> +		RTE_LOG(DEBUG, EAL, "Manual probing enabled.\n");
>> +		return rte_dev_probe_devargs_list();
>> +	}
>> +
>> 	TAILQ_FOREACH(bus, &rte_bus_list, next) {
>> 		if (!strcmp(bus->name, "vdev")) {
>> 			vbus = bus;
>> diff --git a/lib/librte_eal/common/eal_common_dev.c
>> b/lib/librte_eal/common/eal_common_dev.c
>> index 9e4f09d83..368afa273 100644
>> --- a/lib/librte_eal/common/eal_common_dev.c
>> +++ b/lib/librte_eal/common/eal_common_dev.c
>> @@ -21,6 +21,7 @@
>> #include <rte_malloc.h>
>> #include <rte_string_fns.h>
>>
>> +#include "eal_internal_cfg.h"
>> #include "eal_private.h"
>> #include "hotplug_mp.h"
>>
>> @@ -83,6 +84,59 @@ rte_dev_is_probed(const struct rte_device *dev)
>> 	return dev->driver != NULL;
>> }
>>
>> +int
>> +rte_eal_manual_probe(void)
>> +{
>> +	return internal_config.manual_probe;
>> +}
>> +
>> +void
>> +rte_eal_manual_probe_set(int enabled)
>> +{
>> +	internal_config.manual_probe = !!enabled;
>> +}
>> +
>> +int
>> +rte_dev_probe_devargs_list(void)
>> +{
>> +	struct rte_device *dev;
>> +	struct rte_devargs *da;
>> +	int ret;
>> +
>> +	RTE_EAL_DEVARGS_FOREACH(NULL, da) {
>> +		dev = da->bus->find_device(NULL, cmp_dev_name,
>> da->name);
>> +		if (dev == NULL) {
>> +			RTE_LOG(ERR, EAL, "Unable to find device %s
>> on bus %s\n",
>> +				da->name, da->bus->name);
>> +			continue;
>> +		}
>> +
>> +		if (rte_dev_is_probed(dev))
>> +			continue;
>> +
>> +		if (dev->bus->plug == NULL) {
>> +			RTE_LOG(ERR, EAL, "Manual probing (hotplug)
>> not supported by bus %s, "
>> +					  "required by device %s\n",
>> +				dev->bus->name, dev->name);
>> +			continue;
>> +		}
>> +
>> +		ret = dev->bus->plug(dev);
>> +		/* Ignore positive return values, they are possibly
>> +		 * triggered by blacklisted devices on the PCI bus.
>> Probing
>> +		 * should then continue.
>> +		 */
>> +		if (ret < 0) {
>> +			RTE_LOG(ERR, EAL, "Driver cannot attach device
>> %s\n",
>> +				dev->name);
>> +			/* Fail on first real probe error. */
>> +			return ret;
>> +		}
>> +	}
>> +
>> +	return 0;
>> +}
>> +
>> /* helper function to build devargs, caller should free the memory */
>> static int
>> build_devargs(const char *busname, const char *devname,
>> diff --git a/lib/librte_eal/common/eal_common_options.c
>> b/lib/librte_eal/common/eal_common_options.c
>> index a7f9c5f9b..de73970fe 100644
>> --- a/lib/librte_eal/common/eal_common_options.c
>> +++ b/lib/librte_eal/common/eal_common_options.c
>> @@ -82,6 +82,7 @@ eal_long_options[] = {
>> 	{OPT_LEGACY_MEM,        0, NULL, OPT_LEGACY_MEM_NUM
>> },
>> 	{OPT_SINGLE_FILE_SEGMENTS, 0, NULL,
>> OPT_SINGLE_FILE_SEGMENTS_NUM},
>> 	{OPT_MATCH_ALLOCATIONS, 0, NULL,
>> OPT_MATCH_ALLOCATIONS_NUM},
>> +	{OPT_MANUAL_PROBE,      0, NULL,
>> OPT_MANUAL_PROBE_NUM     },
>> 	{0,                     0, NULL, 0                        }
>> };
>>
>> @@ -1446,6 +1447,9 @@ eal_parse_common_option(int opt, const
>> char *optarg,
>> 			return -1;
>> 		}
>> 		break;
>> +	case OPT_MANUAL_PROBE_NUM:
>> +		rte_eal_manual_probe_set(1);
>> +		break;
>>
>> 	/* don't know what to do, leave this to caller */
>> 	default:
>> @@ -1672,6 +1676,10 @@ eal_common_usage(void)
>> 	       "  --"OPT_VDEV"              Add a virtual device.\n"
>> 	       "                      The argument format is
>> <driver><id>[,key=val,...]\n"
>> 	       "                      (ex: --vdev=net_pcap0,iface=eth2).\n"
>> +	       "  --"OPT_MANUAL_PROBE"      Enable manual probing.\n"
>> +	       "                      Disable probe step for all buses.\n"
>> +	       "                      Devices will need to be probed using the hotplug
>> API.\n"
>> +	       "                      PCI and vdev declarations will be treated in
>> order as hotplug commands.\n"
>> 	       "  --"OPT_IOVA_MODE"   Set IOVA mode. 'pa' for
>> IOVA_PA\n"
>> 	       "                      'va' for IOVA_VA\n"
>> 	       "  -d LIB.so|DIR       Add a driver or driver directory\n"
>> diff --git a/lib/librte_eal/common/eal_internal_cfg.h
>> b/lib/librte_eal/common/eal_internal_cfg.h
>> index a42f34923..0006f903f 100644
>> --- a/lib/librte_eal/common/eal_internal_cfg.h
>> +++ b/lib/librte_eal/common/eal_internal_cfg.h
>> @@ -44,6 +44,7 @@ struct internal_config {
>> 	unsigned hugepage_unlink;         /**< true to unlink backing files
>> */
>> 	volatile unsigned no_pci;         /**< true to disable PCI */
>> 	volatile unsigned no_hpet;        /**< true to disable HPET */
>> +	volatile unsigned manual_probe;   /**< true to enable manual
>> device probing. */
>> 	volatile unsigned vmware_tsc_map; /**< true to use VMware
>> TSC mapping
>>
>> 	* instead of native TSC */
>> 	volatile unsigned no_shconf;      /**< true if there is no shared
>> config */
>> diff --git a/lib/librte_eal/common/eal_options.h
>> b/lib/librte_eal/common/eal_options.h
>> index 9855429e5..588fa32a6 100644
>> --- a/lib/librte_eal/common/eal_options.h
>> +++ b/lib/librte_eal/common/eal_options.h
>> @@ -69,6 +69,8 @@ enum {
>> 	OPT_IOVA_MODE_NUM,
>> #define OPT_MATCH_ALLOCATIONS  "match-allocations"
>> 	OPT_MATCH_ALLOCATIONS_NUM,
>> +#define OPT_MANUAL_PROBE "manual-probe"
>> +	OPT_MANUAL_PROBE_NUM,
>> 	OPT_LONG_MAX_NUM
>> };
>>
>> diff --git a/lib/librte_eal/common/eal_private.h
>> b/lib/librte_eal/common/eal_private.h
>> index 798ede553..fd7ac8e37 100644
>> --- a/lib/librte_eal/common/eal_private.h
>> +++ b/lib/librte_eal/common/eal_private.h
>> @@ -381,4 +381,13 @@ rte_option_init(void);
>> void
>> rte_option_usage(void);
>>
>> +/**
>> + * Go through the devargs list and probe everything in order.
>> + *
>> + * @return
>> + *   0 on success, negative on error.
>> + */
>> +int
>> +rte_dev_probe_devargs_list(void);
>> +
>> #endif /* _EAL_PRIVATE_H_ */
>> diff --git a/lib/librte_eal/common/include/rte_eal.h
>> b/lib/librte_eal/common/include/rte_eal.h
>> index b7cf91214..71d9b4ca6 100644
>> --- a/lib/librte_eal/common/include/rte_eal.h
>> +++ b/lib/librte_eal/common/include/rte_eal.h
>> @@ -465,6 +465,42 @@ int rte_eal_has_hugepages(void);
>> int rte_eal_has_pci(void);
>>
>> /**
>> + * Whether EAL probe is manual.
>> + * Enabled by the --manual-probe option or by
>> + * using rte_eal_manual_probe_set().
>> + *
>> + * When manual probing is enabled, batched bus probe of
>> + * their devices is disabled. All devices need to be probed
>> + * using the proper rte_dev API.
>> + *
>> + * In this mode, devices declared on the command line will
>> + * be probed using the bus hotplug API. It is used to enforce
>> + * a specific probe order.
>> + *
>> + * @return
>> + *   Nonzero if manual device probing is enabled.
>> + *
>> + * @see rte_eal_manual_probe_set
>> + */
>> +__rte_experimental
>> +int rte_eal_manual_probe(void);
>> +
>> +/**
>> + * Configure EAL probe mode -- manual or automatic.
>> + *
>> + * Enable or disable manual probe mode in EAL.
>> + * This function can be called at any time, but must be used
>> + * before calling rte_eal_init() to have any effect.
>> + *
>> + * @param enabled
>> + *   zero to disable manual probe, non-zero to enable it.
>> + *
>> + * @see rte_eal_manual_probe
>> + */
>> +__rte_experimental
>> +void rte_eal_manual_probe_set(int enabled);
>> +
>> +/**
>>   * Whether the EAL was asked to create UIO device.
>>   *
>>   * @return
>> diff --git a/lib/librte_eal/rte_eal_version.map
>> b/lib/librte_eal/rte_eal_version.map
>> index 7cbf82d37..7d7ff6531 100644
>> --- a/lib/librte_eal/rte_eal_version.map
>> +++ b/lib/librte_eal/rte_eal_version.map
>> @@ -419,4 +419,8 @@ EXPERIMENTAL {
>> 	rte_mcfg_timer_lock;
>> 	rte_mcfg_timer_unlock;
>> 	rte_rand_max;
>> +
>> +	# added in 19.11
>> +	rte_eal_manual_probe;
>> +	rte_eal_manual_probe_set;
>> };
>> --
>> 2.11.0
>

Patch
diff mbox series

diff --git a/doc/guides/linux_gsg/eal_args.include.rst b/doc/guides/linux_gsg/eal_args.include.rst
index ed8b0e35b..d0717d4a0 100644
--- a/doc/guides/linux_gsg/eal_args.include.rst
+++ b/doc/guides/linux_gsg/eal_args.include.rst
@@ -69,6 +69,19 @@  Device-related options
 
        --vdev 'net_pcap0,rx_pcap=input.pcap,tx_pcap=output.pcap'
 
+*   ``--manual-probe``
+
+    Switch the ``EAL`` probe mode to manual. The main bus probe step
+    is disabled and applications are expected to manually probe
+    devices using ``rte_dev_probe()``.
+
+    Devices declared on the command-line using ``-w`` and ``-vdev``
+    are interpreted as hotplug commands. They are thus probed in the
+    order they are declared.
+
+    This makes this option useful to enforce a specific device probe
+    order, instead of relying on each bus scan implementation details.
+
 *   ``-d <path to shared object or directory>``
 
     Load external drivers. An argument can be a single shared object file, or a
diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
index e77d226b5..7251a17b2 100644
--- a/doc/guides/rel_notes/release_19_11.rst
+++ b/doc/guides/rel_notes/release_19_11.rst
@@ -56,6 +56,15 @@  New Features
      Also, make sure to start the actual text at the margin.
      =========================================================
 
+* **EAL will now allow probing devices manually.**
+
+  Previously, a user could not force an order when probing declared devices.
+  This could cause issues for drivers depending on another device being present.
+  A new option ``--manual-probe`` is now available to do just that.
+  This new option relies on the device bus supporting hotplug. It can
+  also be used to disable automatic probing from the ``PCI`` bus without
+  having to disable the whole bus.
+
 * **FreeBSD now supports `--base-virtaddr` EAL option.**
 
   FreeBSD version now also supports setting base virtual address for mapping
diff --git a/lib/librte_eal/common/eal_common_bus.c b/lib/librte_eal/common/eal_common_bus.c
index baa5b532a..145a96812 100644
--- a/lib/librte_eal/common/eal_common_bus.c
+++ b/lib/librte_eal/common/eal_common_bus.c
@@ -6,6 +6,7 @@ 
 #include <string.h>
 #include <sys/queue.h>
 
+#include <rte_eal.h>
 #include <rte_bus.h>
 #include <rte_debug.h>
 #include <rte_string_fns.h>
@@ -63,6 +64,11 @@  rte_bus_probe(void)
 	int ret;
 	struct rte_bus *bus, *vbus = NULL;
 
+	if (rte_eal_manual_probe()) {
+		RTE_LOG(DEBUG, EAL, "Manual probing enabled.\n");
+		return rte_dev_probe_devargs_list();
+	}
+
 	TAILQ_FOREACH(bus, &rte_bus_list, next) {
 		if (!strcmp(bus->name, "vdev")) {
 			vbus = bus;
diff --git a/lib/librte_eal/common/eal_common_dev.c b/lib/librte_eal/common/eal_common_dev.c
index 9e4f09d83..368afa273 100644
--- a/lib/librte_eal/common/eal_common_dev.c
+++ b/lib/librte_eal/common/eal_common_dev.c
@@ -21,6 +21,7 @@ 
 #include <rte_malloc.h>
 #include <rte_string_fns.h>
 
+#include "eal_internal_cfg.h"
 #include "eal_private.h"
 #include "hotplug_mp.h"
 
@@ -83,6 +84,59 @@  rte_dev_is_probed(const struct rte_device *dev)
 	return dev->driver != NULL;
 }
 
+int
+rte_eal_manual_probe(void)
+{
+	return internal_config.manual_probe;
+}
+
+void
+rte_eal_manual_probe_set(int enabled)
+{
+	internal_config.manual_probe = !!enabled;
+}
+
+int
+rte_dev_probe_devargs_list(void)
+{
+	struct rte_device *dev;
+	struct rte_devargs *da;
+	int ret;
+
+	RTE_EAL_DEVARGS_FOREACH(NULL, da) {
+		dev = da->bus->find_device(NULL, cmp_dev_name, da->name);
+		if (dev == NULL) {
+			RTE_LOG(ERR, EAL, "Unable to find device %s on bus %s\n",
+				da->name, da->bus->name);
+			continue;
+		}
+
+		if (rte_dev_is_probed(dev))
+			continue;
+
+		if (dev->bus->plug == NULL) {
+			RTE_LOG(ERR, EAL, "Manual probing (hotplug) not supported by bus %s, "
+					  "required by device %s\n",
+				dev->bus->name, dev->name);
+			continue;
+		}
+
+		ret = dev->bus->plug(dev);
+		/* Ignore positive return values, they are possibly
+		 * triggered by blacklisted devices on the PCI bus. Probing
+		 * should then continue.
+		 */
+		if (ret < 0) {
+			RTE_LOG(ERR, EAL, "Driver cannot attach device %s\n",
+				dev->name);
+			/* Fail on first real probe error. */
+			return ret;
+		}
+	}
+
+	return 0;
+}
+
 /* helper function to build devargs, caller should free the memory */
 static int
 build_devargs(const char *busname, const char *devname,
diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c
index a7f9c5f9b..de73970fe 100644
--- a/lib/librte_eal/common/eal_common_options.c
+++ b/lib/librte_eal/common/eal_common_options.c
@@ -82,6 +82,7 @@  eal_long_options[] = {
 	{OPT_LEGACY_MEM,        0, NULL, OPT_LEGACY_MEM_NUM       },
 	{OPT_SINGLE_FILE_SEGMENTS, 0, NULL, OPT_SINGLE_FILE_SEGMENTS_NUM},
 	{OPT_MATCH_ALLOCATIONS, 0, NULL, OPT_MATCH_ALLOCATIONS_NUM},
+	{OPT_MANUAL_PROBE,      0, NULL, OPT_MANUAL_PROBE_NUM     },
 	{0,                     0, NULL, 0                        }
 };
 
@@ -1446,6 +1447,9 @@  eal_parse_common_option(int opt, const char *optarg,
 			return -1;
 		}
 		break;
+	case OPT_MANUAL_PROBE_NUM:
+		rte_eal_manual_probe_set(1);
+		break;
 
 	/* don't know what to do, leave this to caller */
 	default:
@@ -1672,6 +1676,10 @@  eal_common_usage(void)
 	       "  --"OPT_VDEV"              Add a virtual device.\n"
 	       "                      The argument format is <driver><id>[,key=val,...]\n"
 	       "                      (ex: --vdev=net_pcap0,iface=eth2).\n"
+	       "  --"OPT_MANUAL_PROBE"      Enable manual probing.\n"
+	       "                      Disable probe step for all buses.\n"
+	       "                      Devices will need to be probed using the hotplug API.\n"
+	       "                      PCI and vdev declarations will be treated in order as hotplug commands.\n"
 	       "  --"OPT_IOVA_MODE"   Set IOVA mode. 'pa' for IOVA_PA\n"
 	       "                      'va' for IOVA_VA\n"
 	       "  -d LIB.so|DIR       Add a driver or driver directory\n"
diff --git a/lib/librte_eal/common/eal_internal_cfg.h b/lib/librte_eal/common/eal_internal_cfg.h
index a42f34923..0006f903f 100644
--- a/lib/librte_eal/common/eal_internal_cfg.h
+++ b/lib/librte_eal/common/eal_internal_cfg.h
@@ -44,6 +44,7 @@  struct internal_config {
 	unsigned hugepage_unlink;         /**< true to unlink backing files */
 	volatile unsigned no_pci;         /**< true to disable PCI */
 	volatile unsigned no_hpet;        /**< true to disable HPET */
+	volatile unsigned manual_probe;   /**< true to enable manual device probing. */
 	volatile unsigned vmware_tsc_map; /**< true to use VMware TSC mapping
 										* instead of native TSC */
 	volatile unsigned no_shconf;      /**< true if there is no shared config */
diff --git a/lib/librte_eal/common/eal_options.h b/lib/librte_eal/common/eal_options.h
index 9855429e5..588fa32a6 100644
--- a/lib/librte_eal/common/eal_options.h
+++ b/lib/librte_eal/common/eal_options.h
@@ -69,6 +69,8 @@  enum {
 	OPT_IOVA_MODE_NUM,
 #define OPT_MATCH_ALLOCATIONS  "match-allocations"
 	OPT_MATCH_ALLOCATIONS_NUM,
+#define OPT_MANUAL_PROBE "manual-probe"
+	OPT_MANUAL_PROBE_NUM,
 	OPT_LONG_MAX_NUM
 };
 
diff --git a/lib/librte_eal/common/eal_private.h b/lib/librte_eal/common/eal_private.h
index 798ede553..fd7ac8e37 100644
--- a/lib/librte_eal/common/eal_private.h
+++ b/lib/librte_eal/common/eal_private.h
@@ -381,4 +381,13 @@  rte_option_init(void);
 void
 rte_option_usage(void);
 
+/**
+ * Go through the devargs list and probe everything in order.
+ *
+ * @return
+ *   0 on success, negative on error.
+ */
+int
+rte_dev_probe_devargs_list(void);
+
 #endif /* _EAL_PRIVATE_H_ */
diff --git a/lib/librte_eal/common/include/rte_eal.h b/lib/librte_eal/common/include/rte_eal.h
index b7cf91214..71d9b4ca6 100644
--- a/lib/librte_eal/common/include/rte_eal.h
+++ b/lib/librte_eal/common/include/rte_eal.h
@@ -465,6 +465,42 @@  int rte_eal_has_hugepages(void);
 int rte_eal_has_pci(void);
 
 /**
+ * Whether EAL probe is manual.
+ * Enabled by the --manual-probe option or by
+ * using rte_eal_manual_probe_set().
+ *
+ * When manual probing is enabled, batched bus probe of
+ * their devices is disabled. All devices need to be probed
+ * using the proper rte_dev API.
+ *
+ * In this mode, devices declared on the command line will
+ * be probed using the bus hotplug API. It is used to enforce
+ * a specific probe order.
+ *
+ * @return
+ *   Nonzero if manual device probing is enabled.
+ *
+ * @see rte_eal_manual_probe_set
+ */
+__rte_experimental
+int rte_eal_manual_probe(void);
+
+/**
+ * Configure EAL probe mode -- manual or automatic.
+ *
+ * Enable or disable manual probe mode in EAL.
+ * This function can be called at any time, but must be used
+ * before calling rte_eal_init() to have any effect.
+ *
+ * @param enabled
+ *   zero to disable manual probe, non-zero to enable it.
+ *
+ * @see rte_eal_manual_probe
+ */
+__rte_experimental
+void rte_eal_manual_probe_set(int enabled);
+
+/**
  * Whether the EAL was asked to create UIO device.
  *
  * @return
diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map
index 7cbf82d37..7d7ff6531 100644
--- a/lib/librte_eal/rte_eal_version.map
+++ b/lib/librte_eal/rte_eal_version.map
@@ -419,4 +419,8 @@  EXPERIMENTAL {
 	rte_mcfg_timer_lock;
 	rte_mcfg_timer_unlock;
 	rte_rand_max;
+
+	# added in 19.11
+	rte_eal_manual_probe;
+	rte_eal_manual_probe_set;
 };