[dpdk-dev,2/3] usertools: add hv_uio_setup script
Checks
Commit Message
Small script to rebind netvsc kernel device to Hyper-V
networking PMD. It could be integrated in dpdk-bind, but dpdk-bind
is focused on PCI, and that would get messy.
Eventually, this functionality will be built into netvsc driver
(see vdev_netvsc as an example).
Signed-off-by: Stephen Hemminger <sthemmin@microsoft.com>
---
usertools/hv_uio_setup.sh | 40 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)
create mode 100755 usertools/hv_uio_setup.sh
Comments
05/04/2018 21:13, Stephen Hemminger:
> Small script to rebind netvsc kernel device to Hyper-V
> networking PMD. It could be integrated in dpdk-bind, but dpdk-bind
> is focused on PCI, and that would get messy.
>
> Eventually, this functionality will be built into netvsc driver
> (see vdev_netvsc as an example).
I believe we should avoid creating such script.
The direction to go, for hotplug, is to remove dpdk-devbind.py,
and implement kernel binding in PMDs (with EAL helpers).
In order to make this change happen, we should not
add this hv_uio_setup.sh script.
On Thu, 05 Apr 2018 22:43:39 +0200
Thomas Monjalon <thomas@monjalon.net> wrote:
> 05/04/2018 21:13, Stephen Hemminger:
> > Small script to rebind netvsc kernel device to Hyper-V
> > networking PMD. It could be integrated in dpdk-bind, but dpdk-bind
> > is focused on PCI, and that would get messy.
> >
> > Eventually, this functionality will be built into netvsc driver
> > (see vdev_netvsc as an example).
>
> I believe we should avoid creating such script.
> The direction to go, for hotplug, is to remove dpdk-devbind.py,
> and implement kernel binding in PMDs (with EAL helpers).
>
> In order to make this change happen, we should not
> add this hv_uio_setup.sh script.
Yes, this is a temporary script like dpdk-bind, want to get rid of it
and do everything inside driver. That is the next step.
On Thu, Apr 05, 2018 at 10:43:39PM +0200, Thomas Monjalon wrote:
> 05/04/2018 21:13, Stephen Hemminger:
> > Small script to rebind netvsc kernel device to Hyper-V
> > networking PMD. It could be integrated in dpdk-bind, but dpdk-bind
> > is focused on PCI, and that would get messy.
> >
> > Eventually, this functionality will be built into netvsc driver
> > (see vdev_netvsc as an example).
>
> I believe we should avoid creating such script.
> The direction to go, for hotplug, is to remove dpdk-devbind.py,
> and implement kernel binding in PMDs (with EAL helpers).
>
I'm not convinced at all that that is the direction to go. I instead would
prefer to see all binding happen outside DPDK. I believe having udev or
similar manage bindings, set up via e.g driverctl[1], is a far better path.
Just my 2c.
/Bruce
[1] https://gitlab.com/driverctl/driverctl
05/04/2018 23:07, Bruce Richardson:
> On Thu, Apr 05, 2018 at 10:43:39PM +0200, Thomas Monjalon wrote:
> > 05/04/2018 21:13, Stephen Hemminger:
> > > Small script to rebind netvsc kernel device to Hyper-V
> > > networking PMD. It could be integrated in dpdk-bind, but dpdk-bind
> > > is focused on PCI, and that would get messy.
> > >
> > > Eventually, this functionality will be built into netvsc driver
> > > (see vdev_netvsc as an example).
> >
> > I believe we should avoid creating such script.
> > The direction to go, for hotplug, is to remove dpdk-devbind.py,
> > and implement kernel binding in PMDs (with EAL helpers).
> >
> I'm not convinced at all that that is the direction to go. I instead would
> prefer to see all binding happen outside DPDK. I believe having udev or
> similar manage bindings, set up via e.g driverctl[1], is a far better path.
This is a system admin tool, and only for Linux.
Having the binding logic inside DPDK, allows the application to control
how hotplug behave.
05/04/2018 23:03, Stephen Hemminger:
> On Thu, 05 Apr 2018 22:43:39 +0200
> Thomas Monjalon <thomas@monjalon.net> wrote:
>
> > 05/04/2018 21:13, Stephen Hemminger:
> > > Small script to rebind netvsc kernel device to Hyper-V
> > > networking PMD. It could be integrated in dpdk-bind, but dpdk-bind
> > > is focused on PCI, and that would get messy.
> > >
> > > Eventually, this functionality will be built into netvsc driver
> > > (see vdev_netvsc as an example).
> >
> > I believe we should avoid creating such script.
> > The direction to go, for hotplug, is to remove dpdk-devbind.py,
> > and implement kernel binding in PMDs (with EAL helpers).
> >
> > In order to make this change happen, we should not
> > add this hv_uio_setup.sh script.
>
> Yes, this is a temporary script like dpdk-bind, want to get rid of it
> and do everything inside driver. That is the next step.
If this is temporary, it is a step in the wrong direction which
could confuse users.
If it becomes definitive (design discussion in progress), then it should
be merged in dpdk-devbind.py.
On Thu, 05 Apr 2018 23:13:54 +0200
Thomas Monjalon <thomas@monjalon.net> wrote:
> 05/04/2018 23:03, Stephen Hemminger:
> > On Thu, 05 Apr 2018 22:43:39 +0200
> > Thomas Monjalon <thomas@monjalon.net> wrote:
> >
> > > 05/04/2018 21:13, Stephen Hemminger:
> > > > Small script to rebind netvsc kernel device to Hyper-V
> > > > networking PMD. It could be integrated in dpdk-bind, but dpdk-bind
> > > > is focused on PCI, and that would get messy.
> > > >
> > > > Eventually, this functionality will be built into netvsc driver
> > > > (see vdev_netvsc as an example).
> > >
> > > I believe we should avoid creating such script.
> > > The direction to go, for hotplug, is to remove dpdk-devbind.py,
> > > and implement kernel binding in PMDs (with EAL helpers).
> > >
> > > In order to make this change happen, we should not
> > > add this hv_uio_setup.sh script.
> >
> > Yes, this is a temporary script like dpdk-bind, want to get rid of it
> > and do everything inside driver. That is the next step.
>
> If this is temporary, it is a step in the wrong direction which
> could confuse users.
>
> If it becomes definitive (design discussion in progress), then it should
> be merged in dpdk-devbind.py.
>
>
This is an experimental driver, if everyone waits until everything is done
there would be no review or testing.
On Thu, 05 Apr 2018 23:13:54 +0200
Thomas Monjalon <thomas@monjalon.net> wrote:
> 05/04/2018 23:03, Stephen Hemminger:
> > On Thu, 05 Apr 2018 22:43:39 +0200
> > Thomas Monjalon <thomas@monjalon.net> wrote:
> >
> > > 05/04/2018 21:13, Stephen Hemminger:
> > > > Small script to rebind netvsc kernel device to Hyper-V
> > > > networking PMD. It could be integrated in dpdk-bind, but dpdk-bind
> > > > is focused on PCI, and that would get messy.
> > > >
> > > > Eventually, this functionality will be built into netvsc driver
> > > > (see vdev_netvsc as an example).
> > >
> > > I believe we should avoid creating such script.
> > > The direction to go, for hotplug, is to remove dpdk-devbind.py,
> > > and implement kernel binding in PMDs (with EAL helpers).
> > >
> > > In order to make this change happen, we should not
> > > add this hv_uio_setup.sh script.
> >
> > Yes, this is a temporary script like dpdk-bind, want to get rid of it
> > and do everything inside driver. That is the next step.
>
> If this is temporary, it is a step in the wrong direction which
> could confuse users.
>
> If it becomes definitive (design discussion in progress), then it should
> be merged in dpdk-devbind.py.
I looked into changing dpdk-devbind.py but it needed lots of work and to be
honest the time for me to do it python was too much trouble and likely to
break existing users.
On Thu, 05 Apr 2018 23:13:54 +0200
Thomas Monjalon <thomas@monjalon.net> wrote:
> 05/04/2018 23:03, Stephen Hemminger:
> > On Thu, 05 Apr 2018 22:43:39 +0200
> > Thomas Monjalon <thomas@monjalon.net> wrote:
> >
> > > 05/04/2018 21:13, Stephen Hemminger:
> > > > Small script to rebind netvsc kernel device to Hyper-V
> > > > networking PMD. It could be integrated in dpdk-bind, but dpdk-bind
> > > > is focused on PCI, and that would get messy.
> > > >
> > > > Eventually, this functionality will be built into netvsc driver
> > > > (see vdev_netvsc as an example).
> > >
> > > I believe we should avoid creating such script.
> > > The direction to go, for hotplug, is to remove dpdk-devbind.py,
> > > and implement kernel binding in PMDs (with EAL helpers).
> > >
> > > In order to make this change happen, we should not
> > > add this hv_uio_setup.sh script.
> >
> > Yes, this is a temporary script like dpdk-bind, want to get rid of it
> > and do everything inside driver. That is the next step.
>
> If this is temporary, it is a step in the wrong direction which
> could confuse users.
>
> If it becomes definitive (design discussion in progress), then it should
> be merged in dpdk-devbind.py.
Right now PCI does it the same way. It doesn't have a good cold plug interface.
The whole PCI probe logic expects that the device will have a vfio/uio already
bound.
When PCI is fixed, then VMBUS can be changed to the same thing.
On Thu, 05 Apr 2018 23:10:33 +0200
Thomas Monjalon <thomas@monjalon.net> wrote:
> 05/04/2018 23:07, Bruce Richardson:
> > On Thu, Apr 05, 2018 at 10:43:39PM +0200, Thomas Monjalon wrote:
> > > 05/04/2018 21:13, Stephen Hemminger:
> > > > Small script to rebind netvsc kernel device to Hyper-V
> > > > networking PMD. It could be integrated in dpdk-bind, but dpdk-bind
> > > > is focused on PCI, and that would get messy.
> > > >
> > > > Eventually, this functionality will be built into netvsc driver
> > > > (see vdev_netvsc as an example).
> > >
> > > I believe we should avoid creating such script.
> > > The direction to go, for hotplug, is to remove dpdk-devbind.py,
> > > and implement kernel binding in PMDs (with EAL helpers).
> > >
> > I'm not convinced at all that that is the direction to go. I instead would
> > prefer to see all binding happen outside DPDK. I believe having udev or
> > similar manage bindings, set up via e.g driverctl[1], is a far better path.
>
> This is a system admin tool, and only for Linux.
> Having the binding logic inside DPDK, allows the application to control
> how hotplug behave.
>
>
What about using driverctl?
That solution would work for both PCI and VMBUS, but not sure how widely
adopted it is by distributions.
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon
> Sent: Thursday, April 5, 2018 10:11 PM
> To: Richardson, Bruce <bruce.richardson@intel.com>
> Cc: Stephen Hemminger <sthemmin@microsoft.com>; dev@dpdk.org; Stephen Hemminger <stephen@networkplumber.org>
> Subject: Re: [dpdk-dev] [PATCH 2/3] usertools: add hv_uio_setup script
>
> 05/04/2018 23:07, Bruce Richardson:
> > On Thu, Apr 05, 2018 at 10:43:39PM +0200, Thomas Monjalon wrote:
> > > 05/04/2018 21:13, Stephen Hemminger:
> > > > Small script to rebind netvsc kernel device to Hyper-V
> > > > networking PMD. It could be integrated in dpdk-bind, but dpdk-bind
> > > > is focused on PCI, and that would get messy.
> > > >
> > > > Eventually, this functionality will be built into netvsc driver
> > > > (see vdev_netvsc as an example).
> > >
> > > I believe we should avoid creating such script.
> > > The direction to go, for hotplug, is to remove dpdk-devbind.py,
> > > and implement kernel binding in PMDs (with EAL helpers).
> > >
> > I'm not convinced at all that that is the direction to go. I instead would
> > prefer to see all binding happen outside DPDK. I believe having udev or
> > similar manage bindings, set up via e.g driverctl[1], is a far better path.
>
> This is a system admin tool, and only for Linux.
> Having the binding logic inside DPDK, allows the application to control
> how hotplug behave.
I also don't think that DPDK application should control hotplug behavior logic.
It is clearly up to the system admin to make such decisions.
Konstantin
On Thu, 5 Apr 2018 23:57:47 +0000
"Ananyev, Konstantin" <konstantin.ananyev@intel.com> wrote:
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon
> > Sent: Thursday, April 5, 2018 10:11 PM
> > To: Richardson, Bruce <bruce.richardson@intel.com>
> > Cc: Stephen Hemminger <sthemmin@microsoft.com>; dev@dpdk.org; Stephen Hemminger <stephen@networkplumber.org>
> > Subject: Re: [dpdk-dev] [PATCH 2/3] usertools: add hv_uio_setup script
> >
> > 05/04/2018 23:07, Bruce Richardson:
> > > On Thu, Apr 05, 2018 at 10:43:39PM +0200, Thomas Monjalon wrote:
> > > > 05/04/2018 21:13, Stephen Hemminger:
> > > > > Small script to rebind netvsc kernel device to Hyper-V
> > > > > networking PMD. It could be integrated in dpdk-bind, but dpdk-bind
> > > > > is focused on PCI, and that would get messy.
> > > > >
> > > > > Eventually, this functionality will be built into netvsc driver
> > > > > (see vdev_netvsc as an example).
> > > >
> > > > I believe we should avoid creating such script.
> > > > The direction to go, for hotplug, is to remove dpdk-devbind.py,
> > > > and implement kernel binding in PMDs (with EAL helpers).
> > > >
> > > I'm not convinced at all that that is the direction to go. I instead would
> > > prefer to see all binding happen outside DPDK. I believe having udev or
> > > similar manage bindings, set up via e.g driverctl[1], is a far better path.
> >
> > This is a system admin tool, and only for Linux.
> > Having the binding logic inside DPDK, allows the application to control
> > how hotplug behave.
>
> I also don't think that DPDK application should control hotplug behavior logic.
> It is clearly up to the system admin to make such decisions.
> Konstantin
My preference would be to get driverctl working as a standard tool.
But it requires kernel changes to work with vmbus.
On Thu, Apr 05, 2018 at 05:22:42PM -0700, Stephen Hemminger wrote:
> On Thu, 5 Apr 2018 23:57:47 +0000
> "Ananyev, Konstantin" <konstantin.ananyev@intel.com> wrote:
>
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon
> > > Sent: Thursday, April 5, 2018 10:11 PM
> > > To: Richardson, Bruce <bruce.richardson@intel.com>
> > > Cc: Stephen Hemminger <sthemmin@microsoft.com>; dev@dpdk.org; Stephen Hemminger <stephen@networkplumber.org>
> > > Subject: Re: [dpdk-dev] [PATCH 2/3] usertools: add hv_uio_setup script
> > >
> > > 05/04/2018 23:07, Bruce Richardson:
> > > > On Thu, Apr 05, 2018 at 10:43:39PM +0200, Thomas Monjalon wrote:
> > > > > 05/04/2018 21:13, Stephen Hemminger:
> > > > > > Small script to rebind netvsc kernel device to Hyper-V
> > > > > > networking PMD. It could be integrated in dpdk-bind, but dpdk-bind
> > > > > > is focused on PCI, and that would get messy.
> > > > > >
> > > > > > Eventually, this functionality will be built into netvsc driver
> > > > > > (see vdev_netvsc as an example).
> > > > >
> > > > > I believe we should avoid creating such script.
> > > > > The direction to go, for hotplug, is to remove dpdk-devbind.py,
> > > > > and implement kernel binding in PMDs (with EAL helpers).
> > > > >
> > > > I'm not convinced at all that that is the direction to go. I instead would
> > > > prefer to see all binding happen outside DPDK. I believe having udev or
> > > > similar manage bindings, set up via e.g driverctl[1], is a far better path.
> > >
> > > This is a system admin tool, and only for Linux.
> > > Having the binding logic inside DPDK, allows the application to control
> > > how hotplug behave.
> >
> > I also don't think that DPDK application should control hotplug behavior logic.
> > It is clearly up to the system admin to make such decisions.
> > Konstantin
>
> My preference would be to get driverctl working as a standard tool.
> But it requires kernel changes to work with vmbus.
>
+1
I don't think that binding should be done by DPDK for a couple of reasons:
1. There are already daemons and kernel supports out there, such as udev,
for managing devices on a system level. I'd rather not see DPDK duplicate
functionality, when we can re-use what is there. Also there exists the
possibility of conflict, e.g. what if udev has a rule for a device, and
DPDK also tries to manage it at the same time.
2. I believe that the app is the wrong place to manage the binding of
devices, since it's up to the system administrator not the app to determine
the exact setup for the platform. If apps are to manage binding, then each
app will have to expose to the user/sysadmin cmdline options to specify
what devices should be hotplugged into the app or not, and what drivers
they should be bound too. Not all NICs hotplugged to a platform are for
DPDK use, and they won't all want to use the igb_uio or the vfio_pci
drivers. Better that that is configured for each platform on the platform
itself.
I really feel that the driverctl approach is the best one - yes it's linux
only for now, but architecturally I think it's the proper solution.
/Bruce
new file mode 100755
@@ -0,0 +1,40 @@
+#! /bin/bash
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2018 Microsoft Corporation
+
+module=uio_hv_generic
+# Hyper-V network device GUID
+net_guid="f8615163-df3e-46c5-913f-f2d2f965ed0e"
+
+if [ $# -ne 1 ]; then
+ echo "Usage: $0 ethN"
+ exit 1
+fi
+
+syspath=/sys/class/net/$1/device
+devpath=$(readlink $syspath)
+if [ $? -ne 0 ]; then
+ echo "$1 no device present"
+ exit 1
+fi
+dev_guid=$(basename $devpath)
+
+driver=$(readlink $syspath/driver)
+if [ $? -ne 0 ]; then
+ echo "$1 driver not found"
+ exit 1
+fi
+existing=$(basename $driver)
+
+if [ "$existing" != "hv_netvsc" ]; then
+ echo "$1 controlled by $existing"
+ exit 1
+fi
+
+if [ ! -d /sys/module/$module ]; then
+ modprobe $module
+ echo $net_guid >/sys/bus/vmbus/drivers/uio_hv_generic/new_id
+fi
+
+echo $dev_guid > /sys/bus/vmbus/drivers/$existing/unbind
+echo $dev_guid > /sys/bus/vmbus/drivers/$module/bind