From patchwork Thu May 3 19:37:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "John Daley (johndale)" X-Patchwork-Id: 39362 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8CE25DED; Thu, 3 May 2018 21:45:46 +0200 (CEST) Received: from rcdn-iport-9.cisco.com (rcdn-iport-9.cisco.com [173.37.86.80]) by dpdk.org (Postfix) with ESMTP id ECE6123C; Thu, 3 May 2018 21:45:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=6615; q=dns/txt; s=iport; t=1525376745; x=1526586345; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=xNRHXj658/U1tE3o7QapKhBntEN+UYXw37PECfak40g=; b=XgZSArwOMwFkQB0NP9tGCVtBCBz07ksRBFhfkKGKnOzJzccXDPnGjWIE T9IYMcW/qyjctRqRpYYCm78CbNtSzeoy5OqWDJrbCzxyY1N3eiHinUFWY uX490O9M2m07rECQVbG1r2bftqAhobMgu5otpxDEU1656WVNMj6caWQB1 4=; X-IronPort-AV: E=Sophos;i="5.49,359,1520899200"; d="scan'208";a="382599512" Received: from rcdn-core-5.cisco.com ([173.37.93.156]) by rcdn-iport-9.cisco.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 May 2018 19:45:43 +0000 Received: from cisco.com (savbu-usnic-a.cisco.com [10.193.184.48]) by rcdn-core-5.cisco.com (8.14.5/8.14.5) with ESMTP id w43JjhIq013344; Thu, 3 May 2018 19:45:43 GMT Received: by cisco.com (Postfix, from userid 392789) id 0B11D20F2001; Thu, 3 May 2018 12:45:43 -0700 (PDT) From: John Daley To: ferruh.yigit@intel.com Cc: dev@dpdk.org, Hyong Youb Kim , stable@dpdk.org Date: Thu, 3 May 2018 12:37:11 -0700 Message-Id: <20180503193713.20622-4-johndale@cisco.com> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180503193713.20622-1-johndale@cisco.com> References: <20180503193713.20622-1-johndale@cisco.com> Subject: [dpdk-dev] [PATCH 4/6] doc: update the enic guide and features X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Hyong Youb Kim Add more descriptions regarding SR-IOV and RSS settings. Remove 'Multicast MAC filter' and add 'Allmulticast mode' to the features. Cc: stable@dpdk.org Signed-off-by: Hyong Youb Kim Reviewed-by: John Daley Reviewed-by: Aaron Conole --- doc/guides/nics/enic.rst | 60 ++++++++++++++++++++++++++++++--------- doc/guides/nics/features/enic.ini | 3 +- 2 files changed, 49 insertions(+), 14 deletions(-) diff --git a/doc/guides/nics/enic.rst b/doc/guides/nics/enic.rst index 49abc7e95..4505dcf48 100644 --- a/doc/guides/nics/enic.rst +++ b/doc/guides/nics/enic.rst @@ -124,6 +124,14 @@ Configuration information least one interrupt for each Rx queue. For example, if the app uses 3 Rx queues and wants to use per-queue interrupts, configure 4 (3 + 1) interrupts. + - **Receive Side Scaling** + + In order to fully utilize RSS in DPDK, enable all RSS related settings in + CIMC or UCSM. These include the following items listed under + Receive Side Scaling: + TCP, IPv4, TCP-IPv4, IPv6, TCP-IPv6, IPv6 Extension, TCP-IPv6 Extension. + + .. _enic-flow-director: Flow director support @@ -145,20 +153,21 @@ perfect filtering of the 5-tuple with no masking of fields supported. SR-IOV mode utilization ----------------------- -UCS blade servers configured with dynamic vNIC connection policies in UCS -manager are capable of supporting assigned devices on virtual machines (VMs) -through a KVM hypervisor. Assigned devices, also known as 'passthrough' -devices, are SR-IOV virtual functions (VFs) on the host which are exposed -to VM instances. +UCS blade servers configured with dynamic vNIC connection policies in UCSM +are capable of supporting SR-IOV. SR-IOV virtual functions (VFs) are +specialized vNICs, distinct from regular Ethernet vNICs. These VFs can be +directly assigned to virtual machines (VMs) as 'passthrough' devices. -The Cisco Virtual Machine Fabric Extender (VM-FEX) gives the VM a dedicated +In UCS, SR-IOV VFs require the use of the Cisco Virtual Machine Fabric Extender +(VM-FEX), which gives the VM a dedicated interface on the Fabric Interconnect (FI). Layer 2 switching is done at the FI. This may eliminate the requirement for software switching on the host to route intra-host VM traffic. Please refer to `Creating a Dynamic vNIC Connection Policy `_ -for information on configuring SR-IOV adapter policies using UCS manager. +for information on configuring SR-IOV adapter policies and port profiles +using UCSM. Once the policies are in place and the host OS is rebooted, VFs should be visible on the host, E.g.: @@ -175,30 +184,37 @@ visible on the host, E.g.: 0d:00.6 Ethernet controller: Cisco Systems Inc VIC SR-IOV VF (rev a2) 0d:00.7 Ethernet controller: Cisco Systems Inc VIC SR-IOV VF (rev a2) -Enable Intel IOMMU on the host and install KVM and libvirt. A VM instance should -be created with an assigned device. When using libvirt, this configuration can -be done within the domain (i.e. VM) config file. For example this entry maps -host VF 0d:00:01 into the VM. +Enable Intel IOMMU on the host and install KVM and libvirt, and reboot again as +required. Then, using libvirt, create a VM instance with an assigned device. +Below is an example ``interface`` block (part of the domain configuration XML) +that adds the host VF 0d:00:01 to the VM. ``profileid='pp-vlan-25'`` indicates +the port profile that has been configured in UCSM. .. code-block:: console +
+ + + + + Alternatively, the configuration can be done in a separate file using the ``network`` keyword. These methods are described in the libvirt documentation for `Network XML format `_. -When the VM instance is started, the ENIC KVM driver will bind the host VF to +When the VM instance is started, libvirt will bind the host VF to vfio, complete provisioning on the FI and bring up the link. .. note:: It is not possible to use a VF directly from the host because it is not - fully provisioned until the hypervisor brings up the VM that it is assigned + fully provisioned until libvirt brings up the VM that it is assigned to. In the VM instance, the VF will now be visible. E.g., here the VF 00:04.0 is @@ -212,9 +228,27 @@ seen on the VM instance and should be available for binding to a DPDK. Follow the normal DPDK install procedure, binding the VF to either ``igb_uio`` or ``vfio`` in non-IOMMU mode. +In the VM, the kernel enic driver may be automatically bound to the VF during +boot. Unbinding it currently hangs due to a known issue with the driver. To +work around the issue, blacklist the enic module as follows. Please see :ref:`Limitations ` for limitations in the use of SR-IOV. +.. code-block:: console + + # cat /etc/modprobe.d/enic.conf + blacklist enic + + # dracut --force + +.. note:: + + Passthrough does not require SR-IOV. If VM-FEX is not desired, the user + may create as many regular vNICs as necessary and assign them to VMs as + passthrough devices. Since these vNICs are not SR-IOV VFs, using them as + passthrough devices do not require libvirt, port profiles, and VM-FEX. + + .. _enic-genic-flow-api: Generic Flow API support diff --git a/doc/guides/nics/features/enic.ini b/doc/guides/nics/features/enic.ini index 99d37708c..ae46d299a 100644 --- a/doc/guides/nics/features/enic.ini +++ b/doc/guides/nics/features/enic.ini @@ -13,8 +13,9 @@ Jumbo frame = Y Scattered Rx = Y TSO = Y Promiscuous mode = Y +Allmulticast mode = Y Unicast MAC filter = Y -Multicast MAC filter = Y +Multicast MAC filter = RSS hash = Y RSS key update = Y RSS reta update = Y