[dpdk-dev,v4,08/12] net/failsafe: support offload capabilities
Checks
Commit Message
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
Acked-by: Olga Shern <olgas@mellanox.com>
---
doc/guides/nics/features/failsafe.ini | 6 ++
drivers/net/failsafe/failsafe_ops.c | 131 +++++++++++++++++++++++++++++++++-
2 files changed, 135 insertions(+), 2 deletions(-)
Comments
On Mon, 29 May 2017 15:42:20 +0200
Gaetan Rivet <gaetan.rivet@6wind.com> wrote:
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> Acked-by: Olga Shern <olgas@mellanox.com>
> ---
> doc/guides/nics/features/failsafe.ini | 6 ++
> drivers/net/failsafe/failsafe_ops.c | 131 +++++++++++++++++++++++++++++++++-
> 2 files changed, 135 insertions(+), 2 deletions(-)
Once again what about case of dumb synthetic NIC combined with SR-IOV VF?
The VF has offloads the virtual NIC does not.
What about late plugin. how do you program the offloads of the later arriving
VF device.
On Wed, May 31, 2017 at 08:23:09AM -0700, Stephen Hemminger wrote:
> On Mon, 29 May 2017 15:42:20 +0200
> Gaetan Rivet <gaetan.rivet@6wind.com> wrote:
>
> > Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> > Acked-by: Olga Shern <olgas@mellanox.com>
> > ---
> > doc/guides/nics/features/failsafe.ini | 6 ++
> > drivers/net/failsafe/failsafe_ops.c | 131 +++++++++++++++++++++++++++++++++-
> > 2 files changed, 135 insertions(+), 2 deletions(-)
>
> Once again what about case of dumb synthetic NIC combined with SR-IOV VF?
> The VF has offloads the virtual NIC does not.
>
The rules for capabilities are a little complicated.
In the case both VF and the virtual NIC are present at launch, then the
logical AND is done both their capabilities sets.
If one has additional capabilities that the user is requesting, and the
fail-safe recognize them (currently, all RX offloads, as TX offloads
were not yet expressed by flags), and this capability is not supported
by one slave, then this offload is disabled in the configuration.
> What about late plugin. how do you program the offloads of the later arriving
> VF device.
If the VF is not present at launch, then the fail-safe reads only the
set of capabilities from the fallback device. It does not have to do any
AND-ing of the flags.
The consequence is that upon plugin of the VF, the latter has to respect
the current running configuration. Probing will actually fail if some
capability is not supported (depending on PMDs), and the running
configuration is not updated as it is considered "live".
There are only two solutions to this, either:
* Complicate a lot the fail-safe design and the rules applied in
the decision made on NIC configuration. The user then has bad
surprises upon seeing that his performance have been degraded
for arcane reasons.
* Emulate in software the offloads and try to advertize as many as
possible. This is done for example in the TAP PMD for some flags,
allowing those offloads to be used with hardware NICs.
The user then has a clear view of the available offloads by comparing
both sets of capabilities.
L;w
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Gaëtan Rivet
> Sent: Thursday, June 01, 2017 5:38 PM
> To: Stephen Hemminger <stephen@networkplumber.org>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v4 08/12] net/failsafe: support offload
> capabilities
>
> On Wed, May 31, 2017 at 08:23:09AM -0700, Stephen Hemminger wrote:
> > On Mon, 29 May 2017 15:42:20 +0200
> > Gaetan Rivet <gaetan.rivet@6wind.com> wrote:
> >
> > > Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> > > Acked-by: Olga Shern <olgas@mellanox.com>
> > > ---
> > > doc/guides/nics/features/failsafe.ini | 6 ++
> > > drivers/net/failsafe/failsafe_ops.c | 131
> +++++++++++++++++++++++++++++++++-
> > > 2 files changed, 135 insertions(+), 2 deletions(-)
> >
> > Once again what about case of dumb synthetic NIC combined with SR-IOV
> VF?
> > The VF has offloads the virtual NIC does not.
> >
>
> The rules for capabilities are a little complicated.
> In the case both VF and the virtual NIC are present at launch, then the logical
> AND is done both their capabilities sets.
> If one has additional capabilities that the user is requesting, and the fail-safe
> recognize them (currently, all RX offloads, as TX offloads were not yet
> expressed by flags), and this capability is not supported by one slave, then
> this offload is disabled in the configuration.
>
> > What about late plugin. how do you program the offloads of the later
> > arriving VF device.
>
> If the VF is not present at launch, then the fail-safe reads only the set of
> capabilities from the fallback device. It does not have to do any AND-ing of
> the flags.
>
> The consequence is that upon plugin of the VF, the latter has to respect the
> current running configuration. Probing will actually fail if some capability is not
> supported (depending on PMDs), and the running configuration is not
> updated as it is considered "live".
>
> There are only two solutions to this, either:
>
> * Complicate a lot the fail-safe design and the rules applied in
> the decision made on NIC configuration. The user then has bad
> surprises upon seeing that his performance have been degraded
> for arcane reasons.
>
> * Emulate in software the offloads and try to advertize as many as
> possible. This is done for example in the TAP PMD for some flags,
> allowing those offloads to be used with hardware NICs.
> The user then has a clear view of the available offloads by comparing
> both sets of capabilities.
>
> --
> Gaëtan Rivet
> 6WIND
@@ -14,6 +14,12 @@ Unicast MAC filter = Y
Multicast MAC filter = Y
VLAN filter = Y
Flow API = Y
+VLAN offload = Y
+QinQ offload = Y
+L3 checksum offload = Y
+L4 checksum offload = Y
+Inner L3 checksum = Y
+Inner L4 checksum = Y
Packet type parsing = Y
Basic stats = Y
Stats per queue = Y
@@ -63,22 +63,149 @@ static struct rte_eth_dev_info default_infos = {
.nb_seg_max = UINT16_MAX,
.nb_mtu_seg_max = UINT16_MAX,
},
- /* Set of understood capabilities */
- .rx_offload_capa = 0x0,
+ /*
+ * Set of capabilities that can be verified upon
+ * configuring a sub-device.
+ */
+ .rx_offload_capa =
+ DEV_RX_OFFLOAD_VLAN_STRIP |
+ DEV_RX_OFFLOAD_QINQ_STRIP |
+ DEV_RX_OFFLOAD_IPV4_CKSUM |
+ DEV_RX_OFFLOAD_UDP_CKSUM |
+ DEV_RX_OFFLOAD_TCP_CKSUM |
+ DEV_RX_OFFLOAD_TCP_LRO,
.tx_offload_capa = 0x0,
.flow_type_rss_offloads = 0x0,
};
+/**
+ * Check whether a specific offloading capability
+ * is supported by a sub_device.
+ *
+ * @return
+ * 0: all requested capabilities are supported by the sub_device
+ * positive value: This flag at least is not supported by the sub_device
+ */
+static int
+fs_port_offload_validate(struct rte_eth_dev *dev,
+ struct sub_device *sdev)
+{
+ struct rte_eth_dev_info infos = {0};
+ struct rte_eth_conf *cf;
+ uint32_t cap;
+
+ cf = &dev->data->dev_conf;
+ SUBOPS(sdev, dev_infos_get)(ETH(sdev), &infos);
+ /* RX capabilities */
+ cap = infos.rx_offload_capa;
+ if (cf->rxmode.hw_vlan_strip &&
+ ((cap & DEV_RX_OFFLOAD_VLAN_STRIP) == 0)) {
+ WARN("VLAN stripping offload requested but not supported by sub_device %d",
+ SUB_ID(sdev));
+ return DEV_RX_OFFLOAD_VLAN_STRIP;
+ }
+ if (cf->rxmode.hw_ip_checksum &&
+ ((cap & (DEV_RX_OFFLOAD_IPV4_CKSUM |
+ DEV_RX_OFFLOAD_UDP_CKSUM |
+ DEV_RX_OFFLOAD_TCP_CKSUM)) !=
+ (DEV_RX_OFFLOAD_IPV4_CKSUM |
+ DEV_RX_OFFLOAD_UDP_CKSUM |
+ DEV_RX_OFFLOAD_TCP_CKSUM))) {
+ WARN("IP checksum offload requested but not supported by sub_device %d",
+ SUB_ID(sdev));
+ return DEV_RX_OFFLOAD_IPV4_CKSUM |
+ DEV_RX_OFFLOAD_UDP_CKSUM |
+ DEV_RX_OFFLOAD_TCP_CKSUM;
+ }
+ if (cf->rxmode.enable_lro &&
+ ((cap & DEV_RX_OFFLOAD_TCP_LRO) == 0)) {
+ WARN("TCP LRO offload requested but not supported by sub_device %d",
+ SUB_ID(sdev));
+ return DEV_RX_OFFLOAD_TCP_LRO;
+ }
+ if (cf->rxmode.hw_vlan_extend &&
+ ((cap & DEV_RX_OFFLOAD_QINQ_STRIP) == 0)) {
+ WARN("Stacked VLAN stripping offload requested but not supported by sub_device %d",
+ SUB_ID(sdev));
+ return DEV_RX_OFFLOAD_QINQ_STRIP;
+ }
+ /* TX capabilities */
+ /* Nothing to do, no tx capa supported */
+ return 0;
+}
+
+/*
+ * Disable the dev_conf flag related to an offload capability flag
+ * within an ethdev configuration.
+ */
+static int
+fs_port_disable_offload(struct rte_eth_conf *cf,
+ uint32_t ol_cap)
+{
+ switch (ol_cap) {
+ case DEV_RX_OFFLOAD_VLAN_STRIP:
+ INFO("Disabling VLAN stripping offload");
+ cf->rxmode.hw_vlan_strip = 0;
+ break;
+ case DEV_RX_OFFLOAD_IPV4_CKSUM:
+ case DEV_RX_OFFLOAD_UDP_CKSUM:
+ case DEV_RX_OFFLOAD_TCP_CKSUM:
+ case (DEV_RX_OFFLOAD_IPV4_CKSUM |
+ DEV_RX_OFFLOAD_UDP_CKSUM |
+ DEV_RX_OFFLOAD_TCP_CKSUM):
+ INFO("Disabling IP checksum offload");
+ cf->rxmode.hw_ip_checksum = 0;
+ break;
+ case DEV_RX_OFFLOAD_TCP_LRO:
+ INFO("Disabling TCP LRO offload");
+ cf->rxmode.enable_lro = 0;
+ break;
+ case DEV_RX_OFFLOAD_QINQ_STRIP:
+ INFO("Disabling stacked VLAN stripping offload");
+ cf->rxmode.hw_vlan_extend = 0;
+ break;
+ default:
+ DEBUG("Unable to disable offload capability: %" PRIx32,
+ ol_cap);
+ return -1;
+ }
+ return 0;
+}
+
static int
fs_dev_configure(struct rte_eth_dev *dev)
{
struct sub_device *sdev;
uint8_t i;
+ int capa_flag;
int ret;
FOREACH_SUBDEV(sdev, i, dev) {
if (sdev->state != DEV_PROBED)
continue;
+ DEBUG("Checking capabilities for sub_device %d", i);
+ while ((capa_flag = fs_port_offload_validate(dev, sdev))) {
+ /*
+ * Refuse to change configuration if multiple devices
+ * are present and we already have configured at least
+ * some of them.
+ */
+ if (PRIV(dev)->state >= DEV_ACTIVE &&
+ PRIV(dev)->subs_tail > 1) {
+ ERROR("device already configured, cannot fix live configuration");
+ return -1;
+ }
+ ret = fs_port_disable_offload(&dev->data->dev_conf,
+ capa_flag);
+ if (ret) {
+ ERROR("Unable to disable offload capability");
+ return ret;
+ }
+ }
+ }
+ FOREACH_SUBDEV(sdev, i, dev) {
+ if (sdev->state != DEV_PROBED)
+ continue;
DEBUG("Configuring sub-device %d", i);
ret = rte_eth_dev_configure(PORT_ID(sdev),
dev->data->nb_rx_queues,