[dpdk-dev,v4,1/3] fm10k: enable FTAG based forwarding
Commit Message
This patch enables reading sglort info into mbuf for RX and inserting
an FTAG at the beginning of the packet for TX. The vlan_tci_outer field
selected from rte_mbuf structure for sglort is not used in fm10k now.
In FTAG based forwarding mode, the switch will forward packets according
to glort info in FTAG rather than mac and vlan table.
To activate this feature, user needs to pass a devargs parameter to eal
for fm10k device like "-w 0000:84:00.0,enable_ftag=1". Currently this
feature is supported only on PF, because FM10K_PFVTCTL register is
read-only for VF.
Signed-off-by: Wang Xiao W <xiao.w.wang@intel.com>
Acked-by: Jing Chen <jing.d.chen@intel.com>
Acked-by: John McNamara <john.mcnamara@intel.com>
---
drivers/net/fm10k/fm10k.h | 2 ++
drivers/net/fm10k/fm10k_ethdev.c | 38 +++++++++++++++++++++++++++++++++++++-
drivers/net/fm10k/fm10k_rxtx.c | 15 +++++++++++++++
drivers/net/fm10k/fm10k_rxtx_vec.c | 4 ++++
4 files changed, 58 insertions(+), 1 deletion(-)
Comments
2016-03-01 13:36, Wang Xiao W:
> +static int
> +fm10k_check_ftag(struct rte_devargs *devargs)
> +{
> + if (devargs == NULL)
> + return 0;
> +
> + if (strstr(devargs->args, "enable_ftag=1") == NULL)
> + return 0;
> +
> + return 1;
> +}
With strstr(), chenable_ftag=12 will work :)
Please check how to use kvargs.
> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> Sent: Tuesday, March 1, 2016 3:36 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>
> Cc: dev@dpdk.org; Chen, Jing D <jing.d.chen@intel.com>
> Subject: Re: [dpdk-dev] [PATCH v4 1/3] fm10k: enable FTAG based forwarding
>
> 2016-03-01 13:36, Wang Xiao W:
> > +static int
> > +fm10k_check_ftag(struct rte_devargs *devargs) {
> > + if (devargs == NULL)
> > + return 0;
> > +
> > + if (strstr(devargs->args, "enable_ftag=1") == NULL)
> > + return 0;
> > +
> > + return 1;
> > +}
>
> With strstr(), chenable_ftag=12 will work :) Please check how to use kvargs.
OK, it's better to standardize the args string processing. I'll rework it.
Best Regards,
Xiao
On Tue, 1 Mar 2016 13:36:39 +0800
Wang Xiao W <xiao.w.wang@intel.com> wrote:
>
> +static int
> +fm10k_check_ftag(struct rte_devargs *devargs)
> +{
> + if (devargs == NULL)
> + return 0;
> +
> + if (strstr(devargs->args, "enable_ftag=1") == NULL)
> + return 0;
> +
> + return 1;
> +}
> +
It is good to see the DPDK keeping up with the leading edge of hardware
support.
My issue is that devargs are the Linux module parameters method of
configuration in the DPDK world. They are an API only a developer
would love..
1. It has to be done at boot
2. Applications have to rewrite (or expect customer) to pass args
3. Can't be changed at runtime
4. Can't be selected on per device basis.
Please find a better way.
Hi,
> -----Original Message-----
> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> Sent: Wednesday, March 2, 2016 6:38 AM
> To: Wang, Xiao W <xiao.w.wang@intel.com>
> Cc: Chen, Jing D <jing.d.chen@intel.com>; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v4 1/3] fm10k: enable FTAG based forwarding
>
> On Tue, 1 Mar 2016 13:36:39 +0800
> Wang Xiao W <xiao.w.wang@intel.com> wrote:
>
> >
> > +static int
> > +fm10k_check_ftag(struct rte_devargs *devargs) {
> > + if (devargs == NULL)
> > + return 0;
> > +
> > + if (strstr(devargs->args, "enable_ftag=1") == NULL)
> > + return 0;
> > +
> > + return 1;
> > +}
> > +
>
> It is good to see the DPDK keeping up with the leading edge of hardware
> support.
>
> My issue is that devargs are the Linux module parameters method of
> configuration in the DPDK world. They are an API only a developer would love..
DPDK supports passing string parameter (devargs) for every pci device to EAL, it's like:
"./test -w 84:00.0,enable_ftag=1 -w 86:00.0,enable_ftag=1 -c f -n 4".
We have discussed (in v3) on how to configure the specific feature for fm10k (by means
of adding a build time config option, or changing the common ethdev structure,
or changing the mbuf structure), neither adding extra build config option nor
changing the common structure looks satisfactory, and we think using devargs is
a good enough solution.
>
> 1. It has to be done at boot
> 2. Applications have to rewrite (or expect customer) to pass args 3. Can't be
> changed at runtime 4. Can't be selected on per device basis.
Customers configure the FTAG feature according to their need.
We just parse the devargs at dev_start and set FTAG flag for RX/TX queues, we
(or the customers) needn't to change the devargs at runtime.
In DPDK devargs is designed for each pci device, each device has its own pointer
for devargs structure.
Thank you for the comment.
Best Regards,
Xiao
v5:
* Used kvargs api to parse the devargs parameter.
* Put release note into the driver patch.
v4:
* Removed the build time config option, used devargs to config FTAG.
* Rebased on head of dpdk-next-net/rel_16_04 branch.
v3:
* Removed "\n" in PMD_INIT_LOG.
* Returned "-ENOTSUP" instead of -1 in VF FTAG use case.
v2:
* Gave an error message for VF FTAG use case.
* Added a notice in the doc to emphasize that application should ensure
an appropriate FTAG for every frame in FTAG based forwarding mode.
Wang Xiao W (2):
fm10k: enable FTAG based forwarding
doc: add introduction for fm10k FTAG based forwarding
doc/guides/nics/fm10k.rst | 16 ++++++++-
doc/guides/rel_notes/release_16_04.rst | 2 ++
drivers/net/fm10k/fm10k.h | 2 ++
drivers/net/fm10k/fm10k_ethdev.c | 65 +++++++++++++++++++++++++++++++++-
drivers/net/fm10k/fm10k_rxtx.c | 15 ++++++++
drivers/net/fm10k/fm10k_rxtx_vec.c | 3 ++
6 files changed, 101 insertions(+), 2 deletions(-)
2016-03-01 14:37, Stephen Hemminger:
> On Tue, 1 Mar 2016 13:36:39 +0800
> Wang Xiao W <xiao.w.wang@intel.com> wrote:
> > +static int
> > +fm10k_check_ftag(struct rte_devargs *devargs)
> > +{
> > + if (devargs == NULL)
> > + return 0;
> > +
> > + if (strstr(devargs->args, "enable_ftag=1") == NULL)
> > + return 0;
> > +
> > + return 1;
> > +}
>
> It is good to see the DPDK keeping up with the leading edge of hardware
> support.
>
> My issue is that devargs are the Linux module parameters method of
> configuration in the DPDK world. They are an API only a developer
> would love..
>
> 1. It has to be done at boot
For vdev it can be done later.
The devargs can be generalized in the driver model to provide a
configuration interface per device.
> 2. Applications have to rewrite (or expect customer) to pass args
Like said above, if the devargs are correctly implemented, there will
be some API to pass them.
> 3. Can't be changed at runtime
Same point as 1.
> 4. Can't be selected on per device basis.
No. The devargs are args per device.
For PCI, they are currently passed in the whitelist.
> Please find a better way.
Another way would be to extend the configuration structures.
I think it's better to re-think the device configuration and the command line
using some devargs and more functions, ops and structs to configure the really
generic stuff.
This devargs goes in the direction of a flexible configuration, so I'd vote +1.
Tested-by: Yong Liu <yong.liu@intel.com>
- Tested Branch: dpdk-next-net/rel_16_04
- Tested Commit: 4ac366ba647909c3b71818f9be9db86ba5e871da
- OS: Fedora20 3.11.10-301.fc20.x86_64
- GCC: gcc version 4.8.3 20140911
- CPU: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz
- NIC: Intel Corporation Device RedrockCanyou [8086:15a4]
- Default x86_64-native-linuxapp-gcc configuration
- Prerequisites:
- Total 1 cases, 1 passed, 0 failed
- Prerequisites command / instruction:
Apply fm_ftag unit test patch.
export Port0 and Port1's GLORT ID to environment variables
export PORT1_GLORT=0x4200
export PORT0_GLORT=0x4000
- Case: Ftag forwarding unit test
Description: check fm10k nic can forwarding packets based on FTAG
Command / instruction:
Start test application and run fm10k_ftag_autotest
test -c f -n 4 -w 83:00.0,enable_ftag=1 -w 85:00.0,enable_ftag=1
RTE>>fm10k_ftag_autotest
Send packet to Port0 and verify packet forwarded to Port1
Receive 1 packets on port 0
test for FTAG RX passed
Send out 1 packets with FTAG on port 0
Receive 1 packets on port 1
test for FTAG TX passed
Test OK
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wang Xiao W
> Sent: Wednesday, March 02, 2016 7:19 PM
> To: Chen, Jing D
> Cc: dev@dpdk.org
> Subject: [dpdk-dev] [PATCH v5 0/2] fm10k: enable FTAG based forwarding
>
> v5:
> * Used kvargs api to parse the devargs parameter.
> * Put release note into the driver patch.
>
> v4:
> * Removed the build time config option, used devargs to config FTAG.
> * Rebased on head of dpdk-next-net/rel_16_04 branch.
>
> v3:
> * Removed "\n" in PMD_INIT_LOG.
> * Returned "-ENOTSUP" instead of -1 in VF FTAG use case.
>
> v2:
> * Gave an error message for VF FTAG use case.
> * Added a notice in the doc to emphasize that application should ensure
> an appropriate FTAG for every frame in FTAG based forwarding mode.
>
> Wang Xiao W (2):
> fm10k: enable FTAG based forwarding
> doc: add introduction for fm10k FTAG based forwarding
>
> doc/guides/nics/fm10k.rst | 16 ++++++++-
> doc/guides/rel_notes/release_16_04.rst | 2 ++
> drivers/net/fm10k/fm10k.h | 2 ++
> drivers/net/fm10k/fm10k_ethdev.c | 65
> +++++++++++++++++++++++++++++++++-
> drivers/net/fm10k/fm10k_rxtx.c | 15 ++++++++
> drivers/net/fm10k/fm10k_rxtx_vec.c | 3 ++
> 6 files changed, 101 insertions(+), 2 deletions(-)
>
> --
> 1.9.3
Hi,
We reached a consensus on configuring FTAG by devargs method, any other suggestion
or concern for this patch?
Best Regards,
Xiao
> -----Original Message-----
> From: Liu, Yong
> Sent: Tuesday, March 8, 2016 3:58 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; Chen, Jing D
> <jing.d.chen@intel.com>
> Cc: dev@dpdk.org
> Subject: RE: [dpdk-dev] [PATCH v5 0/2] fm10k: enable FTAG based forwarding
>
> Tested-by: Yong Liu <yong.liu@intel.com>
>
> - Tested Branch: dpdk-next-net/rel_16_04
> - Tested Commit: 4ac366ba647909c3b71818f9be9db86ba5e871da
> - OS: Fedora20 3.11.10-301.fc20.x86_64
> - GCC: gcc version 4.8.3 20140911
> - CPU: Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz
> - NIC: Intel Corporation Device RedrockCanyou [8086:15a4]
> - Default x86_64-native-linuxapp-gcc configuration
> - Prerequisites:
> - Total 1 cases, 1 passed, 0 failed
>
> - Prerequisites command / instruction:
> Apply fm_ftag unit test patch.
> export Port0 and Port1's GLORT ID to environment variables
> export PORT1_GLORT=0x4200
> export PORT0_GLORT=0x4000
>
> - Case: Ftag forwarding unit test
> Description: check fm10k nic can forwarding packets based on FTAG
> Command / instruction:
> Start test application and run fm10k_ftag_autotest
> test -c f -n 4 -w 83:00.0,enable_ftag=1 -w 85:00.0,enable_ftag=1
> RTE>>fm10k_ftag_autotest
> Send packet to Port0 and verify packet forwarded to Port1
> Receive 1 packets on port 0
> test for FTAG RX passed
> Send out 1 packets with FTAG on port 0
> Receive 1 packets on port 1
> test for FTAG TX passed
> Test OK
>
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wang Xiao W
> > Sent: Wednesday, March 02, 2016 7:19 PM
> > To: Chen, Jing D
> > Cc: dev@dpdk.org
> > Subject: [dpdk-dev] [PATCH v5 0/2] fm10k: enable FTAG based forwarding
> >
> > v5:
> > * Used kvargs api to parse the devargs parameter.
> > * Put release note into the driver patch.
> >
> > v4:
> > * Removed the build time config option, used devargs to config FTAG.
> > * Rebased on head of dpdk-next-net/rel_16_04 branch.
> >
> > v3:
> > * Removed "\n" in PMD_INIT_LOG.
> > * Returned "-ENOTSUP" instead of -1 in VF FTAG use case.
> >
> > v2:
> > * Gave an error message for VF FTAG use case.
> > * Added a notice in the doc to emphasize that application should ensure
> > an appropriate FTAG for every frame in FTAG based forwarding mode.
> >
> > Wang Xiao W (2):
> > fm10k: enable FTAG based forwarding
> > doc: add introduction for fm10k FTAG based forwarding
> >
> > doc/guides/nics/fm10k.rst | 16 ++++++++-
> > doc/guides/rel_notes/release_16_04.rst | 2 ++
> > drivers/net/fm10k/fm10k.h | 2 ++
> > drivers/net/fm10k/fm10k_ethdev.c | 65
> > +++++++++++++++++++++++++++++++++-
> > drivers/net/fm10k/fm10k_rxtx.c | 15 ++++++++
> > drivers/net/fm10k/fm10k_rxtx_vec.c | 3 ++
> > 6 files changed, 101 insertions(+), 2 deletions(-)
> >
> > --
> > 1.9.3
On Wed, Mar 02, 2016 at 07:19:12PM +0800, Wang Xiao W wrote:
> v5:
> * Used kvargs api to parse the devargs parameter.
> * Put release note into the driver patch.
>
> v4:
> * Removed the build time config option, used devargs to config FTAG.
> * Rebased on head of dpdk-next-net/rel_16_04 branch.
>
> v3:
> * Removed "\n" in PMD_INIT_LOG.
> * Returned "-ENOTSUP" instead of -1 in VF FTAG use case.
>
> v2:
> * Gave an error message for VF FTAG use case.
> * Added a notice in the doc to emphasize that application should ensure
> an appropriate FTAG for every frame in FTAG based forwarding mode.
>
> Wang Xiao W (2):
> fm10k: enable FTAG based forwarding
> doc: add introduction for fm10k FTAG based forwarding
>
Applied to dpdk-next-net/rel_16_04
/Bruce
@@ -204,6 +204,7 @@ struct fm10k_rx_queue {
uint8_t port_id;
uint8_t drop_en;
uint8_t rx_deferred_start; /* don't start this queue in dev start. */
+ uint16_t rx_ftag_en; /* indicates FTAG RX supported */
};
/*
@@ -240,6 +241,7 @@ struct fm10k_tx_queue {
uint8_t port_id;
uint8_t tx_deferred_start; /** don't start this queue in dev start. */
uint16_t queue_id;
+ uint16_t tx_ftag_en; /* indicates FTAG TX supported */
};
struct fm10k_txq_ops {
@@ -79,6 +79,7 @@ static void fm10k_tx_queue_release(void *queue);
static void fm10k_rx_queue_release(void *queue);
static void fm10k_set_rx_function(struct rte_eth_dev *dev);
static void fm10k_set_tx_function(struct rte_eth_dev *dev);
+static int fm10k_check_ftag(struct rte_devargs *devargs);
struct fm10k_xstats_name_off {
char name[RTE_ETH_XSTATS_NAME_SIZE];
@@ -668,6 +669,18 @@ fm10k_dev_tx_init(struct rte_eth_dev *dev)
PMD_INIT_LOG(ERR, "failed to disable queue %d", i);
return -1;
}
+ /* Enable use of FTAG bit in TX descriptor, PFVTCTL
+ * register is read-only for VF.
+ */
+ if (fm10k_check_ftag(dev->pci_dev->devargs)) {
+ if (hw->mac.type == fm10k_mac_pf)
+ FM10K_WRITE_REG(hw, FM10K_PFVTCTL(i),
+ FM10K_PFVTCTL_FTAG_DESC_ENABLE);
+ else {
+ PMD_INIT_LOG(ERR, "VF FTAG is not supported.");
+ return -ENOTSUP;
+ }
+ }
/* set location and size for descriptor ring */
FM10K_WRITE_REG(hw, FM10K_TDBAL(i),
@@ -2597,15 +2610,32 @@ static const struct eth_dev_ops fm10k_eth_dev_ops = {
.rss_hash_conf_get = fm10k_rss_hash_conf_get,
};
+static int
+fm10k_check_ftag(struct rte_devargs *devargs)
+{
+ if (devargs == NULL)
+ return 0;
+
+ if (strstr(devargs->args, "enable_ftag=1") == NULL)
+ return 0;
+
+ return 1;
+}
+
static void __attribute__((cold))
fm10k_set_tx_function(struct rte_eth_dev *dev)
{
struct fm10k_tx_queue *txq;
int i;
int use_sse = 1;
+ uint16_t tx_ftag_en = 0;
+
+ if (fm10k_check_ftag(dev->pci_dev->devargs))
+ tx_ftag_en = 1;
for (i = 0; i < dev->data->nb_tx_queues; i++) {
txq = dev->data->tx_queues[i];
+ txq->tx_ftag_en = tx_ftag_en;
/* Check if Vector Tx is satisfied */
if (fm10k_tx_vec_condition_check(txq)) {
use_sse = 0;
@@ -2631,11 +2661,16 @@ fm10k_set_rx_function(struct rte_eth_dev *dev)
{
struct fm10k_dev_info *dev_info = FM10K_DEV_PRIVATE_TO_INFO(dev);
uint16_t i, rx_using_sse;
+ uint16_t rx_ftag_en = 0;
+
+ if (fm10k_check_ftag(dev->pci_dev->devargs))
+ rx_ftag_en = 1;
/* In order to allow Vector Rx there are a few configuration
* conditions to be met.
*/
- if (!fm10k_rx_vec_condition_check(dev) && dev_info->rx_vec_allowed) {
+ if (!fm10k_rx_vec_condition_check(dev) &&
+ dev_info->rx_vec_allowed && !rx_ftag_en) {
if (dev->data->scattered_rx)
dev->rx_pkt_burst = fm10k_recv_scattered_pkts_vec;
else
@@ -2658,6 +2693,7 @@ fm10k_set_rx_function(struct rte_eth_dev *dev)
struct fm10k_rx_queue *rxq = dev->data->rx_queues[i];
rxq->rx_using_sse = rx_using_sse;
+ rxq->rx_ftag_en = rx_ftag_en;
}
}
@@ -152,6 +152,12 @@ fm10k_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
*/
mbuf->ol_flags |= PKT_RX_VLAN_PKT;
mbuf->vlan_tci = desc.w.vlan;
+ /**
+ * mbuf->vlan_tci_outer is an idle field in fm10k driver,
+ * so it can be selected to store sglort value.
+ */
+ if (q->rx_ftag_en)
+ mbuf->vlan_tci_outer = rte_le_to_cpu_16(desc.w.sglort);
rx_pkts[count] = mbuf;
if (++next_dd == q->nb_desc) {
@@ -307,6 +313,13 @@ fm10k_recv_scattered_pkts(void *rx_queue, struct rte_mbuf **rx_pkts,
*/
first_seg->ol_flags |= PKT_RX_VLAN_PKT;
first_seg->vlan_tci = desc.w.vlan;
+ /**
+ * mbuf->vlan_tci_outer is an idle field in fm10k driver,
+ * so it can be selected to store sglort value.
+ */
+ if (q->rx_ftag_en)
+ first_seg->vlan_tci_outer =
+ rte_le_to_cpu_16(desc.w.sglort);
/* Prefetch data of first segment, if configured to do so. */
rte_packet_prefetch((char *)first_seg->buf_addr +
@@ -498,6 +511,8 @@ static inline void tx_xmit_pkt(struct fm10k_tx_queue *q, struct rte_mbuf *mb)
q->nb_free -= mb->nb_segs;
q->hw_ring[q->next_free].flags = 0;
+ if (q->tx_ftag_en)
+ q->hw_ring[q->next_free].flags |= FM10K_TXD_FLAG_FTAG;
/* set checksum flags on first descriptor of packet. SCTP checksum
* offload is not supported, but we do not explicitly check for this
* case in favor of greatly simplified processing. */
@@ -239,6 +239,7 @@ fm10k_rx_vec_condition_check(struct rte_eth_dev *dev)
return -1;
return 0;
+
#else
RTE_SET_USED(dev);
return -1;
@@ -688,6 +689,9 @@ fm10k_tx_vec_condition_check(struct fm10k_tx_queue *txq)
if ((txq->txq_flags & FM10K_SIMPLE_TX_FLAG) != FM10K_SIMPLE_TX_FLAG)
return -1;
+ if (txq->tx_ftag_en)
+ return -1;
+
return 0;
}