Message ID | 1410948102-12740-5-git-send-email-bruce.richardson@intel.com (mailing list archive) |
---|---|
State | Superseded, archived |
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id DDDAEB3AA; Wed, 17 Sep 2014 11:56:40 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id EB452B39B for <dev@dpdk.org>; Wed, 17 Sep 2014 11:56:38 +0200 (CEST) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga101.jf.intel.com with ESMTP; 17 Sep 2014 03:02:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.04,540,1406617200"; d="scan'208";a="604080467" Received: from irvmail001.ir.intel.com ([163.33.26.43]) by orsmga002.jf.intel.com with ESMTP; 17 Sep 2014 03:01:43 -0700 Received: from sivswdev02.ir.intel.com (sivswdev02.ir.intel.com [10.237.217.46]) by irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id s8HA1g0r017179; Wed, 17 Sep 2014 11:01:42 +0100 Received: from sivswdev02.ir.intel.com (localhost [127.0.0.1]) by sivswdev02.ir.intel.com with ESMTP id s8HA1gRW015704; Wed, 17 Sep 2014 11:01:42 +0100 Received: (from bricha3@localhost) by sivswdev02.ir.intel.com with id s8HA1gJK015698; Wed, 17 Sep 2014 11:01:42 +0100 From: Bruce Richardson <bruce.richardson@intel.com> To: dev@dpdk.org Date: Wed, 17 Sep 2014 11:01:41 +0100 Message-Id: <1410948102-12740-5-git-send-email-bruce.richardson@intel.com> X-Mailer: git-send-email 1.7.4.1 In-Reply-To: <1410948102-12740-1-git-send-email-bruce.richardson@intel.com> References: <1410948102-12740-1-git-send-email-bruce.richardson@intel.com> Subject: [dpdk-dev] [PATCH 4/5] mbuf: add userdata pointer field X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Commit Message
Bruce Richardson
Sept. 17, 2014, 10:01 a.m. UTC
While some applications may store metadata about packets in the packet
mbuf headroom, this is not a workable solution for packet metadata which
is either:
* larger than the headroom (or headroom is needed for adding pkt headers)
* needs to be shared or copied among packets
To support these use cases in applications, we reserve a general
"userdata" pointer field inside the second cache-line of the mbuf. This
is better than having the application store the pointer to the external
metadata in the packet headroom, as it saves an additional cache-line
from being used.
Apart from storing metadata, this field also provides a general 8-byte
scratch space inside the mbuf for any other application uses that are
applicable.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h | 3 ++-
lib/librte_mbuf/rte_mbuf.h | 3 +++
2 files changed, 5 insertions(+), 1 deletion(-)
Comments
On Wed, Sep 17, 2014 at 11:01:41AM +0100, Bruce Richardson wrote: > While some applications may store metadata about packets in the packet > mbuf headroom, this is not a workable solution for packet metadata which > is either: > * larger than the headroom (or headroom is needed for adding pkt headers) > * needs to be shared or copied among packets > > To support these use cases in applications, we reserve a general > "userdata" pointer field inside the second cache-line of the mbuf. This > is better than having the application store the pointer to the external > metadata in the packet headroom, as it saves an additional cache-line > from being used. > > Apart from storing metadata, this field also provides a general 8-byte > scratch space inside the mbuf for any other application uses that are > applicable. > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> > --- > lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h | 3 ++- > lib/librte_mbuf/rte_mbuf.h | 3 +++ > 2 files changed, 5 insertions(+), 1 deletion(-) > > diff --git a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > index 25ed672..d27e891 100644 > --- a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > +++ b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > @@ -117,7 +117,8 @@ struct rte_kni_mbuf { > uint16_t data_len; /**< Amount of data in segment buffer. */ > uint32_t pkt_len; /**< Total pkt len: sum of all segment data_len. */ > char pad3[8]; > - void *pool __attribute__((__aligned__(64))); > + void *pad4 __attribute__((__aligned__(64))); > + void *pool; I don't see a comment about this in the changelog, only about the userdata pointer being added below. > void *next; > }; > > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h > index 8e27d2e..b1acfc3 100644 > --- a/lib/librte_mbuf/rte_mbuf.h > +++ b/lib/librte_mbuf/rte_mbuf.h > @@ -172,6 +172,9 @@ struct rte_mbuf { > > /* second cache line - fields only used in slow path or on TX */ > MARKER cacheline1 __rte_cache_aligned; > + > + void *userdata; /**< Can be used for external metadata */ > + Do you want to make this a void* or a char[8]? I ask because if people are going to use is as a scratch space (rather than a pointer), they get a suprise when they build this on 32 bit systems, and their 8 byte scratch space is reduced to 4 bytes. Neil > struct rte_mempool *pool; /**< Pool from which mbuf was allocated. */ > struct rte_mbuf *next; /**< Next segment of scattered packet. */ > > -- > 1.9.3 > >
> -----Original Message----- > From: Neil Horman [mailto:nhorman@tuxdriver.com] > Sent: Wednesday, September 17, 2014 4:35 PM > To: Richardson, Bruce > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH 4/5] mbuf: add userdata pointer field > > On Wed, Sep 17, 2014 at 11:01:41AM +0100, Bruce Richardson wrote: > > While some applications may store metadata about packets in the packet > > mbuf headroom, this is not a workable solution for packet metadata which > > is either: > > * larger than the headroom (or headroom is needed for adding pkt headers) > > * needs to be shared or copied among packets > > > > To support these use cases in applications, we reserve a general > > "userdata" pointer field inside the second cache-line of the mbuf. This > > is better than having the application store the pointer to the external > > metadata in the packet headroom, as it saves an additional cache-line > > from being used. > > > > Apart from storing metadata, this field also provides a general 8-byte > > scratch space inside the mbuf for any other application uses that are > > applicable. > > > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> > > --- > > lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h | 3 ++- > > lib/librte_mbuf/rte_mbuf.h | 3 +++ > > 2 files changed, 5 insertions(+), 1 deletion(-) > > > > diff --git a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > > index 25ed672..d27e891 100644 > > --- a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > > +++ b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > > @@ -117,7 +117,8 @@ struct rte_kni_mbuf { > > uint16_t data_len; /**< Amount of data in segment buffer. */ > > uint32_t pkt_len; /**< Total pkt len: sum of all segment data_len. */ > > char pad3[8]; > > - void *pool __attribute__((__aligned__(64))); > > + void *pad4 __attribute__((__aligned__(64))); > > + void *pool; > I don't see a comment about this in the changelog, only about the userdata > pointer being added below. Yes, this is the userdata pointer - just added as padding here, since it's not actually needed by the kernel-side KNI module. > > > > void *next; > > }; > > > > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h > > index 8e27d2e..b1acfc3 100644 > > --- a/lib/librte_mbuf/rte_mbuf.h > > +++ b/lib/librte_mbuf/rte_mbuf.h > > @@ -172,6 +172,9 @@ struct rte_mbuf { > > > > /* second cache line - fields only used in slow path or on TX */ > > MARKER cacheline1 __rte_cache_aligned; > > + > > + void *userdata; /**< Can be used for external metadata */ > > + > Do you want to make this a void* or a char[8]? I ask because if people are > going to use is as a scratch space (rather than a pointer), they get a suprise > when they build this on 32 bit systems, and their 8 byte scratch space is > reduced to 4 bytes. I think this is better as a pointer, as that is how it is likely to be used. As for 32-bit, I'm torn between wanting to just update the comment and feeling the need to update the code to actually make the thing 8-byte on 32-bit! Changing the comment to be more accurate is easier, unions are ugly looking in the structure IMHO, so maybe I'll just mark the following field (pool) as always 8-byte aligned.... /Bruce > > Neil > > > struct rte_mempool *pool; /**< Pool from which mbuf was allocated. > */ > > struct rte_mbuf *next; /**< Next segment of scattered packet. */ > > > > -- > > 1.9.3 > > > >
On Wed, Sep 17, 2014 at 04:02:07PM +0000, Richardson, Bruce wrote: > > -----Original Message----- > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > Sent: Wednesday, September 17, 2014 4:35 PM > > To: Richardson, Bruce > > Cc: dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH 4/5] mbuf: add userdata pointer field > > > > On Wed, Sep 17, 2014 at 11:01:41AM +0100, Bruce Richardson wrote: > > > While some applications may store metadata about packets in the packet > > > mbuf headroom, this is not a workable solution for packet metadata which > > > is either: > > > * larger than the headroom (or headroom is needed for adding pkt headers) > > > * needs to be shared or copied among packets > > > > > > To support these use cases in applications, we reserve a general > > > "userdata" pointer field inside the second cache-line of the mbuf. This > > > is better than having the application store the pointer to the external > > > metadata in the packet headroom, as it saves an additional cache-line > > > from being used. > > > > > > Apart from storing metadata, this field also provides a general 8-byte > > > scratch space inside the mbuf for any other application uses that are > > > applicable. > > > > > > Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> > > > --- > > > lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h | 3 ++- > > > lib/librte_mbuf/rte_mbuf.h | 3 +++ > > > 2 files changed, 5 insertions(+), 1 deletion(-) > > > > > > diff --git a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > > b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > > > index 25ed672..d27e891 100644 > > > --- a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > > > +++ b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h > > > @@ -117,7 +117,8 @@ struct rte_kni_mbuf { > > > uint16_t data_len; /**< Amount of data in segment buffer. */ > > > uint32_t pkt_len; /**< Total pkt len: sum of all segment data_len. */ > > > char pad3[8]; > > > - void *pool __attribute__((__aligned__(64))); > > > + void *pad4 __attribute__((__aligned__(64))); > > > + void *pool; > > I don't see a comment about this in the changelog, only about the userdata > > pointer being added below. > > Yes, this is the userdata pointer - just added as padding here, since it's not actually needed by the kernel-side KNI module. > Ah, then maybe merge it with the pad3 pointer, to make it look a bit better? > > > > > > > void *next; > > > }; > > > > > > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h > > > index 8e27d2e..b1acfc3 100644 > > > --- a/lib/librte_mbuf/rte_mbuf.h > > > +++ b/lib/librte_mbuf/rte_mbuf.h > > > @@ -172,6 +172,9 @@ struct rte_mbuf { > > > > > > /* second cache line - fields only used in slow path or on TX */ > > > MARKER cacheline1 __rte_cache_aligned; > > > + > > > + void *userdata; /**< Can be used for external metadata */ > > > + > > Do you want to make this a void* or a char[8]? I ask because if people are > > going to use is as a scratch space (rather than a pointer), they get a suprise > > when they build this on 32 bit systems, and their 8 byte scratch space is > > reduced to 4 bytes. > > I think this is better as a pointer, as that is how it is likely to be used. As for 32-bit, I'm torn between wanting to just update the comment and feeling the need to update the code to actually make the thing 8-byte on 32-bit! Changing the comment to be more accurate is easier, unions are ugly looking in the structure IMHO, so maybe I'll just mark the following field (pool) as always 8-byte aligned.... > Ok, however you want to bring it into alignment, as long as its clear that its either 8 bytes always, or a variable size based on the arch. Best Neil > /Bruce > > > > > Neil > > > > > struct rte_mempool *pool; /**< Pool from which mbuf was allocated. > > */ > > > struct rte_mbuf *next; /**< Next segment of scattered packet. */ > > > > > > -- > > > 1.9.3 > > > > > > >
diff --git a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h index 25ed672..d27e891 100644 --- a/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h +++ b/lib/librte_eal/linuxapp/eal/include/exec-env/rte_kni_common.h @@ -117,7 +117,8 @@ struct rte_kni_mbuf { uint16_t data_len; /**< Amount of data in segment buffer. */ uint32_t pkt_len; /**< Total pkt len: sum of all segment data_len. */ char pad3[8]; - void *pool __attribute__((__aligned__(64))); + void *pad4 __attribute__((__aligned__(64))); + void *pool; void *next; }; diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h index 8e27d2e..b1acfc3 100644 --- a/lib/librte_mbuf/rte_mbuf.h +++ b/lib/librte_mbuf/rte_mbuf.h @@ -172,6 +172,9 @@ struct rte_mbuf { /* second cache line - fields only used in slow path or on TX */ MARKER cacheline1 __rte_cache_aligned; + + void *userdata; /**< Can be used for external metadata */ + struct rte_mempool *pool; /**< Pool from which mbuf was allocated. */ struct rte_mbuf *next; /**< Next segment of scattered packet. */