Message ID | 1468542971-48198-1-git-send-email-yongwang@vmware.com (mailing list archive) |
---|---|
State | Rejected, archived |
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id B0E4147CE; Fri, 15 Jul 2016 02:52:53 +0200 (CEST) Received: from EX13-EDG-OU-001.vmware.com (ex13-edg-ou-001.vmware.com [208.91.0.189]) by dpdk.org (Postfix) with ESMTP id D3EBA3990 for <dev@dpdk.org>; Fri, 15 Jul 2016 02:52:51 +0200 (CEST) Received: from sc9-mailhost1.vmware.com (10.113.161.71) by EX13-EDG-OU-001.vmware.com (10.113.208.155) with Microsoft SMTP Server id 15.0.1156.6; Thu, 14 Jul 2016 17:52:23 -0700 Received: from prmh-edge-ivybridge-01.eng.vmware.com (unknown [10.24.235.72]) by sc9-mailhost1.vmware.com (Postfix) with ESMTP id E88DB18540; Thu, 14 Jul 2016 17:52:49 -0700 (PDT) From: Yong Wang <yongwang@vmware.com> To: <david.marchand@6wind.com> CC: <dev@dpdk.org>, Yong Wang <yongwang@vmware.com> Date: Thu, 14 Jul 2016 17:36:11 -0700 Message-ID: <1468542971-48198-1-git-send-email-yongwang@vmware.com> X-Mailer: git-send-email 1.9.1 MIME-Version: 1.0 Content-Type: text/plain Received-SPF: None (EX13-EDG-OU-001.vmware.com: yongwang@vmware.com does not designate permitted sender hosts) Subject: [dpdk-dev] [PATCH] eal: fix rte_intr_dp_is_en() check X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Commit Message
Yong Wang
July 15, 2016, 12:36 a.m. UTC
When binding a device to igb_uio with intr_conf.rxq set to 1, nb_efd
is 1 (for link event) but rte_intr_dp_is_en() will still return true.
rte_intr_dp_is_en() should also consider intr_handle type in addition
to nb_efd.
Signed-off-by: Yong Wang <yongwang@vmware.com>
---
lib/librte_eal/linuxapp/eal/eal_interrupts.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
Comments
Hi Yong, rte_intr_dp_is_en() returns true when rte_intr_efd_enable() (the way to enable data-path interrupt) sets a number of event fds. In this case, "intr_conf.rxq=1" configuration causes "nb_efd=1". The value comes from RTE_MIN($nb_efd, 1) from data-path, but not from link event. Per link event, you shouldn't use rte_intr_dp_is_en() as the indication. As igb_uio only has a single vector, when the conflict(both intr_rxq and intr_lsc turn on) happens, the intr_rxq has high priority than intr_lsc as default PMD behavior. Reference as PG 3.1.9 note in http://dpdk.org/doc/guides/prog_guide/env_abstraction_layer.html Regards, Cunming On 7/15/2016 8:36 AM, Yong Wang wrote: > When binding a device to igb_uio with intr_conf.rxq set to 1, nb_efd > is 1 (for link event) but rte_intr_dp_is_en() will still return true. > rte_intr_dp_is_en() should also consider intr_handle type in addition > to nb_efd. > > Signed-off-by: Yong Wang <yongwang@vmware.com> > --- > lib/librte_eal/linuxapp/eal/eal_interrupts.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/lib/librte_eal/linuxapp/eal/eal_interrupts.c b/lib/librte_eal/linuxapp/eal/eal_interrupts.c > index 47a3b20..71f63e9 100644 > --- a/lib/librte_eal/linuxapp/eal/eal_interrupts.c > +++ b/lib/librte_eal/linuxapp/eal/eal_interrupts.c > @@ -1200,7 +1200,8 @@ rte_intr_efd_disable(struct rte_intr_handle *intr_handle) > int > rte_intr_dp_is_en(struct rte_intr_handle *intr_handle) > { > - return !(!intr_handle->nb_efd); > + return (!(!intr_handle->nb_efd) && > + (intr_handle->type == RTE_INTR_HANDLE_VFIO_MSIX)); > } > > int
Hi Yong, I think the interrupt management should be simpler. If you want to invest some time to rework this API, you are very welcome. 2016-07-18 14:19, Liang, Cunming: > Hi Yong, > > rte_intr_dp_is_en() returns true when rte_intr_efd_enable() (the way to > enable data-path interrupt) sets a number of event fds. > In this case, "intr_conf.rxq=1" configuration causes "nb_efd=1". The > value comes from RTE_MIN($nb_efd, 1) from data-path, but not from link > event. > Per link event, you shouldn't use rte_intr_dp_is_en() as the indication. > As igb_uio only has a single vector, when the conflict(both intr_rxq and > intr_lsc turn on) happens, the intr_rxq has high priority than intr_lsc > as default PMD behavior. > Reference as PG 3.1.9 note in > http://dpdk.org/doc/guides/prog_guide/env_abstraction_layer.html > > Regards, > Cunming > > On 7/15/2016 8:36 AM, Yong Wang wrote: > > When binding a device to igb_uio with intr_conf.rxq set to 1, nb_efd > > is 1 (for link event) but rte_intr_dp_is_en() will still return true. > > rte_intr_dp_is_en() should also consider intr_handle type in addition > > to nb_efd. > > > > Signed-off-by: Yong Wang <yongwang@vmware.com>
> On Jul 18, 2016, at 2:21 AM, Thomas Monjalon <thomas.monjalon@6wind.com> wrote: > > Hi Yong, > > I think the interrupt management should be simpler. > If you want to invest some time to rework this API, you > are very welcome. > > > 2016-07-18 14:19, Liang, Cunming: >> Hi Yong, >> >> rte_intr_dp_is_en() returns true when rte_intr_efd_enable() (the way to >> enable data-path interrupt) sets a number of event fds. >> In this case, "intr_conf.rxq=1" configuration causes "nb_efd=1". The >> value comes from RTE_MIN($nb_efd, 1) from data-path, but not from link >> event. >> Per link event, you shouldn't use rte_intr_dp_is_en() as the indication. >> As igb_uio only has a single vector, when the conflict(both intr_rxq and >> intr_lsc turn on) happens, the intr_rxq has high priority than intr_lsc >> as default PMD behavior. >> Reference as PG 3.1.9 note in >> https://urldefense.proofpoint.com/v2/url?u=http-3A__dpdk.org_doc_guides_prog-5Fguide_env-5Fabstraction-5Flayer.html&d=CwICAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=44mSO5N5yEs4CeCdtQE0xt0F7J0p67_mApYVAzyYms0&m=xFO005GdZMA9a6IC2kvhKYaXW7Rhl3dlcom5HfOSy2g&s=yERFYJTe6TBWFCltmR3xikmxkLKutbot5BfFxK30tZ0&e= >> >> Regards, >> Cunning Thanks Cunming and Thomas for the feedback. I agree with Thomas that the rx queue interrupt APIs could be simplified and it’s not clear to me if the API user needs to be concerned about rte_intr_cap_multiple(), rte_intr_allow_others(), rte_intr_dp_is_en(), etc. if all he needs is to enable intr with certain # of vectors and map the vectors to either efds or callbacks. I feel it’s simpler just returning the number of vectors the system can support if it cannot satisfy nb_efd vectors and let the application decide how should this be handled. The link event and rxq interrupt init/uninit path also looks quite different and some unification will be helpful. The API documentation can also use some more clarification. For example, without reading the program guide, it’s easy to miss the fact that when the device is bound to UIO, the implementation will try to enable rxq instead of lsc when both intr_conf.lsc and intr_conf.rxq is set to 1. I can imagine there are cases where an application would prefer to enable link event instead of rxq interrupt in such cases and currently there is no easy way to achieve that. Or is there any particular reason such a preference is chosen? >> >> On 7/15/2016 8:36 AM, Yong Wang wrote: >>> When binding a device to igb_uio with intr_conf.rxq set to 1, nb_efd >>> is 1 (for link event) but rte_intr_dp_is_en() will still return true. >>> rte_intr_dp_is_en() should also consider intr_handle type in addition >>> to nb_efd. >>> >>> Signed-off-by: Yong Wang <yongwang@vmware.com> >
Hi Yong, > -----Original Message----- > From: Yong Wang [mailto:yongwang@vmware.com] > Sent: Wednesday, July 20, 2016 5:59 AM > To: Thomas Monjalon <thomas.monjalon@6wind.com> > Cc: dev@dpdk.org; Liang, Cunming <cunming.liang@intel.com>; > david.marchand@6wind.com > Subject: Re: [dpdk-dev] [PATCH] eal: fix rte_intr_dp_is_en() check > > > > On Jul 18, 2016, at 2:21 AM, Thomas Monjalon <thomas.monjalon@6wind.com> > wrote: > > > > Hi Yong, > > > > I think the interrupt management should be simpler. > > If you want to invest some time to rework this API, you > > are very welcome. > > > > > > 2016-07-18 14:19, Liang, Cunming: > >> Hi Yong, > >> > >> rte_intr_dp_is_en() returns true when rte_intr_efd_enable() (the way to > >> enable data-path interrupt) sets a number of event fds. > >> In this case, "intr_conf.rxq=1" configuration causes "nb_efd=1". The > >> value comes from RTE_MIN($nb_efd, 1) from data-path, but not from link > >> event. > >> Per link event, you shouldn't use rte_intr_dp_is_en() as the indication. > >> As igb_uio only has a single vector, when the conflict(both intr_rxq and > >> intr_lsc turn on) happens, the intr_rxq has high priority than intr_lsc > >> as default PMD behavior. > >> Reference as PG 3.1.9 note in > >> https://urldefense.proofpoint.com/v2/url?u=http- > 3A__dpdk.org_doc_guides_prog-5Fguide_env-5Fabstraction- > 5Flayer.html&d=CwICAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt- > uEs&r=44mSO5N5yEs4CeCdtQE0xt0F7J0p67_mApYVAzyYms0&m=xFO005GdZMA > 9a6IC2kvhKYaXW7Rhl3dlcom5HfOSy2g&s=yERFYJTe6TBWFCltmR3xikmxkLKutbot > 5BfFxK30tZ0&e= > >> > >> Regards, > >> Cunning > > Thanks Cunming and Thomas for the feedback. I agree with Thomas that the rx > queue interrupt APIs could be simplified and it’s not clear to me if the API user > needs to be concerned about rte_intr_cap_multiple(), rte_intr_allow_others(), > rte_intr_dp_is_en(), etc. if all he needs is to enable intr with certain # of vectors > and map the vectors to either efds or callbacks. I feel it’s simpler just returning > the number of vectors the system can support if it cannot satisfy nb_efd vectors > and let the application decide how should this be handled. It's welcome to make a simpler intr API. At the beginning of introducing rxq intr, it didn't attract much attention, the coordination problem of EAL intr thread remains. Status quo: - The 3 APIs you talked is the helper function for PMD developer, not for end user. EAL provides efds/vectors mapping, hides vfio/uio difference. The causes/vectors mapping is handled by PMD. Then it's not necessary for the end user to aware intr vector, just take aware of event by queue id. Your idea on exposing vectors to end user is an option. In terms of it, the APP has to understand the causes of vectors, misc(e.g. lsc, mb, etc) vs non-misc(rxq). In particular case(e.g. mb), it seems APP is not the right person to turn on/off. If all are acceptable, ethdev can provide new API for causes to vector mapping. There's one note for replacing the 3 API provided to PMD. PMD is blind to the user space IO infrastructure, vfio or uio. The way for PMD to configure mapping for legacy/MSI(e.g. uio case) and MSIX(e.g. vfio case) isn't the same. >The link event and rxq > interrupt init/uninit path also looks quite different and some unification will be > helpful. - 'intr_conf' and interrupt thread for misc To turn on lsc intr, it requires 'intr_conf.lsc=1'. It cases ethdev to register handler for further processing in the interrupt thread. It's not necessary for rxq intr. Probably a clean way is not to handle device external interrupt event in EAL interrupt thread (intr mb may have some problem). The EAL interrupt thread is only used to postpone the delay execution or other background interrupt (e.g. alarm). Then misc/non-misc can be combined, while requiring APP to detect the interrupt causes. > > The API documentation can also use some more clarification. For example, > without reading the program guide, it’s easy to miss the fact that when the > device is bound to UIO, the implementation will try to enable rxq instead of lsc > when both intr_conf.lsc and intr_conf.rxq is set to 1. I can imagine there are cases > where an application would prefer to enable link event instead of rxq interrupt in > such cases and currently there is no easy way to achieve that. Or is there any > particular reason such a preference is chosen? In uio, if do care lsc. The suggestion is to only turn on intr_conf.lsc. It's just a default behavior definition for the single vector tradeoff. Which don't want to introduce another configuration. The uio msix patch was expected to upstream on that time... The uio single vector problem probably will remain for a long time. So welcome to rework these stuffs. Thanks, Cunming > > >> > >> On 7/15/2016 8:36 AM, Yong Wang wrote: > >>> When binding a device to igb_uio with intr_conf.rxq set to 1, nb_efd > >>> is 1 (for link event) but rte_intr_dp_is_en() will still return true. > >>> rte_intr_dp_is_en() should also consider intr_handle type in addition > >>> to nb_efd. > >>> > >>> Signed-off-by: Yong Wang <yongwang@vmware.com> > >
2016-07-20 07:03, Liang, Cunming: > Probably a clean way is not to handle device external interrupt event in EAL > interrupt thread (intr mb may have some problem). The EAL interrupt thread is > only used to postpone the delay execution or other background interrupt > (e.g. alarm). Then misc/non-misc can be combined, while requiring APP to > detect the interrupt causes. I am not sure it was a good idea to have a thread for the link interrupt. It may be simpler and cleaner to let the application do the pthread_create.
Hi Thomas, > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Wednesday, July 20, 2016 4:42 PM > To: Liang, Cunming <cunming.liang@intel.com> > Cc: Yong Wang <yongwang@vmware.com>; dev@dpdk.org; > david.marchand@6wind.com > Subject: Re: [dpdk-dev] [PATCH] eal: fix rte_intr_dp_is_en() check > > 2016-07-20 07:03, Liang, Cunming: > > Probably a clean way is not to handle device external interrupt event in EAL > > interrupt thread (intr mb may have some problem). The EAL interrupt thread > is > > only used to postpone the delay execution or other background interrupt > > (e.g. alarm). Then misc/non-misc can be combined, while requiring APP to > > detect the interrupt causes. > > I am not sure it was a good idea to have a thread for the link interrupt. > It may be simpler and cleaner to let the application do the pthread_create. The EAL intr thread is reserved for all interrupt before, is not design for link interrupt on the first day. Personally I vote to remove link interrupt from interrupt thread. -Cunming
diff --git a/lib/librte_eal/linuxapp/eal/eal_interrupts.c b/lib/librte_eal/linuxapp/eal/eal_interrupts.c index 47a3b20..71f63e9 100644 --- a/lib/librte_eal/linuxapp/eal/eal_interrupts.c +++ b/lib/librte_eal/linuxapp/eal/eal_interrupts.c @@ -1200,7 +1200,8 @@ rte_intr_efd_disable(struct rte_intr_handle *intr_handle) int rte_intr_dp_is_en(struct rte_intr_handle *intr_handle) { - return !(!intr_handle->nb_efd); + return (!(!intr_handle->nb_efd) && + (intr_handle->type == RTE_INTR_HANDLE_VFIO_MSIX)); } int