[0/2] add lock-free mode for l3fwd

  Lock-free mode is supported by hash library and LPM library.
Now we add an option for l3fwd example to enable the lock-free mode.
Necessary preparation steps are added to use lock-free LPM mode.

Patch 2/2 has dependency on RCU QSBR integration with LPM library:
http://patches.dpdk.org/project/dpdk/list/?series=6288


Ruifeng Wang (2):
  examples/l3fwd: add lock-free option for l3fwd
  examples/l3fwd: integrate RCU QSBR for LPM mode

 doc/guides/sample_app_ug/l3_forward.rst |  3 ++
 examples/l3fwd/Makefile                 |  1 +
 examples/l3fwd/l3fwd.h                  |  4 +-
 examples/l3fwd/l3fwd_em.c               | 10 +++-
 examples/l3fwd/l3fwd_lpm.c              | 72 +++++++++++++++++++++++--
 examples/l3fwd/main.c                   | 27 ++++++++--
 examples/l3fwd/meson.build              |  1 +
 7 files changed, 108 insertions(+), 10 deletions(-)
  

Hi,

> 
> Lock-free mode is supported by hash library and LPM library.
> Now we add an option for l3fwd example to enable the lock-free mode.
> Necessary preparation steps are added to use lock-free LPM mode.

Can I ask about the purpose of these changes?
Right now in  l3fwd both lpm and hash tables are static and hard-coded.
we initialize them at startup and then just do read from them.
Do you plan to enhance l3fwd with ability to dynamically update tables contents?
Though fir that we first have to get rid of hard-coded values (config file or so).
Konstantin

> 
> Patch 2/2 has dependency on RCU QSBR integration with LPM library:
> http://patches.dpdk.org/project/dpdk/list/?series=6288
> 
> 
> Ruifeng Wang (2):
>   examples/l3fwd: add lock-free option for l3fwd
>   examples/l3fwd: integrate RCU QSBR for LPM mode
> 
>  doc/guides/sample_app_ug/l3_forward.rst |  3 ++
>  examples/l3fwd/Makefile                 |  1 +
>  examples/l3fwd/l3fwd.h                  |  4 +-
>  examples/l3fwd/l3fwd_em.c               | 10 +++-
>  examples/l3fwd/l3fwd_lpm.c              | 72 +++++++++++++++++++++++--
>  examples/l3fwd/main.c                   | 27 ++++++++--
>  examples/l3fwd/meson.build              |  1 +
>  7 files changed, 108 insertions(+), 10 deletions(-)
> 
> --
> 2.17.1
  

On Fri,  6 Sep 2019 18:26:13 +0800
Ruifeng Wang <ruifeng.wang@arm.com> wrote:

> Lock-free mode is supported by hash library and LPM library.
> Now we add an option for l3fwd example to enable the lock-free mode.
> Necessary preparation steps are added to use lock-free LPM mode.

If lock-free mode works it should just do that.
Having options mean that there are two test cases; which inevitably
leads to one of them being broken.
  

Hi,

> -----Original Message-----
> From: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> Sent: Friday, September 6, 2019 18:35
> To: Ruifeng Wang (Arm Technology China) <Ruifeng.Wang@arm.com>;
> Kantecki, Tomasz <tomasz.kantecki@intel.com>
> Cc: dev@dpdk.org; Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>;
> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd
> 
> Hi,
> 
> >
> > Lock-free mode is supported by hash library and LPM library.
> > Now we add an option for l3fwd example to enable the lock-free mode.
> > Necessary preparation steps are added to use lock-free LPM mode.
> 
> Can I ask about the purpose of these changes?
> Right now in  l3fwd both lpm and hash tables are static and hard-coded.
> we initialize them at startup and then just do read from them.
> Do you plan to enhance l3fwd with ability to dynamically update tables
> contents?
> Though fir that we first have to get rid of hard-coded values (config file or so).
> Konstantin
> 
Thanks for your questions.
Currently, we have no plan to enhance l3fwd with ability to dynamically update table contents.
Lock-free method is being integrated into Hash library and LPM library.  Lock-free algorithms
are not only about control plane (adding or deleting routes), they affect the data path performance
as well.
Since l3fwd application is showcasing data path performance, we need to show the impact of
including the quiescent state reporting on data path.
This change also serves as an example of using the RCU APIs.

> >
> > Patch 2/2 has dependency on RCU QSBR integration with LPM library:
> > http://patches.dpdk.org/project/dpdk/list/?series=6288
> >
> >
> > Ruifeng Wang (2):
> >   examples/l3fwd: add lock-free option for l3fwd
> >   examples/l3fwd: integrate RCU QSBR for LPM mode
> >
> >  doc/guides/sample_app_ug/l3_forward.rst |  3 ++
> >  examples/l3fwd/Makefile                 |  1 +
> >  examples/l3fwd/l3fwd.h                  |  4 +-
> >  examples/l3fwd/l3fwd_em.c               | 10 +++-
> >  examples/l3fwd/l3fwd_lpm.c              | 72 +++++++++++++++++++++++--
> >  examples/l3fwd/main.c                   | 27 ++++++++--
> >  examples/l3fwd/meson.build              |  1 +
> >  7 files changed, 108 insertions(+), 10 deletions(-)
> >
> > --
> > 2.17.1
  

> -----Original Message-----
> From: Stephen Hemminger <stephen@networkplumber.org>
> Sent: Saturday, September 7, 2019 01:29
> To: Ruifeng Wang (Arm Technology China) <Ruifeng.Wang@arm.com>
> Cc: tomasz.kantecki@intel.com; dev@dpdk.org; Gavin Hu (Arm Technology
> China) <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd
> 
> On Fri,  6 Sep 2019 18:26:13 +0800
> Ruifeng Wang <ruifeng.wang@arm.com> wrote:
> 
> > Lock-free mode is supported by hash library and LPM library.
> > Now we add an option for l3fwd example to enable the lock-free mode.
> > Necessary preparation steps are added to use lock-free LPM mode.
> 
> If lock-free mode works it should just do that.
> Having options mean that there are two test cases; which inevitably leads to
> one of them being broken.

Agree that having options will add scenarios that being tested.
Since these different scenarios are supported by Hash / LPM library, the tests on
them should be valid. 
As l3fwd application is always used to benchmark data path performance, make
both supported modes available can help user to easily collect data and compare.
In the long run, we can make lock-free mode the default used by l3fwd when it
is fine tuned.
  

<snip>

> >
> > Hi,
> >
> > >
> > > Lock-free mode is supported by hash library and LPM library.
> > > Now we add an option for l3fwd example to enable the lock-free mode.
> > > Necessary preparation steps are added to use lock-free LPM mode.
> >
> > Can I ask about the purpose of these changes?
> > Right now in  l3fwd both lpm and hash tables are static and hard-coded.
> > we initialize them at startup and then just do read from them.
> > Do you plan to enhance l3fwd with ability to dynamically update tables
> > contents?
> > Though fir that we first have to get rid of hard-coded values (config file or
> so).
> > Konstantin
> >
> Thanks for your questions.
> Currently, we have no plan to enhance l3fwd with ability to dynamically
> update table contents.
> Lock-free method is being integrated into Hash library and LPM library.  Lock-
> free algorithms are not only about control plane (adding or deleting routes),
> they affect the data path performance as well.
> Since l3fwd application is showcasing data path performance, we need to
> show the impact of including the quiescent state reporting on data path.
> This change also serves as an example of using the RCU APIs.
> 
Without the dynamic deletes the quiescent state reporting overhead is not captured completely.
I suggest that we add and delete a small set of unrelated routes (the routes that are not used currently) on a regular basis.

> > >
> > > Patch 2/2 has dependency on RCU QSBR integration with LPM library:
> > > http://patches.dpdk.org/project/dpdk/list/?series=6288
> > >
> > >
> > > Ruifeng Wang (2):
> > >   examples/l3fwd: add lock-free option for l3fwd
> > >   examples/l3fwd: integrate RCU QSBR for LPM mode
> > >
> > >  doc/guides/sample_app_ug/l3_forward.rst |  3 ++
> > >  examples/l3fwd/Makefile                 |  1 +
> > >  examples/l3fwd/l3fwd.h                  |  4 +-
> > >  examples/l3fwd/l3fwd_em.c               | 10 +++-
> > >  examples/l3fwd/l3fwd_lpm.c              | 72 +++++++++++++++++++++++--
> > >  examples/l3fwd/main.c                   | 27 ++++++++--
> > >  examples/l3fwd/meson.build              |  1 +
> > >  7 files changed, 108 insertions(+), 10 deletions(-)
> > >
> > > --
> > > 2.17.1
>
  

> -----Original Message-----
> From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> Sent: Tuesday, September 10, 2019 06:45
> To: Ruifeng Wang (Arm Technology China) <Ruifeng.Wang@arm.com>;
> Ananyev, Konstantin <konstantin.ananyev@intel.com>; Kantecki, Tomasz
> <tomasz.kantecki@intel.com>
> Cc: dev@dpdk.org; Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>;
> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd
> <nd@arm.com>; nd <nd@arm.com>
> Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd
> 
> <snip>
> 
> > >
> > > Hi,
> > >
> > > >
> > > > Lock-free mode is supported by hash library and LPM library.
> > > > Now we add an option for l3fwd example to enable the lock-free mode.
> > > > Necessary preparation steps are added to use lock-free LPM mode.
> > >
> > > Can I ask about the purpose of these changes?
> > > Right now in  l3fwd both lpm and hash tables are static and hard-coded.
> > > we initialize them at startup and then just do read from them.
> > > Do you plan to enhance l3fwd with ability to dynamically update
> > > tables contents?
> > > Though fir that we first have to get rid of hard-coded values
> > > (config file or
> > so).
> > > Konstantin
> > >
> > Thanks for your questions.
> > Currently, we have no plan to enhance l3fwd with ability to
> > dynamically update table contents.
> > Lock-free method is being integrated into Hash library and LPM
> > library.  Lock- free algorithms are not only about control plane
> > (adding or deleting routes), they affect the data path performance as well.
> > Since l3fwd application is showcasing data path performance, we need
> > to show the impact of including the quiescent state reporting on data path.
> > This change also serves as an example of using the RCU APIs.
> >
> Without the dynamic deletes the quiescent state reporting overhead is not
> captured completely.
> I suggest that we add and delete a small set of unrelated routes (the routes
> that are not used currently) on a regular basis.
> 
Add and delete unrelated routes on a regular basis will simulate overhead on
control path. However, control path performance is not a l3fwd showcase, and
it is covered by LPM performance unit test.
On data path, quiescent state reporting overhead is constant (calling of rte_rcu_qsbr_quiescent).
It will not be impacted by route addition / deletion. 

> > > >
> > > > Patch 2/2 has dependency on RCU QSBR integration with LPM library:
> > > > http://patches.dpdk.org/project/dpdk/list/?series=6288
> > > >
> > > >
> > > > Ruifeng Wang (2):
> > > >   examples/l3fwd: add lock-free option for l3fwd
> > > >   examples/l3fwd: integrate RCU QSBR for LPM mode
> > > >
> > > >  doc/guides/sample_app_ug/l3_forward.rst |  3 ++
> > > >  examples/l3fwd/Makefile                 |  1 +
> > > >  examples/l3fwd/l3fwd.h                  |  4 +-
> > > >  examples/l3fwd/l3fwd_em.c               | 10 +++-
> > > >  examples/l3fwd/l3fwd_lpm.c              | 72
> +++++++++++++++++++++++--
> > > >  examples/l3fwd/main.c                   | 27 ++++++++--
> > > >  examples/l3fwd/meson.build              |  1 +
> > > >  7 files changed, 108 insertions(+), 10 deletions(-)
> > > >
> > > > --
> > > > 2.17.1
> >
>
  

> >
> > >
> > > Lock-free mode is supported by hash library and LPM library.
> > > Now we add an option for l3fwd example to enable the lock-free mode.
> > > Necessary preparation steps are added to use lock-free LPM mode.
> >
> > Can I ask about the purpose of these changes?
> > Right now in  l3fwd both lpm and hash tables are static and hard-coded.
> > we initialize them at startup and then just do read from them.
> > Do you plan to enhance l3fwd with ability to dynamically update tables
> > contents?
> > Though fir that we first have to get rid of hard-coded values (config file or so).
> > Konstantin
> >
> Thanks for your questions.
> Currently, we have no plan to enhance l3fwd with ability to dynamically update table contents.
> Lock-free method is being integrated into Hash library and LPM library.  Lock-free algorithms
> are not only about control plane (adding or deleting routes), they affect the data path performance
> as well.
> Since l3fwd application is showcasing data path performance, we need to show the impact of
> including the quiescent state reporting on data path.
> This change also serves as an example of using the RCU APIs.


But what you suggest doesn't provide the complete picture.
With dynamic updates in place (via control path) the data-path impact might
be completely different then without.
Again without dynamic updates how can you test that your data-path lock-free
approach does work as expected?
Also it can't even be used as a reference implementation for users,
as half of the functionality they need to implement is simply missing.
My opinion - we either need to leave l3fwd as it is (static routes),
or implement a proper control path with ability to dynamically update routes
before starting to introduce some synchronization schemes (RCU or whatever). 

Konstantin

> 
> > >
> > > Patch 2/2 has dependency on RCU QSBR integration with LPM library:
> > > http://patches.dpdk.org/project/dpdk/list/?series=6288
> > >
> > >
> > > Ruifeng Wang (2):
> > >   examples/l3fwd: add lock-free option for l3fwd
> > >   examples/l3fwd: integrate RCU QSBR for LPM mode
> > >
> > >  doc/guides/sample_app_ug/l3_forward.rst |  3 ++
> > >  examples/l3fwd/Makefile                 |  1 +
> > >  examples/l3fwd/l3fwd.h                  |  4 +-
> > >  examples/l3fwd/l3fwd_em.c               | 10 +++-
> > >  examples/l3fwd/l3fwd_lpm.c              | 72 +++++++++++++++++++++++--
> > >  examples/l3fwd/main.c                   | 27 ++++++++--
> > >  examples/l3fwd/meson.build              |  1 +
> > >  7 files changed, 108 insertions(+), 10 deletions(-)
> > >
> > > --
> > > 2.17.1
  

> -----Original Message-----
> From: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> Sent: Tuesday, September 10, 2019 17:06
> To: Ruifeng Wang (Arm Technology China) <Ruifeng.Wang@arm.com>;
> Kantecki, Tomasz <tomasz.kantecki@intel.com>
> Cc: dev@dpdk.org; Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>;
> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd
> <nd@arm.com>; nd <nd@arm.com>
> Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd
> 
> 
> 
> > >
> > > >
> > > > Lock-free mode is supported by hash library and LPM library.
> > > > Now we add an option for l3fwd example to enable the lock-free mode.
> > > > Necessary preparation steps are added to use lock-free LPM mode.
> > >
> > > Can I ask about the purpose of these changes?
> > > Right now in  l3fwd both lpm and hash tables are static and hard-coded.
> > > we initialize them at startup and then just do read from them.
> > > Do you plan to enhance l3fwd with ability to dynamically update
> > > tables contents?
> > > Though fir that we first have to get rid of hard-coded values (config file or
> so).
> > > Konstantin
> > >
> > Thanks for your questions.
> > Currently, we have no plan to enhance l3fwd with ability to dynamically
> update table contents.
> > Lock-free method is being integrated into Hash library and LPM
> > library.  Lock-free algorithms are not only about control plane
> > (adding or deleting routes), they affect the data path performance as well.
> > Since l3fwd application is showcasing data path performance, we need
> > to show the impact of including the quiescent state reporting on data path.
> > This change also serves as an example of using the RCU APIs.
> 
> 
> But what you suggest doesn't provide the complete picture.
> With dynamic updates in place (via control path) the data-path impact might
> be completely different then without.
> Again without dynamic updates how can you test that your data-path lock-
> free approach does work as expected?
> Also it can't even be used as a reference implementation for users, as half of
> the functionality they need to implement is simply missing.
> My opinion - we either need to leave l3fwd as it is (static routes), or
> implement a proper control path with ability to dynamically update routes
> before starting to introduce some synchronization schemes (RCU or
> whatever).
> 
> Konstantin
> 

Agree that dynamic control path updates should be included for a whole picture.
I will add dynamic update to l3fwd and reroll the patch series.
Thanks.

> >
> > > >
> > > > Patch 2/2 has dependency on RCU QSBR integration with LPM library:
> > > > http://patches.dpdk.org/project/dpdk/list/?series=6288
> > > >
> > > >
> > > > Ruifeng Wang (2):
> > > >   examples/l3fwd: add lock-free option for l3fwd
> > > >   examples/l3fwd: integrate RCU QSBR for LPM mode
> > > >
> > > >  doc/guides/sample_app_ug/l3_forward.rst |  3 ++
> > > >  examples/l3fwd/Makefile                 |  1 +
> > > >  examples/l3fwd/l3fwd.h                  |  4 +-
> > > >  examples/l3fwd/l3fwd_em.c               | 10 +++-
> > > >  examples/l3fwd/l3fwd_lpm.c              | 72
> +++++++++++++++++++++++--
> > > >  examples/l3fwd/main.c                   | 27 ++++++++--
> > > >  examples/l3fwd/meson.build              |  1 +
> > > >  7 files changed, 108 insertions(+), 10 deletions(-)
> > > >
> > > > --
> > > > 2.17.1
  

<snip>

> >
> > On Fri,  6 Sep 2019 18:26:13 +0800
> > Ruifeng Wang <ruifeng.wang@arm.com> wrote:
> >
> > > Lock-free mode is supported by hash library and LPM library.
> > > Now we add an option for l3fwd example to enable the lock-free mode.
> > > Necessary preparation steps are added to use lock-free LPM mode.
> >
> > If lock-free mode works it should just do that.
> > Having options mean that there are two test cases; which inevitably
> > leads to one of them being broken.
> 
> Agree that having options will add scenarios that being tested.
> Since these different scenarios are supported by Hash / LPM library, the tests
> on them should be valid.
> As l3fwd application is always used to benchmark data path performance,
> make both supported modes available can help user to easily collect data and
> compare.
> In the long run, we can make lock-free mode the default used by l3fwd when
> it is fine tuned.
I agree that if lock-free works we should just do that. It makes L3fwd application more practical.

The LPM has always been lock free on the data plane side. The writer side parts and quiescent state reporting on the data plane are being added now.

With the optimizations in the last release, the lock free hash algorithm's performance has come very close to RW lock based algorithm both for hit and miss cases for single core. Unfortunately (😊), any further optimizations seem to increase the performance of the RW lock based algorithm as well. So, looks like there will be that small difference between the 2 algorithms. However, a practical application will use other cores in the SoC and lock-free algorithms scale better with increasing number of cores. IMO, we should bite the bullet and make hash lock-free algorithm default.
  

> >
> > <snip>
> >
> > > >
> > > > Hi,
> > > >
> > > > >
> > > > > Lock-free mode is supported by hash library and LPM library.
> > > > > Now we add an option for l3fwd example to enable the lock-free
> mode.
> > > > > Necessary preparation steps are added to use lock-free LPM mode.
> > > >
> > > > Can I ask about the purpose of these changes?
> > > > Right now in  l3fwd both lpm and hash tables are static and hard-coded.
> > > > we initialize them at startup and then just do read from them.
> > > > Do you plan to enhance l3fwd with ability to dynamically update
> > > > tables contents?
> > > > Though fir that we first have to get rid of hard-coded values
> > > > (config file or
> > > so).
> > > > Konstantin
> > > >
> > > Thanks for your questions.
> > > Currently, we have no plan to enhance l3fwd with ability to
> > > dynamically update table contents.
> > > Lock-free method is being integrated into Hash library and LPM
> > > library.  Lock- free algorithms are not only about control plane
> > > (adding or deleting routes), they affect the data path performance as well.
> > > Since l3fwd application is showcasing data path performance, we need
> > > to show the impact of including the quiescent state reporting on data
> path.
> > > This change also serves as an example of using the RCU APIs.
> > >
> > Without the dynamic deletes the quiescent state reporting overhead is
> > not captured completely.
> > I suggest that we add and delete a small set of unrelated routes (the
> > routes that are not used currently) on a regular basis.
> >
> Add and delete unrelated routes on a regular basis will simulate overhead on
> control path. However, control path performance is not a l3fwd showcase,
> and it is covered by LPM performance unit test.
> On data path, quiescent state reporting overhead is constant (calling of
> rte_rcu_qsbr_quiescent).
> It will not be impacted by route addition / deletion.
The recent changes to RCU [1] are such that the reader threads avoid a write if there are no deletes. So, without the deletion, the impact on readers is complete.

[1] https://patchwork.dpdk.org/patch/58961/
> 
> > > > >
> > > > > Patch 2/2 has dependency on RCU QSBR integration with LPM library:
> > > > > http://patches.dpdk.org/project/dpdk/list/?series=6288
> > > > >
> > > > >
> > > > > Ruifeng Wang (2):
> > > > >   examples/l3fwd: add lock-free option for l3fwd
> > > > >   examples/l3fwd: integrate RCU QSBR for LPM mode
> > > > >
> > > > >  doc/guides/sample_app_ug/l3_forward.rst |  3 ++
> > > > >  examples/l3fwd/Makefile                 |  1 +
> > > > >  examples/l3fwd/l3fwd.h                  |  4 +-
> > > > >  examples/l3fwd/l3fwd_em.c               | 10 +++-
> > > > >  examples/l3fwd/l3fwd_lpm.c              | 72
> > +++++++++++++++++++++++--
> > > > >  examples/l3fwd/main.c                   | 27 ++++++++--
> > > > >  examples/l3fwd/meson.build              |  1 +
> > > > >  7 files changed, 108 insertions(+), 10 deletions(-)
> > > > >
> > > > > --
> > > > > 2.17.1
> > >
> >
>
  

<snip>
> >
> >
> >
> > > >
> > > > >
> > > > > Lock-free mode is supported by hash library and LPM library.
> > > > > Now we add an option for l3fwd example to enable the lock-free
> mode.
> > > > > Necessary preparation steps are added to use lock-free LPM mode.
> > > >
> > > > Can I ask about the purpose of these changes?
> > > > Right now in  l3fwd both lpm and hash tables are static and hard-coded.
> > > > we initialize them at startup and then just do read from them.
> > > > Do you plan to enhance l3fwd with ability to dynamically update
> > > > tables contents?
> > > > Though fir that we first have to get rid of hard-coded values
> > > > (config file or
> > so).
> > > > Konstantin
> > > >
> > > Thanks for your questions.
> > > Currently, we have no plan to enhance l3fwd with ability to
> > > dynamically
> > update table contents.
> > > Lock-free method is being integrated into Hash library and LPM
> > > library.  Lock-free algorithms are not only about control plane
> > > (adding or deleting routes), they affect the data path performance as well.
> > > Since l3fwd application is showcasing data path performance, we need
> > > to show the impact of including the quiescent state reporting on data
> path.
> > > This change also serves as an example of using the RCU APIs.
> >
> >
> > But what you suggest doesn't provide the complete picture.
> > With dynamic updates in place (via control path) the data-path impact
> > might be completely different then without.
> > Again without dynamic updates how can you test that your data-path
> > lock- free approach does work as expected?
> > Also it can't even be used as a reference implementation for users, as
> > half of the functionality they need to implement is simply missing.
> > My opinion - we either need to leave l3fwd as it is (static routes),
> > or implement a proper control path with ability to dynamically update
> > routes before starting to introduce some synchronization schemes (RCU
> > or whatever).
> >
> > Konstantin
> >
> 
> Agree that dynamic control path updates should be included for a whole
> picture.
> I will add dynamic update to l3fwd and reroll the patch series.
> Thanks.
I think we should have an agreement on what exactly we mean by 'dynamically update routes'.
IMO, we should not disturb the existing static routes as there might be automated tests running in the labs. I suggest that we should add/delete new routes/hash entries which are different from the existing routes/hash entries. This should be sufficient to showcase the functionality as well as measure the impact.

> 
> > >
> > > > >
> > > > > Patch 2/2 has dependency on RCU QSBR integration with LPM library:
> > > > > http://patches.dpdk.org/project/dpdk/list/?series=6288
> > > > >
> > > > >
> > > > > Ruifeng Wang (2):
> > > > >   examples/l3fwd: add lock-free option for l3fwd
> > > > >   examples/l3fwd: integrate RCU QSBR for LPM mode
> > > > >
> > > > >  doc/guides/sample_app_ug/l3_forward.rst |  3 ++
> > > > >  examples/l3fwd/Makefile                 |  1 +
> > > > >  examples/l3fwd/l3fwd.h                  |  4 +-
> > > > >  examples/l3fwd/l3fwd_em.c               | 10 +++-
> > > > >  examples/l3fwd/l3fwd_lpm.c              | 72
> > +++++++++++++++++++++++--
> > > > >  examples/l3fwd/main.c                   | 27 ++++++++--
> > > > >  examples/l3fwd/meson.build              |  1 +
> > > > >  7 files changed, 108 insertions(+), 10 deletions(-)
> > > > >
> > > > > --
> > > > > 2.17.1
>
  

> -----Original Message-----
> From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> Sent: Wednesday, September 11, 2019 13:33
> To: Ruifeng Wang (Arm Technology China) <Ruifeng.Wang@arm.com>;
> Ananyev, Konstantin <konstantin.ananyev@intel.com>; Kantecki, Tomasz
> <tomasz.kantecki@intel.com>
> Cc: dev@dpdk.org; Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>;
> nd <nd@arm.com>; nd <nd@arm.com>
> Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd
> 
> > >
> > > <snip>
> > >
> > > > >
> > > > > Hi,
> > > > >
> > > > > >
> > > > > > Lock-free mode is supported by hash library and LPM library.
> > > > > > Now we add an option for l3fwd example to enable the lock-free
> > mode.
> > > > > > Necessary preparation steps are added to use lock-free LPM mode.
> > > > >
> > > > > Can I ask about the purpose of these changes?
> > > > > Right now in  l3fwd both lpm and hash tables are static and hard-
> coded.
> > > > > we initialize them at startup and then just do read from them.
> > > > > Do you plan to enhance l3fwd with ability to dynamically update
> > > > > tables contents?
> > > > > Though fir that we first have to get rid of hard-coded values
> > > > > (config file or
> > > > so).
> > > > > Konstantin
> > > > >
> > > > Thanks for your questions.
> > > > Currently, we have no plan to enhance l3fwd with ability to
> > > > dynamically update table contents.
> > > > Lock-free method is being integrated into Hash library and LPM
> > > > library.  Lock- free algorithms are not only about control plane
> > > > (adding or deleting routes), they affect the data path performance as
> well.
> > > > Since l3fwd application is showcasing data path performance, we
> > > > need to show the impact of including the quiescent state reporting
> > > > on data
> > path.
> > > > This change also serves as an example of using the RCU APIs.
> > > >
> > > Without the dynamic deletes the quiescent state reporting overhead
> > > is not captured completely.
> > > I suggest that we add and delete a small set of unrelated routes
> > > (the routes that are not used currently) on a regular basis.
> > >
> > Add and delete unrelated routes on a regular basis will simulate
> > overhead on control path. However, control path performance is not a
> > l3fwd showcase, and it is covered by LPM performance unit test.
> > On data path, quiescent state reporting overhead is constant (calling
> > of rte_rcu_qsbr_quiescent).
> > It will not be impacted by route addition / deletion.
> The recent changes to RCU [1] are such that the reader threads avoid a write
> if there are no deletes. So, without the deletion, the impact on readers is
> complete.
> 
> [1] https://patchwork.dpdk.org/patch/58961/

OK, understand that. 
Assumptions based on current implementation of rte_rcu_qsbr_quiescent is not reliable. 
The part on control path will be added.

> >
> > > > > >
> > > > > > Patch 2/2 has dependency on RCU QSBR integration with LPM
> library:
> > > > > > http://patches.dpdk.org/project/dpdk/list/?series=6288
> > > > > >
> > > > > >
> > > > > > Ruifeng Wang (2):
> > > > > >   examples/l3fwd: add lock-free option for l3fwd
> > > > > >   examples/l3fwd: integrate RCU QSBR for LPM mode
> > > > > >
> > > > > >  doc/guides/sample_app_ug/l3_forward.rst |  3 ++
> > > > > >  examples/l3fwd/Makefile                 |  1 +
> > > > > >  examples/l3fwd/l3fwd.h                  |  4 +-
> > > > > >  examples/l3fwd/l3fwd_em.c               | 10 +++-
> > > > > >  examples/l3fwd/l3fwd_lpm.c              | 72
> > > +++++++++++++++++++++++--
> > > > > >  examples/l3fwd/main.c                   | 27 ++++++++--
> > > > > >  examples/l3fwd/meson.build              |  1 +
> > > > > >  7 files changed, 108 insertions(+), 10 deletions(-)
> > > > > >
> > > > > > --
> > > > > > 2.17.1
> > > >
> > >
> >
>
  

> -----Original Message-----
> From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> Sent: Wednesday, September 11, 2019 13:38
> To: Ruifeng Wang (Arm Technology China) <Ruifeng.Wang@arm.com>;
> Ananyev, Konstantin <konstantin.ananyev@intel.com>; Kantecki, Tomasz
> <tomasz.kantecki@intel.com>
> Cc: dev@dpdk.org; Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>;
> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd
> <nd@arm.com>; nd <nd@arm.com>
> Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd
> 
> <snip>
> > >
> > >
> > >
> > > > >
> > > > > >
> > > > > > Lock-free mode is supported by hash library and LPM library.
> > > > > > Now we add an option for l3fwd example to enable the lock-free
> > mode.
> > > > > > Necessary preparation steps are added to use lock-free LPM mode.
> > > > >
> > > > > Can I ask about the purpose of these changes?
> > > > > Right now in  l3fwd both lpm and hash tables are static and hard-
> coded.
> > > > > we initialize them at startup and then just do read from them.
> > > > > Do you plan to enhance l3fwd with ability to dynamically update
> > > > > tables contents?
> > > > > Though fir that we first have to get rid of hard-coded values
> > > > > (config file or
> > > so).
> > > > > Konstantin
> > > > >
> > > > Thanks for your questions.
> > > > Currently, we have no plan to enhance l3fwd with ability to
> > > > dynamically
> > > update table contents.
> > > > Lock-free method is being integrated into Hash library and LPM
> > > > library.  Lock-free algorithms are not only about control plane
> > > > (adding or deleting routes), they affect the data path performance as
> well.
> > > > Since l3fwd application is showcasing data path performance, we
> > > > need to show the impact of including the quiescent state reporting
> > > > on data
> > path.
> > > > This change also serves as an example of using the RCU APIs.
> > >
> > >
> > > But what you suggest doesn't provide the complete picture.
> > > With dynamic updates in place (via control path) the data-path
> > > impact might be completely different then without.
> > > Again without dynamic updates how can you test that your data-path
> > > lock- free approach does work as expected?
> > > Also it can't even be used as a reference implementation for users,
> > > as half of the functionality they need to implement is simply missing.
> > > My opinion - we either need to leave l3fwd as it is (static routes),
> > > or implement a proper control path with ability to dynamically
> > > update routes before starting to introduce some synchronization
> > > schemes (RCU or whatever).
> > >
> > > Konstantin
> > >
> >
> > Agree that dynamic control path updates should be included for a whole
> > picture.
> > I will add dynamic update to l3fwd and reroll the patch series.
> > Thanks.
> I think we should have an agreement on what exactly we mean by
> 'dynamically update routes'.
> IMO, we should not disturb the existing static routes as there might be
> automated tests running in the labs. I suggest that we should add/delete
> new routes/hash entries which are different from the existing routes/hash
> entries. This should be sufficient to showcase the functionality as well as
> measure the impact.
> 
Yes, existing static routes should be kept intact.
To perform regular route/hash entries add/delete, a dedicated lcore will be needed.
An interactive prompt is not an option since we need automatic add/delete.
We can skip master core for data path main loop. And perform unrelated route/hash entries add/delete regularly on master core.
The impact is that command lines used in tests will need update since master core will no longer do data path work.
> >
> > > >
> > > > > >
> > > > > > Patch 2/2 has dependency on RCU QSBR integration with LPM
> library:
> > > > > > http://patches.dpdk.org/project/dpdk/list/?series=6288
> > > > > >
> > > > > >
> > > > > > Ruifeng Wang (2):
> > > > > >   examples/l3fwd: add lock-free option for l3fwd
> > > > > >   examples/l3fwd: integrate RCU QSBR for LPM mode
> > > > > >
> > > > > >  doc/guides/sample_app_ug/l3_forward.rst |  3 ++
> > > > > >  examples/l3fwd/Makefile                 |  1 +
> > > > > >  examples/l3fwd/l3fwd.h                  |  4 +-
> > > > > >  examples/l3fwd/l3fwd_em.c               | 10 +++-
> > > > > >  examples/l3fwd/l3fwd_lpm.c              | 72
> > > +++++++++++++++++++++++--
> > > > > >  examples/l3fwd/main.c                   | 27 ++++++++--
> > > > > >  examples/l3fwd/meson.build              |  1 +
> > > > > >  7 files changed, 108 insertions(+), 10 deletions(-)
> > > > > >
> > > > > > --
> > > > > > 2.17.1
> >
>
  

> -----Original Message-----
> From: Ruifeng Wang (Arm Technology China) [mailto:Ruifeng.Wang@arm.com]
> Sent: Wednesday, September 11, 2019 7:58 AM
> To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Kantecki, Tomasz
> <tomasz.kantecki@intel.com>
> Cc: dev@dpdk.org; Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>; nd <nd@arm.com>; nd <nd@arm.com>; nd <nd@arm.com>
> Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd
> 
> > -----Original Message-----
> > From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> > Sent: Wednesday, September 11, 2019 13:38
> > To: Ruifeng Wang (Arm Technology China) <Ruifeng.Wang@arm.com>;
> > Ananyev, Konstantin <konstantin.ananyev@intel.com>; Kantecki, Tomasz
> > <tomasz.kantecki@intel.com>
> > Cc: dev@dpdk.org; Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>;
> > Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; nd
> > <nd@arm.com>; nd <nd@arm.com>
> > Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd
> >
> > <snip>
> > > >
> > > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > Lock-free mode is supported by hash library and LPM library.
> > > > > > > Now we add an option for l3fwd example to enable the lock-free
> > > mode.
> > > > > > > Necessary preparation steps are added to use lock-free LPM mode.
> > > > > >
> > > > > > Can I ask about the purpose of these changes?
> > > > > > Right now in  l3fwd both lpm and hash tables are static and hard-
> > coded.
> > > > > > we initialize them at startup and then just do read from them.
> > > > > > Do you plan to enhance l3fwd with ability to dynamically update
> > > > > > tables contents?
> > > > > > Though fir that we first have to get rid of hard-coded values
> > > > > > (config file or
> > > > so).
> > > > > > Konstantin
> > > > > >
> > > > > Thanks for your questions.
> > > > > Currently, we have no plan to enhance l3fwd with ability to
> > > > > dynamically
> > > > update table contents.
> > > > > Lock-free method is being integrated into Hash library and LPM
> > > > > library.  Lock-free algorithms are not only about control plane
> > > > > (adding or deleting routes), they affect the data path performance as
> > well.
> > > > > Since l3fwd application is showcasing data path performance, we
> > > > > need to show the impact of including the quiescent state reporting
> > > > > on data
> > > path.
> > > > > This change also serves as an example of using the RCU APIs.
> > > >
> > > >
> > > > But what you suggest doesn't provide the complete picture.
> > > > With dynamic updates in place (via control path) the data-path
> > > > impact might be completely different then without.
> > > > Again without dynamic updates how can you test that your data-path
> > > > lock- free approach does work as expected?
> > > > Also it can't even be used as a reference implementation for users,
> > > > as half of the functionality they need to implement is simply missing.
> > > > My opinion - we either need to leave l3fwd as it is (static routes),
> > > > or implement a proper control path with ability to dynamically
> > > > update routes before starting to introduce some synchronization
> > > > schemes (RCU or whatever).
> > > >
> > > > Konstantin
> > > >
> > >
> > > Agree that dynamic control path updates should be included for a whole
> > > picture.
> > > I will add dynamic update to l3fwd and reroll the patch series.
> > > Thanks.
> > I think we should have an agreement on what exactly we mean by
> > 'dynamically update routes'.
> > IMO, we should not disturb the existing static routes as there might be
> > automated tests running in the labs. I suggest that we should add/delete
> > new routes/hash entries which are different from the existing routes/hash
> > entries. This should be sufficient to showcase the functionality as well as
> > measure the impact.
> >
> Yes, existing static routes should be kept intact.
> To perform regular route/hash entries add/delete, a dedicated lcore will be needed.
> An interactive prompt is not an option since we need automatic add/delete.
> We can skip master core for data path main loop. And perform unrelated route/hash entries add/delete regularly on master core.
> The impact is that command lines used in tests will need update since master core will no longer do data path work.

Not sure why it has to be  master core?
Why interrupt thread wouldn't do?
I think what we need to:
1. introduce reading routes from config file instead of having them hard-coded within the app.
2. add ability to update routes dynamically.
    Probably the easiest (and commonly used way) re-read conf file and update routes on the signal (SIGUSR1 or so).
Konstantin


> > >
> > > > >
> > > > > > >
> > > > > > > Patch 2/2 has dependency on RCU QSBR integration with LPM
> > library:
> > > > > > > http://patches.dpdk.org/project/dpdk/list/?series=6288
> > > > > > >
> > > > > > >
> > > > > > > Ruifeng Wang (2):
> > > > > > >   examples/l3fwd: add lock-free option for l3fwd
> > > > > > >   examples/l3fwd: integrate RCU QSBR for LPM mode
> > > > > > >
> > > > > > >  doc/guides/sample_app_ug/l3_forward.rst |  3 ++
> > > > > > >  examples/l3fwd/Makefile                 |  1 +
> > > > > > >  examples/l3fwd/l3fwd.h                  |  4 +-
> > > > > > >  examples/l3fwd/l3fwd_em.c               | 10 +++-
> > > > > > >  examples/l3fwd/l3fwd_lpm.c              | 72
> > > > +++++++++++++++++++++++--
> > > > > > >  examples/l3fwd/main.c                   | 27 ++++++++--
> > > > > > >  examples/l3fwd/meson.build              |  1 +
> > > > > > >  7 files changed, 108 insertions(+), 10 deletions(-)
> > > > > > >
> > > > > > > --
> > > > > > > 2.17.1
> > >
> >
  

> -----Original Message-----
> From: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> Sent: Wednesday, September 11, 2019 16:35
> To: Ruifeng Wang (Arm Technology China) <Ruifeng.Wang@arm.com>;
> Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; Kantecki, Tomasz
> <tomasz.kantecki@intel.com>
> Cc: dev@dpdk.org; Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>;
> nd <nd@arm.com>; nd <nd@arm.com>; nd <nd@arm.com>
> Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd
> 
> 
> 
> > -----Original Message-----
> > From: Ruifeng Wang (Arm Technology China)
> > [mailto:Ruifeng.Wang@arm.com]
> > Sent: Wednesday, September 11, 2019 7:58 AM
> > To: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>; Ananyev,
> > Konstantin <konstantin.ananyev@intel.com>; Kantecki, Tomasz
> > <tomasz.kantecki@intel.com>
> > Cc: dev@dpdk.org; Gavin Hu (Arm Technology China)
> <Gavin.Hu@arm.com>;
> > nd <nd@arm.com>; nd <nd@arm.com>; nd <nd@arm.com>
> > Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd
> >
> > > -----Original Message-----
> > > From: Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>
> > > Sent: Wednesday, September 11, 2019 13:38
> > > To: Ruifeng Wang (Arm Technology China) <Ruifeng.Wang@arm.com>;
> > > Ananyev, Konstantin <konstantin.ananyev@intel.com>; Kantecki,
> Tomasz
> > > <tomasz.kantecki@intel.com>
> > > Cc: dev@dpdk.org; Gavin Hu (Arm Technology China)
> > > <Gavin.Hu@arm.com>; Honnappa Nagarahalli
> > > <Honnappa.Nagarahalli@arm.com>; nd <nd@arm.com>; nd
> <nd@arm.com>
> > > Subject: RE: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd
> > >
> > > <snip>
> > > > >
> > > > >
> > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > Lock-free mode is supported by hash library and LPM library.
> > > > > > > > Now we add an option for l3fwd example to enable the
> > > > > > > > lock-free
> > > > mode.
> > > > > > > > Necessary preparation steps are added to use lock-free LPM
> mode.
> > > > > > >
> > > > > > > Can I ask about the purpose of these changes?
> > > > > > > Right now in  l3fwd both lpm and hash tables are static and
> > > > > > > hard-
> > > coded.
> > > > > > > we initialize them at startup and then just do read from them.
> > > > > > > Do you plan to enhance l3fwd with ability to dynamically
> > > > > > > update tables contents?
> > > > > > > Though fir that we first have to get rid of hard-coded
> > > > > > > values (config file or
> > > > > so).
> > > > > > > Konstantin
> > > > > > >
> > > > > > Thanks for your questions.
> > > > > > Currently, we have no plan to enhance l3fwd with ability to
> > > > > > dynamically
> > > > > update table contents.
> > > > > > Lock-free method is being integrated into Hash library and LPM
> > > > > > library.  Lock-free algorithms are not only about control
> > > > > > plane (adding or deleting routes), they affect the data path
> > > > > > performance as
> > > well.
> > > > > > Since l3fwd application is showcasing data path performance,
> > > > > > we need to show the impact of including the quiescent state
> > > > > > reporting on data
> > > > path.
> > > > > > This change also serves as an example of using the RCU APIs.
> > > > >
> > > > >
> > > > > But what you suggest doesn't provide the complete picture.
> > > > > With dynamic updates in place (via control path) the data-path
> > > > > impact might be completely different then without.
> > > > > Again without dynamic updates how can you test that your
> > > > > data-path
> > > > > lock- free approach does work as expected?
> > > > > Also it can't even be used as a reference implementation for
> > > > > users, as half of the functionality they need to implement is simply
> missing.
> > > > > My opinion - we either need to leave l3fwd as it is (static
> > > > > routes), or implement a proper control path with ability to
> > > > > dynamically update routes before starting to introduce some
> > > > > synchronization schemes (RCU or whatever).
> > > > >
> > > > > Konstantin
> > > > >
> > > >
> > > > Agree that dynamic control path updates should be included for a
> > > > whole picture.
> > > > I will add dynamic update to l3fwd and reroll the patch series.
> > > > Thanks.
> > > I think we should have an agreement on what exactly we mean by
> > > 'dynamically update routes'.
> > > IMO, we should not disturb the existing static routes as there might
> > > be automated tests running in the labs. I suggest that we should
> > > add/delete new routes/hash entries which are different from the
> > > existing routes/hash entries. This should be sufficient to showcase
> > > the functionality as well as measure the impact.
> > >
> > Yes, existing static routes should be kept intact.
> > To perform regular route/hash entries add/delete, a dedicated lcore will be
> needed.
> > An interactive prompt is not an option since we need automatic add/delete.
> > We can skip master core for data path main loop. And perform unrelated
> route/hash entries add/delete regularly on master core.
> > The impact is that command lines used in tests will need update since
> master core will no longer do data path work.
> 
> Not sure why it has to be  master core?
> Why interrupt thread wouldn't do?
> I think what we need to:
> 1. introduce reading routes from config file instead of having them hard-
> coded within the app.
> 2. add ability to update routes dynamically.
>     Probably the easiest (and commonly used way) re-read conf file and
> update routes on the signal (SIGUSR1 or so).
> Konstantin
> 
> 
Sorry for delayed response. Just back from vacation. 
Thanks for your suggestion.
I will try the config file based updating approach and get back with new version.

> > > >
> > > > > >
> > > > > > > >
> > > > > > > > Patch 2/2 has dependency on RCU QSBR integration with LPM
> > > library:
> > > > > > > > http://patches.dpdk.org/project/dpdk/list/?series=6288
> > > > > > > >
> > > > > > > >
> > > > > > > > Ruifeng Wang (2):
> > > > > > > >   examples/l3fwd: add lock-free option for l3fwd
> > > > > > > >   examples/l3fwd: integrate RCU QSBR for LPM mode
> > > > > > > >
> > > > > > > >  doc/guides/sample_app_ug/l3_forward.rst |  3 ++
> > > > > > > >  examples/l3fwd/Makefile                 |  1 +
> > > > > > > >  examples/l3fwd/l3fwd.h                  |  4 +-
> > > > > > > >  examples/l3fwd/l3fwd_em.c               | 10 +++-
> > > > > > > >  examples/l3fwd/l3fwd_lpm.c              | 72
> > > > > +++++++++++++++++++++++--
> > > > > > > >  examples/l3fwd/main.c                   | 27 ++++++++--
> > > > > > > >  examples/l3fwd/meson.build              |  1 +
> > > > > > > >  7 files changed, 108 insertions(+), 10 deletions(-)
> > > > > > > >
> > > > > > > > --
> > > > > > > > 2.17.1
> > > >
> > >
  

Hello Ruifeng,

On Mon, Sep 16, 2019 at 4:30 AM Ruifeng Wang (Arm Technology China)
<Ruifeng.Wang@arm.com> wrote:
> > Not sure why it has to be  master core?
> > Why interrupt thread wouldn't do?
> > I think what we need to:
> > 1. introduce reading routes from config file instead of having them hard-
> > coded within the app.
> > 2. add ability to update routes dynamically.
> >     Probably the easiest (and commonly used way) re-read conf file and
> > update routes on the signal (SIGUSR1 or so).
> > Konstantin
> >
> >
> Sorry for delayed response. Just back from vacation.
> Thanks for your suggestion.
> I will try the config file based updating approach and get back with new version.

Any update on this series?
  

Hi David,

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Wednesday, November 6, 2019 22:04
> To: Ruifeng Wang (Arm Technology China) <Ruifeng.Wang@arm.com>
> Cc: Ananyev, Konstantin <konstantin.ananyev@intel.com>; Honnappa
> Nagarahalli <Honnappa.Nagarahalli@arm.com>; Kantecki, Tomasz
> <tomasz.kantecki@intel.com>; dev@dpdk.org; Gavin Hu (Arm Technology
> China) <Gavin.Hu@arm.com>; nd <nd@arm.com>
> Subject: Re: [dpdk-dev] [PATCH 0/2] add lock-free mode for l3fwd
> 
> Hello Ruifeng,
> 
> On Mon, Sep 16, 2019 at 4:30 AM Ruifeng Wang (Arm Technology China)
> <Ruifeng.Wang@arm.com> wrote:
> > > Not sure why it has to be  master core?
> > > Why interrupt thread wouldn't do?
> > > I think what we need to:
> > > 1. introduce reading routes from config file instead of having them
> > > hard- coded within the app.
> > > 2. add ability to update routes dynamically.
> > >     Probably the easiest (and commonly used way) re-read conf file
> > > and update routes on the signal (SIGUSR1 or so).
> > > Konstantin
> > >
> > >
> > Sorry for delayed response. Just back from vacation.
> > Thanks for your suggestion.
> > I will try the config file based updating approach and get back with new
> version.
> 
> Any update on this series?

I heard different opinions on the approach. Currently no new version patch set available.
Will have more discussion before respin.
> 
> 
> --
> David Marchand