[v5,3/5] test/hash: add additional thash tests

  This patch adds tests for predictable RSS feature.

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Yipeng Wang <yipeng1.wang@intel.com>
---
 app/test/test_thash.c | 469 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 463 insertions(+), 6 deletions(-)
  

Hello Vladimir,

On Mon, Apr 19, 2021 at 6:00 PM Vladimir Medvedkin
<vladimir.medvedkin@intel.com> wrote:
> +static int
> +test_adjust_tuple(void)
> +{
> +       struct rte_thash_ctx *ctx;
> +       struct rte_thash_subtuple_helper *h;
> +       const int key_len = 40;
> +       const uint8_t *new_key;
> +       uint8_t tuple[TUPLE_SZ];
> +       uint32_t tmp_tuple[TUPLE_SZ / sizeof(uint32_t)];
> +       uint32_t tuple_copy[TUPLE_SZ / sizeof(uint32_t)];
> +       uint32_t hash;
> +       int reta_sz = CHAR_BIT;
> +       int ret;
> +       unsigned int i, desired_value = rte_rand() & HASH_MSK(reta_sz);
> +
> +       memset(tuple, 0xab, TUPLE_SZ);
> +
> +       ctx = rte_thash_init_ctx("test", key_len, reta_sz, NULL, 0);
> +       RTE_TEST_ASSERT(ctx != NULL, "can not create thash ctx\n");
> +
> +       /*
> +        * set offset to be in the middle of a byte
> +        * set size of the subtuple to be 2 * rets_sz
> +        * to have the room for random bits
> +        */
> +       ret = rte_thash_add_helper(ctx, "test", reta_sz * 2,
> +               (5 * CHAR_BIT) + 4);
> +       RTE_TEST_ASSERT(ret == 0, "can not add helper, ret %d\n", ret);
> +
> +       new_key = rte_thash_get_key(ctx);
> +
> +       h = rte_thash_get_helper(ctx, "test");
> +       RTE_TEST_ASSERT(h != NULL, "can not find helper\n");
> +
> +       ret = rte_thash_adjust_tuple(ctx, h, tuple, TUPLE_SZ, desired_value,
> +               1, NULL, NULL);
> +       RTE_TEST_ASSERT(ret == 0, "can not adjust tuple, ret %d\n", ret);
> +
> +       for (i = 0; i < (TUPLE_SZ / 4); i++)
> +               tmp_tuple[i] =
> +                       rte_be_to_cpu_32(*(uint32_t *)&tuple[i * 4]);
> +
> +       hash = rte_softrss(tmp_tuple, TUPLE_SZ / 4, new_key);
> +       RTE_TEST_ASSERT((hash & HASH_MSK(reta_sz)) ==
> +               desired_value, "bad desired value\n");
> +
> +
> +       /* Pass previously calculated tuple to callback function */
> +       memcpy(tuple_copy, tuple, TUPLE_SZ);
> +
> +       memset(tuple, 0xab, TUPLE_SZ);
> +       ret = rte_thash_adjust_tuple(ctx, h, tuple, TUPLE_SZ, desired_value,
> +               1, cmp_tuple_eq, tuple_copy);
> +       RTE_TEST_ASSERT(ret == -EEXIST,
> +               "adjust tuple didn't indicate collision\n");
> +
> +       /*
> +        * Make the function to generate random bits into subtuple
> +        * after first adjustment attempt.
> +        */
> +       memset(tuple, 0xab, TUPLE_SZ);
> +       ret = rte_thash_adjust_tuple(ctx, h, tuple, TUPLE_SZ, desired_value,
> +               2, cmp_tuple_eq, tuple_copy);
> +       RTE_TEST_ASSERT(ret == 0, "can not adjust tuple, ret %d\n", ret);


This check failed at least once in the CI, for no obvious (for me) reason.
https://github.com/ovsrobot/dpdk/actions/runs/794713806

87/96 DPDK:fast-tests / thash_autotest        FAIL     0.17 s (exit
status 255 or signal 127 SIGinvalid)

--- command ---
DPDK_TEST='thash_autotest'
/home/runner/work/dpdk/dpdk/build/app/test/dpdk-test -l 0-1
--file-prefix=thash_autotest
--- stdout ---
RTE>>thash_autotest
 + ------------------------------------------------------- +
 + Test Suite : thash autotest
 + ------------------------------------------------------- +
 + TestCase [ 0] : test_toeplitz_hash_calc succeeded
 + TestCase [ 1] : test_create_invalid succeeded
 + TestCase [ 2] : test_multiple_create succeeded
 + TestCase [ 3] : test_free_null succeeded
 + TestCase [ 4] : test_add_invalid_helper succeeded
 + TestCase [ 5] : test_find_existing succeeded
 + TestCase [ 6] : test_get_helper succeeded
 + TestCase [ 7] : test_period_overflow succeeded
 + TestCase [ 8] : test_predictable_rss_min_seq succeeded
 + TestCase [ 9] : test_predictable_rss_multirange succeeded
 + TestCase [10] : test_adjust_tuple failed
 + ------------------------------------------------------- +
 + Test Suite Summary
 + Tests Total :       11
 + Tests Skipped :      0
 + Tests Executed :    11
 + Tests Unsupported:   0
 + Tests Passed :      10
 + Tests Failed :       1
 + ------------------------------------------------------- +
Test Failed
RTE>>
--- stderr ---
EAL: Detected 2 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/thash_autotest/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: No available 1048576 kB hugepages reported
EAL: VFIO support initialized
APP: HPET is not enabled, using TSC as default timer
HASH: Can't add helper  due to conflict with existing helper second_range
HASH: Can't generate m-sequence due to period overflow
EAL: Test assert test_adjust_tuple line 559 failed: can not adjust
tuple, ret -17

-------



> +
> +       for (i = 0; i < (TUPLE_SZ / 4); i++)
> +               tmp_tuple[i] =
> +                       rte_be_to_cpu_32(*(uint32_t *)&tuple[i * 4]);
> +
> +       hash = rte_softrss(tmp_tuple, TUPLE_SZ / 4, new_key);
> +       RTE_TEST_ASSERT((hash & HASH_MSK(reta_sz)) ==
> +               desired_value, "bad desired value\n");
> +
> +       rte_thash_free_ctx(ctx);
> +
> +       return TEST_SUCCESS;
> +}
  

Hello David,

Oh, interesting, thanks for the report, I'll take a look!

On 29/04/2021 12:13, David Marchand wrote:
> Hello Vladimir,
> 
> On Mon, Apr 19, 2021 at 6:00 PM Vladimir Medvedkin
> <vladimir.medvedkin@intel.com> wrote:
>> +static int
>> +test_adjust_tuple(void)
>> +{
>> +       struct rte_thash_ctx *ctx;
>> +       struct rte_thash_subtuple_helper *h;
>> +       const int key_len = 40;
>> +       const uint8_t *new_key;
>> +       uint8_t tuple[TUPLE_SZ];
>> +       uint32_t tmp_tuple[TUPLE_SZ / sizeof(uint32_t)];
>> +       uint32_t tuple_copy[TUPLE_SZ / sizeof(uint32_t)];
>> +       uint32_t hash;
>> +       int reta_sz = CHAR_BIT;
>> +       int ret;
>> +       unsigned int i, desired_value = rte_rand() & HASH_MSK(reta_sz);
>> +
>> +       memset(tuple, 0xab, TUPLE_SZ);
>> +
>> +       ctx = rte_thash_init_ctx("test", key_len, reta_sz, NULL, 0);
>> +       RTE_TEST_ASSERT(ctx != NULL, "can not create thash ctx\n");
>> +
>> +       /*
>> +        * set offset to be in the middle of a byte
>> +        * set size of the subtuple to be 2 * rets_sz
>> +        * to have the room for random bits
>> +        */
>> +       ret = rte_thash_add_helper(ctx, "test", reta_sz * 2,
>> +               (5 * CHAR_BIT) + 4);
>> +       RTE_TEST_ASSERT(ret == 0, "can not add helper, ret %d\n", ret);
>> +
>> +       new_key = rte_thash_get_key(ctx);
>> +
>> +       h = rte_thash_get_helper(ctx, "test");
>> +       RTE_TEST_ASSERT(h != NULL, "can not find helper\n");
>> +
>> +       ret = rte_thash_adjust_tuple(ctx, h, tuple, TUPLE_SZ, desired_value,
>> +               1, NULL, NULL);
>> +       RTE_TEST_ASSERT(ret == 0, "can not adjust tuple, ret %d\n", ret);
>> +
>> +       for (i = 0; i < (TUPLE_SZ / 4); i++)
>> +               tmp_tuple[i] =
>> +                       rte_be_to_cpu_32(*(uint32_t *)&tuple[i * 4]);
>> +
>> +       hash = rte_softrss(tmp_tuple, TUPLE_SZ / 4, new_key);
>> +       RTE_TEST_ASSERT((hash & HASH_MSK(reta_sz)) ==
>> +               desired_value, "bad desired value\n");
>> +
>> +
>> +       /* Pass previously calculated tuple to callback function */
>> +       memcpy(tuple_copy, tuple, TUPLE_SZ);
>> +
>> +       memset(tuple, 0xab, TUPLE_SZ);
>> +       ret = rte_thash_adjust_tuple(ctx, h, tuple, TUPLE_SZ, desired_value,
>> +               1, cmp_tuple_eq, tuple_copy);
>> +       RTE_TEST_ASSERT(ret == -EEXIST,
>> +               "adjust tuple didn't indicate collision\n");
>> +
>> +       /*
>> +        * Make the function to generate random bits into subtuple
>> +        * after first adjustment attempt.
>> +        */
>> +       memset(tuple, 0xab, TUPLE_SZ);
>> +       ret = rte_thash_adjust_tuple(ctx, h, tuple, TUPLE_SZ, desired_value,
>> +               2, cmp_tuple_eq, tuple_copy);
>> +       RTE_TEST_ASSERT(ret == 0, "can not adjust tuple, ret %d\n", ret);
> 
> 
> This check failed at least once in the CI, for no obvious (for me) reason.
> https://github.com/ovsrobot/dpdk/actions/runs/794713806
> 
> 87/96 DPDK:fast-tests / thash_autotest        FAIL     0.17 s (exit
> status 255 or signal 127 SIGinvalid)
> 
> --- command ---
> DPDK_TEST='thash_autotest'
> /home/runner/work/dpdk/dpdk/build/app/test/dpdk-test -l 0-1
> --file-prefix=thash_autotest
> --- stdout ---
> RTE>>thash_autotest
>   + ------------------------------------------------------- +
>   + Test Suite : thash autotest
>   + ------------------------------------------------------- +
>   + TestCase [ 0] : test_toeplitz_hash_calc succeeded
>   + TestCase [ 1] : test_create_invalid succeeded
>   + TestCase [ 2] : test_multiple_create succeeded
>   + TestCase [ 3] : test_free_null succeeded
>   + TestCase [ 4] : test_add_invalid_helper succeeded
>   + TestCase [ 5] : test_find_existing succeeded
>   + TestCase [ 6] : test_get_helper succeeded
>   + TestCase [ 7] : test_period_overflow succeeded
>   + TestCase [ 8] : test_predictable_rss_min_seq succeeded
>   + TestCase [ 9] : test_predictable_rss_multirange succeeded
>   + TestCase [10] : test_adjust_tuple failed
>   + ------------------------------------------------------- +
>   + Test Suite Summary
>   + Tests Total :       11
>   + Tests Skipped :      0
>   + Tests Executed :    11
>   + Tests Unsupported:   0
>   + Tests Passed :      10
>   + Tests Failed :       1
>   + ------------------------------------------------------- +
> Test Failed
> RTE>>
> --- stderr ---
> EAL: Detected 2 lcore(s)
> EAL: Detected 1 NUMA nodes
> EAL: Detected shared linkage of DPDK
> EAL: Multi-process socket /var/run/dpdk/thash_autotest/mp_socket
> EAL: Selected IOVA mode 'PA'
> EAL: No available 1048576 kB hugepages reported
> EAL: VFIO support initialized
> APP: HPET is not enabled, using TSC as default timer
> HASH: Can't add helper  due to conflict with existing helper second_range
> HASH: Can't generate m-sequence due to period overflow
> EAL: Test assert test_adjust_tuple line 559 failed: can not adjust
> tuple, ret -17
> 
> -------
> 
> 
> 
>> +
>> +       for (i = 0; i < (TUPLE_SZ / 4); i++)
>> +               tmp_tuple[i] =
>> +                       rte_be_to_cpu_32(*(uint32_t *)&tuple[i * 4]);
>> +
>> +       hash = rte_softrss(tmp_tuple, TUPLE_SZ / 4, new_key);
>> +       RTE_TEST_ASSERT((hash & HASH_MSK(reta_sz)) ==
>> +               desired_value, "bad desired value\n");
>> +
>> +       rte_thash_free_ctx(ctx);
>> +
>> +       return TEST_SUCCESS;
>> +}
> 
>
  

On Thu, Apr 29, 2021 at 12:17:08PM +0300, Medvedkin, Vladimir wrote:
<snip>
> > Test Failed
> > RTE>>
> > --- stderr ---
> > EAL: Detected 2 lcore(s)
> > EAL: Detected 1 NUMA nodes
> > EAL: Detected shared linkage of DPDK
> > EAL: Multi-process socket /var/run/dpdk/thash_autotest/mp_socket
> > EAL: Selected IOVA mode 'PA'
> > EAL: No available 1048576 kB hugepages reported
> > EAL: VFIO support initialized
> > APP: HPET is not enabled, using TSC as default timer
> > HASH: Can't add helper  due to conflict with existing helper second_range
> > HASH: Can't generate m-sequence due to period overflow
> > EAL: Test assert test_adjust_tuple line 559 failed: can not adjust
> > tuple, ret -17
> > 

I can see the same issue on my side. Happening randomly, more often on a
RISC-V target than on my laptop (i5-10210U). Though the reproduction
seems to be a lot of patience and the following:

  meson test --repeat 100000 DPDK:fast-tests / thash_autotest

I wonder if it can be related to the desired_value in test_adjust_tuple
being a randomized value without setting the seed prior to the test?
I haven't analyzed the code in-depth but it seems that the
rte_thash_add_helper() also uses a random lfsr which is then used in the
subkey generation. Could this contribute to the randomness of the issue?
  

Hi Stanislaw,

On 29/04/2021 21H:45, Stanislaw Kardach wrote:
> On Thu, Apr 29, 2021 at 12:17:08PM +0300, Medvedkin, Vladimir wrote:
> <snip>
>>> Test Failed
>>> RTE>>
>>> --- stderr ---
>>> EAL: Detected 2 lcore(s)
>>> EAL: Detected 1 NUMA nodes
>>> EAL: Detected shared linkage of DPDK
>>> EAL: Multi-process socket /var/run/dpdk/thash_autotest/mp_socket
>>> EAL: Selected IOVA mode 'PA'
>>> EAL: No available 1048576 kB hugepages reported
>>> EAL: VFIO support initialized
>>> APP: HPET is not enabled, using TSC as default timer
>>> HASH: Can't add helper  due to conflict with existing helper second_range
>>> HASH: Can't generate m-sequence due to period overflow
>>> EAL: Test assert test_adjust_tuple line 559 failed: can not adjust
>>> tuple, ret -17
>>>
> 
> I can see the same issue on my side. Happening randomly, more often on a
> RISC-V target than on my laptop (i5-10210U). Though the reproduction
> seems to be a lot of patience and the following:
> 
>    meson test --repeat 100000 DPDK:fast-tests / thash_autotest
> 
> I wonder if it can be related to the desired_value in test_adjust_tuple
> being a randomized value without setting the seed prior to the test?
> I haven't analyzed the code in-depth but it seems that the
> rte_thash_add_helper() also uses a random lfsr which is then used in the
> subkey generation. Could this contribute to the randomness of the issue?
> 

The problem here is that the rte_thash_adjust_tuple() function does not 
guarantee that it will find a tuple in a given number of attempts in the 
case when the fn() callback is passed. It depends on random, the logic 
of the fn() callback, and the content of the userdata.

So, in the test we have:
- 96-bit("sizeof(tuple) * CHAR_BITS") tuple
- 16-bit("reta_sz * 2") changeable part of the tuple (i.e. subtuple)
- we want to have a collision in the hash value for 8("reta_sz") least 
significant bits

In other words there are 16 changeable bits inside the subtuple and 8 
least significant bits of them are changed by the 
rte_thash_get_complement() in order to produce collision. The rest 8 
bits are our entropy, i.e. 2^8 different subtuples can produce the 
required collision.

in the problematic call
ret = rte_thash_adjust_tuple(ctx, h, tuple, TUPLE_SZ, desired_value,
                 2, cmp_tuple_eq, tuple_copy);

the original tuple is passed to be changed and the tuple_copy we got 
from the previous invocation. Content of the tuple_copy previously was 
derived from the original tuple applying 8 bit complement on the 
subtuple part.

On the first attempt the function gets the complement and applies it to 
the original tuple, tuple becomes equal to the tuple_copy. After it 
calls the callback and finds that tuple is equal to the tuple_copy. Then 
on the second attempt random bits are xored with 16 bit subtuple value. 
If the first 8 MSBs of random are zeroes, then after applying a new 
complement to the tuple it will be equal to the tuple_copy. While we 
allow only 2 attempts function returns -EEXIST.

I reworked rte_thash_adjust_tuple() removing the randomness. Instead it 
increments the subtuple part which is free of complementary bits.