diff mbox series

[v7] ip_frag: add IPv4 options fragment and test data

Message ID 1649993210-1854-1-git-send-email-chcchc88@163.com (mailing list archive)
State Accepted
Delegated to: Thomas Monjalon
Headers show
Series [v7] ip_frag: add IPv4 options fragment and test data | expand

Checks

Context Check Description
ci/iol-intel-Performance success Performance Testing PASS
ci/iol-intel-Functional success Functional Testing PASS
ci/intel-Testing success Testing PASS
ci/iol-x86_64-unit-testing success Testing PASS
ci/iol-aarch64-unit-testing success Testing PASS
ci/iol-x86_64-compile-testing success Testing PASS
ci/iol-aarch64-compile-testing success Testing PASS
ci/iol-abi-testing success Testing PASS
ci/github-robot: build success github build: passed
ci/iol-mellanox-Performance success Performance Testing PASS
ci/Intel-compilation success Compilation OK
ci/checkpatch success coding style OK

Commit Message

Huichao Cai April 15, 2022, 3:26 a.m. UTC
According to RFC791,the options may appear or not in datagrams.
They must be implemented by all IP modules (host and gateways).
What is optional is their transmission in any particular datagram,
not their implementation.So we have to deal with it during the
fragmenting process.Add some test data for the IPv4 header optional
field fragmenting.

Signed-off-by: Huichao Cai <chcchc88@163.com>
---
 app/test/test_ipfrag.c               | 219 ++++++++++++++++++++++++++++++++---
 lib/ip_frag/rte_ipv4_fragmentation.c |  70 ++++++++++-
 lib/net/rte_ip.h                     |   6 +
 3 files changed, 272 insertions(+), 23 deletions(-)

Comments

Ananyev, Konstantin April 15, 2022, 8:29 a.m. UTC | #1
> According to RFC791,the options may appear or not in datagrams.
> They must be implemented by all IP modules (host and gateways).
> What is optional is their transmission in any particular datagram,
> not their implementation.So we have to deal with it during the
> fragmenting process.Add some test data for the IPv4 header optional
> field fragmenting.
> 
> Signed-off-by: Huichao Cai <chcchc88@163.com>
> ---

Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

> 1.8.3.1
Huichao Cai May 29, 2022, 8:50 a.m. UTC | #2
Hi Konstantin,
This patch has been around for a long time, so what's next?
Huichao,Cai
At 2022-04-15 16:29:10, "Ananyev, Konstantin" <konstantin.ananyev@intel.com> wrote:
>> According to RFC791,the options may appear or not in datagrams.
>> They must be implemented by all IP modules (host and gateways).
>> What is optional is their transmission in any particular datagram,
>> not their implementation.So we have to deal with it during the
>> fragmenting process.Add some test data for the IPv4 header optional
>> field fragmenting.
>> 
>> Signed-off-by: Huichao Cai <chcchc88@163.com>
>> ---
>
>Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
>
>> 1.8.3.1
Huichao Cai May 29, 2022, 8:57 a.m. UTC | #3
Hi Konstantin,
This patch has been around for a long time, so what's next?
Huichao,Cai
At 2022-04-15 16:29:10, "Ananyev, Konstantin" <konstantin.ananyev@intel.com> wrote:
>> According to RFC791,the options may appear or not in datagrams.
>> They must be implemented by all IP modules (host and gateways).
>> What is optional is their transmission in any particular datagram,
>> not their implementation.So we have to deal with it during the
>> fragmenting process.Add some test data for the IPv4 header optional
>> field fragmenting.
>> 
>> Signed-off-by: Huichao Cai <chcchc88@163.com>
>> ---
>
>Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
>
>> 1.8.3.1
Konstantin Ananyev May 29, 2022, 10:38 a.m. UTC | #4
Hi  Huichao,


> Hi Konstantin,
> This patch has been around for a long time, so what's next?

I acked it, which means that I am ok with that patch to go in.
Now it is up to main tree maintainers to pull it in.
Konstantin

> Huichao,Cai
> 
> At 2022-04-15 16:29:10, "Ananyev, Konstantin" <konstantin.ananyev@intel.com> wrote:
>>> According to RFC791,the options may appear or not in datagrams.
>>> They must be implemented by all IP modules (host and gateways).
>>> What is optional is their transmission in any particular datagram,
>>> not their implementation.So we have to deal with it during the
>>> fragmenting process.Add some test data for the IPv4 header optional
>>> field fragmenting.
>>> 
>>> Signed-off-by: Huichao Cai <chcchc88@163.com>
>>> ---
>>
>>Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
>>
>>> 1.8.3.1
> 
> 
> 
> 
>
Thomas Monjalon May 31, 2022, 9:23 p.m. UTC | #5
15/04/2022 10:29, Ananyev, Konstantin:
> > According to RFC791,the options may appear or not in datagrams.
> > They must be implemented by all IP modules (host and gateways).
> > What is optional is their transmission in any particular datagram,
> > not their implementation.So we have to deal with it during the
> > fragmenting process.Add some test data for the IPv4 header optional
> > field fragmenting.
> > 
> > Signed-off-by: Huichao Cai <chcchc88@163.com>
> Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

Applied, thanks.
David Marchand June 16, 2022, 3:10 p.m. UTC | #6
On Fri, Apr 15, 2022 at 5:27 AM Huichao Cai <chcchc88@163.com> wrote:
>
> According to RFC791,the options may appear or not in datagrams.
> They must be implemented by all IP modules (host and gateways).
> What is optional is their transmission in any particular datagram,
> not their implementation.So we have to deal with it during the
> fragmenting process.Add some test data for the IPv4 header optional
> field fragmenting.
>
> Signed-off-by: Huichao Cai <chcchc88@163.com>

gcc-12 raises warnings on both the unit test code and the library code.
See below.

> ---
>  app/test/test_ipfrag.c               | 219 ++++++++++++++++++++++++++++++++---
>  lib/ip_frag/rte_ipv4_fragmentation.c |  70 ++++++++++-
>  lib/net/rte_ip.h                     |   6 +
>  3 files changed, 272 insertions(+), 23 deletions(-)
>
> diff --git a/app/test/test_ipfrag.c b/app/test/test_ipfrag.c
> index 1ced25a..610a86b 100644
> --- a/app/test/test_ipfrag.c
> +++ b/app/test/test_ipfrag.c
> @@ -18,10 +18,50 @@
>  #define NUM_MBUFS 128
>  #define BURST 32
>
> +uint8_t expected_first_frag_ipv4_opts_copied[] = {
> +       0x07, 0x0b, 0x04, 0x00,
> +       0x00, 0x00, 0x00, 0x00,
> +       0x00, 0x00, 0x00, 0x83,
> +       0x07, 0x04, 0xc0, 0xa8,
> +       0xe3, 0x96, 0x00, 0x00,
> +};
> +
> +uint8_t expected_sub_frag_ipv4_opts_copied[] = {
> +       0x83, 0x07, 0x04, 0xc0,
> +       0xa8, 0xe3, 0x96, 0x00,
> +};
> +
> +uint8_t expected_first_frag_ipv4_opts_nocopied[] = {
> +       0x07, 0x0b, 0x04, 0x00,
> +       0x00, 0x00, 0x00, 0x00,
> +       0x00, 0x00, 0x00, 0x00,
> +};
> +
> +uint8_t expected_sub_frag_ipv4_opts_nocopied[0];
> +
> +struct test_opt_data {
> +       bool is_first_frag;              /**< offset is 0 */
> +       bool opt_copied;                 /**< ip option copied flag */
> +       uint16_t len;                    /**< option data len */
> +       uint8_t data[RTE_IPV4_HDR_OPT_MAX_LEN]; /**< option data */
> +};
> +
>  static struct rte_mempool *pkt_pool,
>                           *direct_pool,
>                           *indirect_pool;
>
> +static inline void
> +hex_to_str(uint8_t *hex, uint16_t len, char *str)
> +{
> +       int i;
> +
> +       for (i = 0; i < len; i++) {
> +               sprintf(str, "%02x", hex[i]);
> +               str += 2;
> +       }
> +       *str = 0;
> +}
> +
>  static int
>  setup_buf_pool(void)
>  {
> @@ -88,23 +128,67 @@ static void ut_teardown(void)
>  {
>  }
>
> +static inline void
> +test_get_ipv4_opt(bool is_first_frag, bool opt_copied,
> +       struct test_opt_data *expected_opt)
> +{
> +       if (is_first_frag) {
> +               if (opt_copied) {
> +                       expected_opt->len =
> +                               sizeof(expected_first_frag_ipv4_opts_copied);
> +                       rte_memcpy(expected_opt->data,
> +                               expected_first_frag_ipv4_opts_copied,
> +                               sizeof(expected_first_frag_ipv4_opts_copied));
> +               } else {
> +                       expected_opt->len =
> +                               sizeof(expected_first_frag_ipv4_opts_nocopied);
> +                       rte_memcpy(expected_opt->data,
> +                               expected_first_frag_ipv4_opts_nocopied,
> +                               sizeof(expected_first_frag_ipv4_opts_nocopied));
> +               }
> +       } else {
> +               if (opt_copied) {
> +                       expected_opt->len =
> +                               sizeof(expected_sub_frag_ipv4_opts_copied);
> +                       rte_memcpy(expected_opt->data,
> +                               expected_sub_frag_ipv4_opts_copied,
> +                               sizeof(expected_sub_frag_ipv4_opts_copied));
> +               } else {
> +                       expected_opt->len =
> +                               sizeof(expected_sub_frag_ipv4_opts_nocopied);
> +                       rte_memcpy(expected_opt->data,
> +                               expected_sub_frag_ipv4_opts_nocopied,
> +                               sizeof(expected_sub_frag_ipv4_opts_nocopied));
> +               }
> +       }
> +}
> +
>  static void
> -v4_allocate_packet_of(struct rte_mbuf *b, int fill,
> -                     size_t s, int df, uint8_t mf, uint16_t off,
> -                     uint8_t ttl, uint8_t proto, uint16_t pktid)
> +v4_allocate_packet_of(struct rte_mbuf *b, int fill, size_t s,
> +       int df, uint8_t mf, uint16_t off, uint8_t ttl, uint8_t proto,
> +       uint16_t pktid, bool have_opt, bool is_first_frag, bool opt_copied)
>  {
>         /* Create a packet, 2k bytes long */
>         b->data_off = 0;
>         char *data = rte_pktmbuf_mtod(b, char *);
> -       rte_be16_t fragment_offset = 0; /**< fragmentation offset */
> +       rte_be16_t fragment_offset = 0; /* fragmentation offset */
> +       uint16_t iph_len;
> +       struct test_opt_data opt;
> +
> +       opt.len = 0;
> +
> +       if (have_opt)
> +               test_get_ipv4_opt(is_first_frag, opt_copied, &opt);


FAILED: app/test/dpdk-test.p/test_ipfrag.c.o
ccache gcc -Iapp/test/dpdk-test.p -Iapp/test -I../app/test -I. -I..
-Iconfig -I../config -Ilib/eal/include -I../lib/eal/include
-Ilib/eal/linux/include -I../lib/eal/linux/include
-Ilib/eal/x86/include -I../lib/eal/x86/include -Ilib/kvargs
-I../lib/kvargs -Ilib/metrics -I../lib/metrics -Ilib/telemetry
-I../lib/telemetry -Ilib/eal/common -I../lib/eal/common -Ilib/eal
-I../lib/eal -Ilib/ring -I../lib/ring -Ilib/rcu -I../lib/rcu
-Ilib/mempool -I../lib/mempool -Ilib/mbuf -I../lib/mbuf -Ilib/net
-I../lib/net -Ilib/meter -I../lib/meter -Ilib/ethdev -I../lib/ethdev
-Ilib/pci -I../lib/pci -Ilib/cmdline -I../lib/cmdline -Ilib/hash
-I../lib/hash -Ilib/timer -I../lib/timer -Ilib/acl -I../lib/acl
-Ilib/bbdev -I../lib/bbdev -Ilib/bitratestats -I../lib/bitratestats
-Ilib/bpf -I../lib/bpf -Ilib/cfgfile -I../lib/cfgfile
-Ilib/compressdev -I../lib/compressdev -Ilib/cryptodev
-I../lib/cryptodev -Ilib/distributor -I../lib/distributor -Ilib/efd
-I../lib/efd -Ilib/eventdev -I../lib/eventdev -Ilib/gpudev
-I../lib/gpudev -Ilib/gro -I../lib/gro -Ilib/gso -I../lib/gso
-Ilib/ip_frag -I../lib/ip_frag -Ilib/jobstats -I../lib/jobstats
-Ilib/kni -I../lib/kni -Ilib/latencystats -I../lib/latencystats
-Ilib/lpm -I../lib/lpm -Ilib/member -I../lib/member -Ilib/pcapng
-I../lib/pcapng -Ilib/power -I../lib/power -Ilib/rawdev
-I../lib/rawdev -Ilib/regexdev -I../lib/regexdev -Ilib/dmadev
-I../lib/dmadev -Ilib/rib -I../lib/rib -Ilib/reorder -I../lib/reorder
-Ilib/sched -I../lib/sched -Ilib/security -I../lib/security
-Ilib/stack -I../lib/stack -Ilib/vhost -I../lib/vhost -Ilib/ipsec
-I../lib/ipsec -Ilib/fib -I../lib/fib -Ilib/port -I../lib/port
-Ilib/pdump -I../lib/pdump -Ilib/table -I../lib/table -Ilib/pipeline
-I../lib/pipeline -Ilib/flow_classify -I../lib/flow_classify
-Ilib/graph -I../lib/graph -Ilib/node -I../lib/node -Idrivers/bus/pci
-I../drivers/bus/pci -I../drivers/bus/pci/linux -Idrivers/bus/vdev
-I../drivers/bus/vdev -Idrivers/mempool/ring -I../drivers/mempool/ring
-Idrivers/mempool/stack -I../drivers/mempool/stack
-Idrivers/event/skeleton -I../drivers/event/skeleton
-Idrivers/net/bonding -I../drivers/net/bonding -Idrivers/net/ring
-I../drivers/net/ring -Idrivers/net/null -I../drivers/net/null
-Idrivers/crypto/scheduler -I../drivers/crypto/scheduler
-fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch
-Wextra -Werror -O3 -include rte_config.h -Wcast-qual -Wdeprecated
-Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations
-Wmissing-prototypes -Wnested-externs -Wold-style-definition
-Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef
-Wwrite-strings -Wno-address-of-packed-member -Wno-packed-not-aligned
-Wno-missing-field-initializers -Wno-zero-length-bounds -D_GNU_SOURCE
-march=native -DALLOW_EXPERIMENTAL_API -Wno-format-truncation
-fno-strict-aliasing -DALLOW_INTERNAL_API -MD -MQ
app/test/dpdk-test.p/test_ipfrag.c.o -MF
app/test/dpdk-test.p/test_ipfrag.c.o.d -o
app/test/dpdk-test.p/test_ipfrag.c.o -c ../app/test/test_ipfrag.c
In file included from
/usr/lib/gcc/x86_64-redhat-linux/12/include/immintrin.h:43,
                 from
/usr/lib/gcc/x86_64-redhat-linux/12/include/x86intrin.h:32,
                 from ../lib/eal/x86/include/rte_vect.h:31,
                 from ../lib/eal/x86/include/rte_memcpy.h:17,
                 from ../lib/mempool/rte_mempool.h:46,
                 from ../lib/mbuf/rte_mbuf.h:38,
                 from ../lib/net/rte_ip.h:32,
                 from ../app/test/test_ipfrag.c:12:
In function ‘_mm256_loadu_si256’,
    inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:346:9,
    inlined from ‘rte_mov128’ at ../lib/eal/x86/include/rte_memcpy.h:369:2,
    inlined from ‘rte_memcpy_generic’ at
../lib/eal/x86/include/rte_memcpy.h:445:4,
    inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:851:10,
    inlined from ‘v4_allocate_packet_of’ at ../app/test/test_ipfrag.c:230:2,
    inlined from ‘test_ip_frag’ at ../app/test/test_ipfrag.c:402:4:
/usr/lib/gcc/x86_64-redhat-linux/12/include/avxintrin.h:929:10: error:
array subscript ‘__m256i_u[1]’ is partly outside array bounds of
‘struct test_opt_data[1]’ [-Werror=array-bounds]
  929 |   return *__P;
      |          ^~~~
../app/test/test_ipfrag.c: In function ‘test_ip_frag’:
../app/test/test_ipfrag.c:187:30: note: at offset 36 into object ‘opt’
of size 44
  187 |         struct test_opt_data opt;
      |                              ^~~
In function ‘_mm256_loadu_si256’,
    inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:346:9,
    inlined from ‘rte_mov128’ at ../lib/eal/x86/include/rte_memcpy.h:370:2,
    inlined from ‘rte_memcpy_generic’ at
../lib/eal/x86/include/rte_memcpy.h:445:4,
    inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:851:10,
    inlined from ‘v4_allocate_packet_of’ at ../app/test/test_ipfrag.c:230:2,
    inlined from ‘test_ip_frag’ at ../app/test/test_ipfrag.c:402:4:
/usr/lib/gcc/x86_64-redhat-linux/12/include/avxintrin.h:929:10: error:
array subscript 2 is outside array bounds of ‘struct test_opt_data[1]’
[-Werror=array-bounds]
  929 |   return *__P;
      |          ^~~~
../app/test/test_ipfrag.c: In function ‘test_ip_frag’:
../app/test/test_ipfrag.c:187:30: note: at offset 68 into object ‘opt’
of size 44
  187 |         struct test_opt_data opt;
      |                              ^~~
In function ‘_mm256_loadu_si256’,
    inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:346:9,
    inlined from ‘rte_mov128’ at ../lib/eal/x86/include/rte_memcpy.h:371:2,
    inlined from ‘rte_memcpy_generic’ at
../lib/eal/x86/include/rte_memcpy.h:445:4,
    inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:851:10,
    inlined from ‘v4_allocate_packet_of’ at ../app/test/test_ipfrag.c:230:2,
    inlined from ‘test_ip_frag’ at ../app/test/test_ipfrag.c:402:4:
/usr/lib/gcc/x86_64-redhat-linux/12/include/avxintrin.h:929:10: error:
array subscript 3 is outside array bounds of ‘struct test_opt_data[1]’
[-Werror=array-bounds]
  929 |   return *__P;
      |          ^~~~
../app/test/test_ipfrag.c: In function ‘test_ip_frag’:
../app/test/test_ipfrag.c:187:30: note: at offset 100 into object
‘opt’ of size 44
  187 |         struct test_opt_data opt;
      |                              ^~~
In function ‘_mm256_loadu_si256’,
    inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:346:9,
    inlined from ‘rte_mov64’ at ../lib/eal/x86/include/rte_memcpy.h:358:2,
    inlined from ‘rte_memcpy_generic’ at
../lib/eal/x86/include/rte_memcpy.h:452:4,
    inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:851:10,
    inlined from ‘v4_allocate_packet_of’ at ../app/test/test_ipfrag.c:230:2,
    inlined from ‘test_ip_frag’ at ../app/test/test_ipfrag.c:402:4:
/usr/lib/gcc/x86_64-redhat-linux/12/include/avxintrin.h:929:10: error:
array subscript ‘__m256i_u[1]’ is partly outside array bounds of
‘const void[44]’ [-Werror=array-bounds]
  929 |   return *__P;
      |          ^~~~
../app/test/test_ipfrag.c: In function ‘test_ip_frag’:
../app/test/test_ipfrag.c:57:17: note: at offset 36 into object ‘data’
of size 40
   57 |         uint8_t data[RTE_IPV4_HDR_OPT_MAX_LEN]; /**< option data */
      |                 ^~~~
../app/test/test_ipfrag.c:57:17: note: at offset [37, 40] into object
‘data’ of size 40
../app/test/test_ipfrag.c:187:30: note: at offset 168 into object
‘opt’ of size 44
  187 |         struct test_opt_data opt;
      |                              ^~~
../app/test/test_ipfrag.c:57:17: note: at offset 36 into object ‘data’
of size 40
   57 |         uint8_t data[RTE_IPV4_HDR_OPT_MAX_LEN]; /**< option data */
      |                 ^~~~
In function ‘_mm256_loadu_si256’,
    inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:346:9,
    inlined from ‘rte_memcpy_generic’ at
../lib/eal/x86/include/rte_memcpy.h:457:4,
    inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:851:10,
    inlined from ‘v4_allocate_packet_of’ at ../app/test/test_ipfrag.c:230:2,
    inlined from ‘test_ip_frag’ at ../app/test/test_ipfrag.c:402:4:
/usr/lib/gcc/x86_64-redhat-linux/12/include/avxintrin.h:929:10: error:
array subscript [2, 2051] is outside array bounds of ‘const void[44]’
[-Werror=array-bounds]
  929 |   return *__P;
      |          ^~~~
../app/test/test_ipfrag.c: In function ‘test_ip_frag’:
../app/test/test_ipfrag.c:57:17: note: at offset [4, 40] into object
‘data’ of size 40
   57 |         uint8_t data[RTE_IPV4_HDR_OPT_MAX_LEN]; /**< option data */
      |                 ^~~~
../app/test/test_ipfrag.c:57:17: note: at offset [5, 40] into object
‘data’ of size 40
../app/test/test_ipfrag.c:187:30: note: at offset [136, 200] into
object ‘opt’ of size 44
  187 |         struct test_opt_data opt;
      |                              ^~~
../app/test/test_ipfrag.c:57:17: note: at offset [4, 40] into object
‘data’ of size 40
   57 |         uint8_t data[RTE_IPV4_HDR_OPT_MAX_LEN]; /**< option data */
      |                 ^~~~
../app/test/test_ipfrag.c:57:17: note: at offset [5, 40] into object
‘data’ of size 40
../app/test/test_ipfrag.c:187:30: note: at offset [136, 200] into
object ‘opt’ of size 44
  187 |         struct test_opt_data opt;
      |                              ^~~
../app/test/test_ipfrag.c:57:17: note: at offset [4, 40] into object
‘data’ of size 40
   57 |         uint8_t data[RTE_IPV4_HDR_OPT_MAX_LEN]; /**< option data */
      |                 ^~~~
In function ‘_mm256_loadu_si256’,
    inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:346:9,
    inlined from ‘rte_memcpy_generic’ at
../lib/eal/x86/include/rte_memcpy.h:458:4,
    inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:851:10,
    inlined from ‘v4_allocate_packet_of’ at ../app/test/test_ipfrag.c:230:2,
    inlined from ‘test_ip_frag’ at ../app/test/test_ipfrag.c:402:4:
/usr/lib/gcc/x86_64-redhat-linux/12/include/avxintrin.h:929:10: error:
array subscript [2, 2052] is outside array bounds of ‘const void[44]’
[-Werror=array-bounds]
  929 |   return *__P;
      |          ^~~~
../app/test/test_ipfrag.c: In function ‘test_ip_frag’:
../app/test/test_ipfrag.c:57:17: note: at offset [5, 40] into object
‘data’ of size 40
   57 |         uint8_t data[RTE_IPV4_HDR_OPT_MAX_LEN]; /**< option data */
      |                 ^~~~
../app/test/test_ipfrag.c:57:17: note: at offset [6, 40] into object
‘data’ of size 40
../app/test/test_ipfrag.c:187:30: note: at offset [137, 201] into
object ‘opt’ of size 44
  187 |         struct test_opt_data opt;
      |                              ^~~
../app/test/test_ipfrag.c:57:17: note: at offset [5, 40] into object
‘data’ of size 40
   57 |         uint8_t data[RTE_IPV4_HDR_OPT_MAX_LEN]; /**< option data */
      |                 ^~~~
../app/test/test_ipfrag.c:57:17: note: at offset [6, 40] into object
‘data’ of size 40
../app/test/test_ipfrag.c:187:30: note: at offset [137, 201] into
object ‘opt’ of size 44
  187 |         struct test_opt_data opt;
      |                              ^~~
../app/test/test_ipfrag.c:57:17: note: at offset [5, 40] into object
‘data’ of size 40
   57 |         uint8_t data[RTE_IPV4_HDR_OPT_MAX_LEN]; /**< option data */
      |                 ^~~~
In function ‘_mm256_loadu_si256’,
    inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:346:9,
    inlined from ‘rte_memcpy_generic’ at
../lib/eal/x86/include/rte_memcpy.h:438:3,
    inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:851:10,
    inlined from ‘v4_allocate_packet_of’ at ../app/test/test_ipfrag.c:230:2,
    inlined from ‘test_ip_frag’ at ../app/test/test_ipfrag.c:402:4:
/usr/lib/gcc/x86_64-redhat-linux/12/include/avxintrin.h:929:10: error:
array subscript ‘__m256i_u[0]’ is partly outside array bounds of
‘struct test_opt_data[1]’ [-Werror=array-bounds]
  929 |   return *__P;
      |          ^~~~
../app/test/test_ipfrag.c: In function ‘test_ip_frag’:
../app/test/test_ipfrag.c:187:30: note: at offset [21, 36] into object
‘opt’ of size 44
  187 |         struct test_opt_data opt;
      |                              ^~~
cc1: all warnings being treated as errors
ninja: build stopped: subcommand failed.


>
> -       memset(data, fill, sizeof(struct rte_ipv4_hdr) + s);
> +       iph_len = sizeof(struct rte_ipv4_hdr) + opt.len;
> +       memset(data, fill, iph_len + s);
>
>         struct rte_ipv4_hdr *hdr = (struct rte_ipv4_hdr *)data;
>
> -       hdr->version_ihl = 0x45; /* standard IP header... */
> +       hdr->version_ihl = 0x40; /* ipv4 */
> +       hdr->version_ihl += (iph_len / 4);
>         hdr->type_of_service = 0;
> -       b->pkt_len = s + sizeof(struct rte_ipv4_hdr);
> +       b->pkt_len = s + iph_len;
>         b->data_len = b->pkt_len;
>         hdr->total_length = rte_cpu_to_be_16(b->pkt_len);
>         hdr->packet_id = rte_cpu_to_be_16(pktid);
> @@ -131,6 +215,8 @@ static void ut_teardown(void)
>         hdr->hdr_checksum = 0;
>         hdr->src_addr = rte_cpu_to_be_32(0x8080808);
>         hdr->dst_addr = rte_cpu_to_be_32(0x8080404);
> +
> +       rte_memcpy(hdr + 1, opt.data, opt.len);
>  }
>
>  static void
> @@ -187,6 +273,45 @@ static void ut_teardown(void)
>         }
>  }
>
> +static inline void
> +test_get_frag_opt(struct rte_mbuf **mb, int32_t num,
> +       struct test_opt_data *opt, int ipv, bool opt_copied)
> +{
> +       int32_t i;
> +
> +       for (i = 0; i < num; i++) {
> +               if (ipv == 4) {
> +                       struct rte_ipv4_hdr *iph =
> +                           rte_pktmbuf_mtod(mb[i], struct rte_ipv4_hdr *);
> +                       uint16_t header_len = (iph->version_ihl &
> +                               RTE_IPV4_HDR_IHL_MASK) *
> +                               RTE_IPV4_IHL_MULTIPLIER;
> +                       uint16_t opt_len = header_len -
> +                               sizeof(struct rte_ipv4_hdr);
> +
> +                       opt->opt_copied = opt_copied;
> +
> +                       if ((rte_be_to_cpu_16(iph->fragment_offset) &
> +                                   RTE_IPV4_HDR_OFFSET_MASK) == 0)
> +                               opt->is_first_frag = true;
> +                       else
> +                               opt->is_first_frag = false;
> +
> +                       if (likely(opt_len <= RTE_IPV4_HDR_OPT_MAX_LEN)) {
> +                               char *iph_opt = rte_pktmbuf_mtod_offset(mb[i],
> +                                   char *, sizeof(struct rte_ipv4_hdr));
> +                               opt->len = opt_len;
> +                               rte_memcpy(opt->data, iph_opt, opt_len);
> +                       } else {
> +                               opt->len = RTE_IPV4_HDR_OPT_MAX_LEN;
> +                               memset(opt->data, RTE_IPV4_HDR_OPT_EOL,
> +                                   sizeof(opt->data));
> +                       }
> +                       opt++;
> +               }
> +       }
> +}
> +
>  static int
>  test_ip_frag(void)
>  {
> @@ -206,32 +331,52 @@ static void ut_teardown(void)
>                 uint16_t pkt_id;
>                 int      expected_frags;
>                 uint16_t expected_fragment_offset[BURST];
> +               bool have_opt;
> +               bool is_first_frag;
> +               bool opt_copied;
>         } tests[] = {
>                  {4, 1280, 1400, 0, 0, 0, 64, IPPROTO_ICMP, RND_ID,       2,
> -                 {0x2000, 0x009D}},
> +                 {0x2000, 0x009D}, false},
>                  {4, 1280, 1400, 0, 0, 0, 64, IPPROTO_ICMP, 0,            2,
> -                 {0x2000, 0x009D}},
> +                 {0x2000, 0x009D}, false},
>                  {4,  600, 1400, 0, 0, 0, 64, IPPROTO_ICMP, RND_ID,       3,
> -                 {0x2000, 0x2048, 0x0090}},
> +                 {0x2000, 0x2048, 0x0090}, false},
>                  {4, 4, 1400, 0, 0, 0, 64, IPPROTO_ICMP, RND_ID,    -EINVAL},
>                  {4, 600, 1400, 1, 0, 0, 64, IPPROTO_ICMP, RND_ID, -ENOTSUP},
>                  {4, 600, 1400, 0, 0, 0, 0, IPPROTO_ICMP, RND_ID,         3,
> -                 {0x2000, 0x2048, 0x0090}},
> +                 {0x2000, 0x2046, 0x008C}, true, true, true},
> +                /* The first fragment */
> +                {4, 68, 104, 0, 1, 0, 0, IPPROTO_ICMP, RND_ID,           5,
> +                 {0x2000, 0x2003, 0x2006, 0x2009, 0x200C}, true, true, true},
> +                /* The middle fragment */
>                  {4, 68, 104, 0, 1, 13, 0, IPPROTO_ICMP, RND_ID,          3,
> -                 {0x200D, 0x2013, 0x2019}},
> -
> +                 {0x200D, 0x2012, 0x2017}, true, false, true},
> +                /* The last fragment */
> +                {4, 68, 104, 0, 0, 26, 0, IPPROTO_ICMP, RND_ID,          3,
> +                 {0x201A, 0x201F, 0x0024}, true, false, true},
> +                /* The first fragment */
> +                {4, 68, 104, 0, 1, 0, 0, IPPROTO_ICMP, RND_ID,           4,
> +                 {0x2000, 0x2004, 0x2008, 0x200C}, true, true, false},
> +                /* The middle fragment */
> +                {4, 68, 104, 0, 1, 13, 0, IPPROTO_ICMP, RND_ID,          3,
> +                 {0x200D, 0x2013, 0x2019}, true, false, false},
> +                /* The last fragment */
> +                {4, 68, 104, 0, 0, 26, 0, IPPROTO_ICMP, RND_ID,          3,
> +                 {0x201A, 0x2020, 0x0026}, true, false, false},
>                  {6, 1280, 1400, 0, 0, 0, 64, IPPROTO_ICMP, RND_ID,       2,
> -                 {0x0001, 0x04D0}},
> +                 {0x0001, 0x04D0}, false},
>                  {6, 1300, 1400, 0, 0, 0, 64, IPPROTO_ICMP, RND_ID,       2,
> -                 {0x0001, 0x04E0}},
> +                 {0x0001, 0x04E0}, false},
>                  {6, 4, 1400, 0, 0, 0, 64, IPPROTO_ICMP, RND_ID,    -EINVAL},
>                  {6, 1300, 1400, 0, 0, 0, 0, IPPROTO_ICMP, RND_ID,        2,
> -                 {0x0001, 0x04E0}},
> +                 {0x0001, 0x04E0}, false},
>         };
>
>         for (i = 0; i < RTE_DIM(tests); i++) {
>                 int32_t len = 0;
>                 uint16_t fragment_offset[BURST];
> +               struct test_opt_data opt_res[BURST];
> +               struct test_opt_data opt_exp;
>                 uint16_t pktid = tests[i].pkt_id;
>                 struct rte_mbuf *pkts_out[BURST];
>                 struct rte_mbuf *b = rte_pktmbuf_alloc(pkt_pool);
> @@ -250,7 +395,10 @@ static void ut_teardown(void)
>                                               tests[i].set_of,
>                                               tests[i].ttl,
>                                               tests[i].proto,
> -                                             pktid);
> +                                             pktid,
> +                                             tests[i].have_opt,
> +                                             tests[i].is_first_frag,
> +                                             tests[i].opt_copied);
>                 } else if (tests[i].ipv == 6) {
>                         v6_allocate_packet_of(b, 0x41414141,
>                                               tests[i].pkt_size,
> @@ -275,17 +423,20 @@ static void ut_teardown(void)
>                 if (len > 0) {
>                         test_get_offset(pkts_out, len,
>                             fragment_offset, tests[i].ipv);
> +                       if (tests[i].have_opt)
> +                               test_get_frag_opt(pkts_out, len, opt_res,
> +                                       tests[i].ipv, tests[i].opt_copied);
>                         test_free_fragments(pkts_out, len);
>                 }
>
> -               printf("%zd: checking %d with %d\n", i, len,
> +               printf("[check frag number]%zd: checking %d with %d\n", i, len,
>                        tests[i].expected_frags);
>                 RTE_TEST_ASSERT_EQUAL(len, tests[i].expected_frags,
>                                       "Failed case %zd.\n", i);
>
>                 if (len > 0) {
>                         for (j = 0; j < (size_t)len; j++) {
> -                               printf("%zd-%zd: checking %d with %d\n",
> +                               printf("[check offset]%zd-%zd: checking %d with %d\n",
>                                     i, j, fragment_offset[j],
>                                     rte_cpu_to_be_16(
>                                         tests[i].expected_fragment_offset[j]));
> @@ -294,6 +445,36 @@ static void ut_teardown(void)
>                                         tests[i].expected_fragment_offset[j]),
>                                     "Failed case %zd.\n", i);
>                         }
> +
> +                       if (tests[i].have_opt && (tests[i].ipv == 4)) {
> +                               for (j = 0; j < (size_t)len; j++) {
> +                                       char opt_res_str[2 *
> +                                               RTE_IPV4_HDR_OPT_MAX_LEN + 1];
> +                                       char opt_exp_str[2 *
> +                                               RTE_IPV4_HDR_OPT_MAX_LEN + 1];
> +
> +                                       test_get_ipv4_opt(
> +                                               opt_res[j].is_first_frag,
> +                                               opt_res[j].opt_copied,
> +                                               &opt_exp);
> +                                       hex_to_str(opt_res[j].data,
> +                                               opt_res[j].len,
> +                                               opt_res_str);
> +                                       hex_to_str(opt_exp.data,
> +                                               opt_exp.len,
> +                                               opt_exp_str);
> +
> +                                       printf(
> +                                               "[check ipv4 option]%zd-%zd: checking (len:%u)%s with (len:%u)%s\n",
> +                                               i, j,
> +                                               opt_res[j].len, opt_res_str,
> +                                               opt_exp.len, opt_exp_str);
> +                                               RTE_TEST_ASSERT_SUCCESS(
> +                                                       strcmp(opt_res_str,
> +                                                               opt_exp_str),
> +                                               "Failed case %zd.\n", i);
> +                               }
> +                       }
>                 }
>
>         }
> diff --git a/lib/ip_frag/rte_ipv4_fragmentation.c b/lib/ip_frag/rte_ipv4_fragmentation.c
> index 2e7739d..a562424 100644
> --- a/lib/ip_frag/rte_ipv4_fragmentation.c
> +++ b/lib/ip_frag/rte_ipv4_fragmentation.c
> @@ -22,6 +22,8 @@
>
>  #define        IPV4_HDR_FO_ALIGN                       (1 << RTE_IPV4_HDR_FO_SHIFT)
>
> +#define IPV4_HDR_MAX_LEN                       60
> +
>  static inline void __fill_ipv4hdr_frag(struct rte_ipv4_hdr *dst,
>                 const struct rte_ipv4_hdr *src, uint16_t header_len,
>                 uint16_t len, uint16_t fofs, uint16_t dofs, uint32_t mf)
> @@ -41,6 +43,49 @@ static inline void __free_fragments(struct rte_mbuf *mb[], uint32_t num)
>                 rte_pktmbuf_free(mb[i]);
>  }
>
> +static inline uint16_t __create_ipopt_frag_hdr(uint8_t *iph,
> +       uint16_t ipopt_len, uint8_t *ipopt_frag_hdr)
> +{
> +       uint16_t len = ipopt_len;
> +       struct rte_ipv4_hdr *iph_opt = (struct rte_ipv4_hdr *)ipopt_frag_hdr;
> +
> +       ipopt_len = 0;
> +       rte_memcpy(ipopt_frag_hdr, iph, sizeof(struct rte_ipv4_hdr));
> +       ipopt_frag_hdr += sizeof(struct rte_ipv4_hdr);
> +
> +       uint8_t *p_opt = iph + sizeof(struct rte_ipv4_hdr);
> +
> +       while (len > 0) {
> +               if (unlikely(*p_opt == RTE_IPV4_HDR_OPT_NOP)) {
> +                       len--;
> +                       p_opt++;
> +                       continue;
> +               } else if (unlikely(*p_opt == RTE_IPV4_HDR_OPT_EOL))
> +                       break;
> +
> +               if (unlikely(p_opt[1] < 2 || p_opt[1] > len))
> +                       break;
> +
> +               if (RTE_IPV4_HDR_OPT_COPIED(*p_opt)) {
> +                       rte_memcpy(ipopt_frag_hdr + ipopt_len,
> +                               p_opt, p_opt[1]);
> +                       ipopt_len += p_opt[1];
> +               }
> +
> +               len -= p_opt[1];
> +               p_opt += p_opt[1];
> +       }
> +
> +       len = RTE_ALIGN_CEIL(ipopt_len, RTE_IPV4_IHL_MULTIPLIER);
> +       memset(ipopt_frag_hdr + ipopt_len,
> +               RTE_IPV4_HDR_OPT_EOL, len - ipopt_len);
> +       ipopt_len = len;
> +       iph_opt->ihl = (sizeof(struct rte_ipv4_hdr) + ipopt_len) /
> +               RTE_IPV4_IHL_MULTIPLIER;
> +
> +       return ipopt_len;
> +}
> +
>  /**
>   * IPv4 fragmentation.
>   *
> @@ -76,6 +121,8 @@ static inline void __free_fragments(struct rte_mbuf *mb[], uint32_t num)
>         uint32_t more_in_segs;
>         uint16_t fragment_offset, flag_offset, frag_size, header_len;
>         uint16_t frag_bytes_remaining;
> +       uint8_t ipopt_frag_hdr[IPV4_HDR_MAX_LEN];
> +       uint16_t ipopt_len;
>
>         /*
>          * Formal parameter checking.
> @@ -118,6 +165,10 @@ static inline void __free_fragments(struct rte_mbuf *mb[], uint32_t num)
>         out_pkt_pos = 0;
>         fragment_offset = 0;
>
> +       ipopt_len = header_len - sizeof(struct rte_ipv4_hdr);
> +       if (unlikely(ipopt_len > RTE_IPV4_HDR_OPT_MAX_LEN))
> +               return -EINVAL;
> +
>         more_in_segs = 1;
>         while (likely(more_in_segs)) {
>                 struct rte_mbuf *out_pkt = NULL, *out_seg_prev = NULL;
> @@ -188,10 +239,21 @@ static inline void __free_fragments(struct rte_mbuf *mb[], uint32_t num)
>                     (uint16_t)out_pkt->pkt_len,
>                     flag_offset, fragment_offset, more_in_segs);
>
> -               fragment_offset = (uint16_t)(fragment_offset +
> -                   out_pkt->pkt_len - header_len);
> -
> -               out_pkt->l3_len = header_len;
> +               if (unlikely((fragment_offset == 0) && (ipopt_len) &&
> +                           ((flag_offset & RTE_IPV4_HDR_OFFSET_MASK) == 0))) {
> +                       ipopt_len = __create_ipopt_frag_hdr((uint8_t *)in_hdr,
> +                               ipopt_len, ipopt_frag_hdr);
> +                       fragment_offset = (uint16_t)(fragment_offset +
> +                               out_pkt->pkt_len - header_len);
> +                       out_pkt->l3_len = header_len;
> +
> +                       header_len = sizeof(struct rte_ipv4_hdr) + ipopt_len;
> +                       in_hdr = (struct rte_ipv4_hdr *)ipopt_frag_hdr;
> +               } else {
> +                       fragment_offset = (uint16_t)(fragment_offset +
> +                               out_pkt->pkt_len - header_len);
> +                       out_pkt->l3_len = header_len;
> +               }

FAILED: lib/librte_ip_frag.a.p/ip_frag_rte_ipv4_fragmentation.c.o
ccache gcc -Ilib/librte_ip_frag.a.p -Ilib -I../lib -Ilib/ip_frag
-I../lib/ip_frag -I. -I.. -Iconfig -I../config -Ilib/eal/include
-I../lib/eal/include -Ilib/eal/linux/include
-I../lib/eal/linux/include -Ilib/eal/x86/include
-I../lib/eal/x86/include -Ilib/eal/common -I../lib/eal/common
-Ilib/eal -I../lib/eal -Ilib/kvargs -I../lib/kvargs -Ilib/metrics
-I../lib/metrics -Ilib/telemetry -I../lib/telemetry -Ilib/ethdev
-I../lib/ethdev -Ilib/net -I../lib/net -Ilib/mbuf -I../lib/mbuf
-Ilib/mempool -I../lib/mempool -Ilib/ring -I../lib/ring -Ilib/meter
-I../lib/meter -Ilib/hash -I../lib/hash -Ilib/rcu -I../lib/rcu
-fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch
-Wextra -Werror -O3 -g -include rte_config.h -Wcast-qual -Wdeprecated
-Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations
-Wmissing-prototypes -Wnested-externs -Wold-style-definition
-Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef
-Wwrite-strings -Wno-address-of-packed-member -Wno-packed-not-aligned
-Wno-missing-field-initializers -Wno-zero-length-bounds -D_GNU_SOURCE
-fPIC -march=native -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API
-Wno-format-truncation -DRTE_LOG_DEFAULT_LOGTYPE=lib.ip_frag -MD -MQ
lib/librte_ip_frag.a.p/ip_frag_rte_ipv4_fragmentation.c.o -MF
lib/librte_ip_frag.a.p/ip_frag_rte_ipv4_fragmentation.c.o.d -o
lib/librte_ip_frag.a.p/ip_frag_rte_ipv4_fragmentation.c.o -c
../lib/ip_frag/rte_ipv4_fragmentation.c
In file included from
/usr/lib/gcc/x86_64-redhat-linux/12/include/immintrin.h:43,
                 from
/usr/lib/gcc/x86_64-redhat-linux/12/include/x86intrin.h:32,
                 from ../lib/eal/x86/include/rte_vect.h:31,
                 from ../lib/eal/x86/include/rte_memcpy.h:17,
                 from ../lib/ip_frag/rte_ipv4_fragmentation.c:8:
In function ‘_mm256_storeu_si256’,
    inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:347:2,
    inlined from ‘rte_mov128’ at ../lib/eal/x86/include/rte_memcpy.h:369:2,
    inlined from ‘rte_memcpy_generic’ at
../lib/eal/x86/include/rte_memcpy.h:445:4,
    inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:851:10,
    inlined from ‘__create_ipopt_frag_hdr’ at
../lib/ip_frag/rte_ipv4_fragmentation.c:68:4,
    inlined from ‘rte_ipv4_fragment_packet’ at
../lib/ip_frag/rte_ipv4_fragmentation.c:242:16:
/usr/lib/gcc/x86_64-redhat-linux/12/include/avxintrin.h:935:8: error:
array subscript ‘__m256i_u[1]’ is partly outside array bounds of
‘uint8_t[60]’ {aka ‘unsigned char[60]’} [-Werror=array-bounds]
  935 |   *__P = __A;
      |   ~~~~~^~~~~
../lib/ip_frag/rte_ipv4_fragmentation.c: In function ‘rte_ipv4_fragment_packet’:
../lib/ip_frag/rte_ipv4_fragmentation.c:122:17: note: at offset [52,
60] into object ‘ipopt_frag_hdr’ of size 60
  122 |         uint8_t ipopt_frag_hdr[IPV4_HDR_MAX_LEN];
      |                 ^~~~~~~~~~~~~~
In function ‘_mm256_storeu_si256’,
    inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:347:2,
    inlined from ‘rte_mov128’ at ../lib/eal/x86/include/rte_memcpy.h:370:2,
    inlined from ‘rte_memcpy_generic’ at
../lib/eal/x86/include/rte_memcpy.h:445:4,
    inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:851:10,
    inlined from ‘__create_ipopt_frag_hdr’ at
../lib/ip_frag/rte_ipv4_fragmentation.c:68:4,
    inlined from ‘rte_ipv4_fragment_packet’ at
../lib/ip_frag/rte_ipv4_fragmentation.c:242:16:
/usr/lib/gcc/x86_64-redhat-linux/12/include/avxintrin.h:935:8: error:
array subscript [2, 3] is outside array bounds of ‘uint8_t[60]’ {aka
‘unsigned char[60]’} [-Werror=array-bounds]
  935 |   *__P = __A;
      |   ~~~~~^~~~~
../lib/ip_frag/rte_ipv4_fragmentation.c: In function ‘rte_ipv4_fragment_packet’:
../lib/ip_frag/rte_ipv4_fragmentation.c:122:17: note: at offset [84,
124] into object ‘ipopt_frag_hdr’ of size 60
  122 |         uint8_t ipopt_frag_hdr[IPV4_HDR_MAX_LEN];
      |                 ^~~~~~~~~~~~~~
In function ‘_mm256_storeu_si256’,
    inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:347:2,
    inlined from ‘rte_mov128’ at ../lib/eal/x86/include/rte_memcpy.h:371:2,
    inlined from ‘rte_memcpy_generic’ at
../lib/eal/x86/include/rte_memcpy.h:445:4,
    inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:851:10,
    inlined from ‘__create_ipopt_frag_hdr’ at
../lib/ip_frag/rte_ipv4_fragmentation.c:68:4,
    inlined from ‘rte_ipv4_fragment_packet’ at
../lib/ip_frag/rte_ipv4_fragmentation.c:242:16:
/usr/lib/gcc/x86_64-redhat-linux/12/include/avxintrin.h:935:8: error:
array subscript [3, 4] is outside array bounds of ‘uint8_t[60]’ {aka
‘unsigned char[60]’} [-Werror=array-bounds]
  935 |   *__P = __A;
      |   ~~~~~^~~~~
../lib/ip_frag/rte_ipv4_fragmentation.c: In function ‘rte_ipv4_fragment_packet’:
../lib/ip_frag/rte_ipv4_fragmentation.c:122:17: note: at offset [116,
156] into object ‘ipopt_frag_hdr’ of size 60
  122 |         uint8_t ipopt_frag_hdr[IPV4_HDR_MAX_LEN];
      |                 ^~~~~~~~~~~~~~
In function ‘_mm256_storeu_si256’,
    inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:347:2,
    inlined from ‘rte_mov64’ at ../lib/eal/x86/include/rte_memcpy.h:358:2,
    inlined from ‘rte_memcpy_generic’ at
../lib/eal/x86/include/rte_memcpy.h:452:4,
    inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:851:10,
    inlined from ‘__create_ipopt_frag_hdr’ at
../lib/ip_frag/rte_ipv4_fragmentation.c:68:4,
    inlined from ‘rte_ipv4_fragment_packet’ at
../lib/ip_frag/rte_ipv4_fragmentation.c:242:16:
/usr/lib/gcc/x86_64-redhat-linux/12/include/avxintrin.h:935:8: error:
array subscript ‘__m256i_u[1]’ is partly outside array bounds of
‘void[60]’ [-Werror=array-bounds]
  935 |   *__P = __A;
      |   ~~~~~^~~~~
../lib/ip_frag/rte_ipv4_fragmentation.c: In function ‘rte_ipv4_fragment_packet’:
../lib/ip_frag/rte_ipv4_fragmentation.c:122:17: note: at offset [180,
240] into object ‘ipopt_frag_hdr’ of size 60
  122 |         uint8_t ipopt_frag_hdr[IPV4_HDR_MAX_LEN];
      |                 ^~~~~~~~~~~~~~
../lib/ip_frag/rte_ipv4_fragmentation.c:122:17: note: at offset [52,
60] into object ‘ipopt_frag_hdr’ of size 60
In function ‘_mm256_storeu_si256’,
    inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:347:2,
    inlined from ‘rte_memcpy_generic’ at
../lib/eal/x86/include/rte_memcpy.h:457:4,
    inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:851:10,
    inlined from ‘__create_ipopt_frag_hdr’ at
../lib/ip_frag/rte_ipv4_fragmentation.c:68:4,
    inlined from ‘rte_ipv4_fragment_packet’ at
../lib/ip_frag/rte_ipv4_fragmentation.c:242:16:
/usr/lib/gcc/x86_64-redhat-linux/12/include/avxintrin.h:935:8: error:
array subscript [2, 7] is outside array bounds of ‘void[60]’
[-Werror=array-bounds]
  935 |   *__P = __A;
      |   ~~~~~^~~~~
../lib/ip_frag/rte_ipv4_fragmentation.c: In function ‘rte_ipv4_fragment_packet’:
../lib/ip_frag/rte_ipv4_fragmentation.c:122:17: note: at offset [148,
272] into object ‘ipopt_frag_hdr’ of size 60
  122 |         uint8_t ipopt_frag_hdr[IPV4_HDR_MAX_LEN];
      |                 ^~~~~~~~~~~~~~
../lib/ip_frag/rte_ipv4_fragmentation.c:122:17: note: at offset [148,
272] into object ‘ipopt_frag_hdr’ of size 60
../lib/ip_frag/rte_ipv4_fragmentation.c:122:17: note: at offset [20,
60] into object ‘ipopt_frag_hdr’ of size 60
In function ‘_mm256_storeu_si256’,
    inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:347:2,
    inlined from ‘rte_memcpy_generic’ at
../lib/eal/x86/include/rte_memcpy.h:458:4,
    inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:851:10,
    inlined from ‘__create_ipopt_frag_hdr’ at
../lib/ip_frag/rte_ipv4_fragmentation.c:68:4,
    inlined from ‘rte_ipv4_fragment_packet’ at
../lib/ip_frag/rte_ipv4_fragmentation.c:242:16:
/usr/lib/gcc/x86_64-redhat-linux/12/include/avxintrin.h:935:8: error:
array subscript [2, 8] is outside array bounds of ‘void[60]’
[-Werror=array-bounds]
  935 |   *__P = __A;
      |   ~~~~~^~~~~
../lib/ip_frag/rte_ipv4_fragmentation.c: In function ‘rte_ipv4_fragment_packet’:
../lib/ip_frag/rte_ipv4_fragmentation.c:122:17: note: at offset [149,
273] into object ‘ipopt_frag_hdr’ of size 60
  122 |         uint8_t ipopt_frag_hdr[IPV4_HDR_MAX_LEN];
      |                 ^~~~~~~~~~~~~~
../lib/ip_frag/rte_ipv4_fragmentation.c:122:17: note: at offset [149,
273] into object ‘ipopt_frag_hdr’ of size 60
../lib/ip_frag/rte_ipv4_fragmentation.c:122:17: note: at offset [21,
60] into object ‘ipopt_frag_hdr’ of size 60
In function ‘_mm256_storeu_si256’,
    inlined from ‘rte_mov32’ at ../lib/eal/x86/include/rte_memcpy.h:347:2,
    inlined from ‘rte_memcpy_generic’ at
../lib/eal/x86/include/rte_memcpy.h:438:3,
    inlined from ‘rte_memcpy’ at ../lib/eal/x86/include/rte_memcpy.h:851:10,
    inlined from ‘__create_ipopt_frag_hdr’ at
../lib/ip_frag/rte_ipv4_fragmentation.c:68:4,
    inlined from ‘rte_ipv4_fragment_packet’ at
../lib/ip_frag/rte_ipv4_fragmentation.c:242:16:
/usr/lib/gcc/x86_64-redhat-linux/12/include/avxintrin.h:935:8: error:
array subscript ‘__m256i_u[1]’ is partly outside array bounds of
‘uint8_t[60]’ {aka ‘unsigned char[60]’} [-Werror=array-bounds]
  935 |   *__P = __A;
      |   ~~~~~^~~~~
../lib/ip_frag/rte_ipv4_fragmentation.c: In function ‘rte_ipv4_fragment_packet’:
../lib/ip_frag/rte_ipv4_fragmentation.c:122:17: note: at offset [37,
60] into object ‘ipopt_frag_hdr’ of size 60
  122 |         uint8_t ipopt_frag_hdr[IPV4_HDR_MAX_LEN];
      |                 ^~~~~~~~~~~~~~
cc1: all warnings being treated as errors
Stephen Hemminger June 16, 2022, 4:31 p.m. UTC | #7
On Thu, 16 Jun 2022 17:10:46 +0200
David Marchand <david.marchand@redhat.com> wrote:

> On Fri, Apr 15, 2022 at 5:27 AM Huichao Cai <chcchc88@163.com> wrote:
> >
> > According to RFC791,the options may appear or not in datagrams.
> > They must be implemented by all IP modules (host and gateways).
> > What is optional is their transmission in any particular datagram,
> > not their implementation.So we have to deal with it during the
> > fragmenting process.Add some test data for the IPv4 header optional
> > field fragmenting.
> >
> > Signed-off-by: Huichao Cai <chcchc88@163.com>  
> 
> gcc-12 raises warnings on both the unit test code and the library code.
> See below.

Since the copies will all be short why bother using rte_memcpy() all over
the place.  Especially in the test code, just use memcpy().
Huichao Cai June 17, 2022, 3:52 a.m. UTC | #8
Hi,Stephen


There are some things I don't quite understand.Hope you can answer that.
This will help me avoid similar errors in subsequent patch submissions.Thanks!


There are places where rte_memcpy functions are used:
============================================
In test_ipfrag.c:
from func test_get_ipv4_opt: 
rte_memcpy(expected_opt->data,expected_first_frag_ipv4_opts_copied,sizeof(expected_first_frag_ipv4_opts_copied));
rte_memcpy(expected_opt>data,expected_first_frag_ipv4_opts_nocopied,sizeof(expected_first_frag_ipv4_opts_nocopied));
rte_memcpy(expected_opt->data,expected_sub_frag_ipv4_opts_copied,sizeof(expected_sub_frag_ipv4_opts_copied));
rte_memcpy(expected_opt->data,expected_sub_frag_ipv4_opts_nocopied,sizeof(expected_sub_frag_ipv4_opts_nocopied));
from func v4_allocate_packet_of:
rte_memcpy(hdr + 1, opt.data, opt.len);
from func test_get_frag_opt:
rte_memcpy(opt->data, iph_opt, opt_len);


In rte_ipv4_fragmentation.c:
from func v4_allocate_packet_of:
rte_memcpy(dst, src, header_len);
from func __create_ipopt_frag_hdr:

rte_memcpy(ipopt_frag_hdr, iph, sizeof(struct rte_ipv4_hdr));
rte_memcpy(ipopt_frag_hdr + ipopt_len, p_opt, p_opt[1]);
============================================


These are the compilation errors:
============================================
test_ipfrag.c:230
In test_ipfrag.c:
from func v4_allocate_packet_of:
rte_memcpy(hdr + 1, opt.data, opt.len);
rte_ipv4_fragmentation.c:68
In rte_ipv4_fragmentation.c:
from func __create_ipopt_frag_hdr:
rte_memcpy(ipopt_frag_hdr + ipopt_len, p_opt, p_opt[1]);
============================================


1.Do I need to replace all rte_memcpy with memcpy or only the two rte_memcpy that compile the error are replaced by memcpy?
2.
>Since the copies will all be short why bother using rte_memcpy() all over
>the place.  Especially in the test code, just use memcpy().
For example,in app/test-pmd/cmdline.c:from func cmd_set_vxlan_parsed:rte_memcpy(vxlan_encap_conf.vni, &id.vni[1], 3);Why this place can be used rte_memcpy?
3.For example, how such a compilation error occurs:
../app/test/test_ipfrag.c:57:17: note: at offset [5, 40] into object‘data’ of size 40
4.Under what circumstances can we use rte_memcpy?


Huichao,Cai
Stephen Hemminger June 17, 2022, 4:31 p.m. UTC | #9
On Fri, 17 Jun 2022 11:52:25 +0800 (CST)
"Huichao Cai" <chcchc88@163.com> wrote:

> Hi,Stephen
> 
> 
> There are some things I don't quite understand.Hope you can answer that.
> This will help me avoid similar errors in subsequent patch submissions.Thanks!
> 
> 
> There are places where rte_memcpy functions are used:
> ============================================
> In test_ipfrag.c:
> from func test_get_ipv4_opt: 
> rte_memcpy(expected_opt->data,expected_first_frag_ipv4_opts_copied,sizeof(expected_first_frag_ipv4_opts_copied));
> rte_memcpy(expected_opt>data,expected_first_frag_ipv4_opts_nocopied,sizeof(expected_first_frag_ipv4_opts_nocopied));  
> rte_memcpy(expected_opt->data,expected_sub_frag_ipv4_opts_copied,sizeof(expected_sub_frag_ipv4_opts_copied));
> rte_memcpy(expected_opt->data,expected_sub_frag_ipv4_opts_nocopied,sizeof(expected_sub_frag_ipv4_opts_nocopied));
> from func v4_allocate_packet_of:
> rte_memcpy(hdr + 1, opt.data, opt.len);
> from func test_get_frag_opt:
> rte_memcpy(opt->data, iph_opt, opt_len);
> 
> 
> In rte_ipv4_fragmentation.c:
> from func v4_allocate_packet_of:
> rte_memcpy(dst, src, header_len);
> from func __create_ipopt_frag_hdr:
> 
> rte_memcpy(ipopt_frag_hdr, iph, sizeof(struct rte_ipv4_hdr));
> rte_memcpy(ipopt_frag_hdr + ipopt_len, p_opt, p_opt[1]);
> ============================================
> 
> 
> These are the compilation errors:
> ============================================
> test_ipfrag.c:230
> In test_ipfrag.c:
> from func v4_allocate_packet_of:
> rte_memcpy(hdr + 1, opt.data, opt.len);
> rte_ipv4_fragmentation.c:68
> In rte_ipv4_fragmentation.c:
> from func __create_ipopt_frag_hdr:
> rte_memcpy(ipopt_frag_hdr + ipopt_len, p_opt, p_opt[1]);
> ============================================
> 
> 
> 1.Do I need to replace all rte_memcpy with memcpy or only the two rte_memcpy that compile the error are replaced by memcpy?

I would just replace all of the rte_memcpy with memcpy

> 2.
> >Since the copies will all be short why bother using rte_memcpy() all over
> >the place.  Especially in the test code, just use memcpy().  
> For example,in app/test-pmd/cmdline.c:from func cmd_set_vxlan_parsed:rte_memcpy(vxlan_encap_conf.vni, &id.vni[1], 3);Why this place can be used rte_memcpy?
> 3.For example, how such a compilation error occurs:
> ../app/test/test_ipfrag.c:57:17: note: at offset [5, 40] into object‘data’ of size 40
> 4.Under what circumstances can we use rte_memcpy?


It depends. The recommendation here was that fixing warnings is higher priority that saving a few cycles
in an underutilized part of DPDK.

Rte_memcpy() was added in early versions of DPDK because the standard toolchain gcc/glibc
was not using the optimum set of instructions on x86.  Rather than fix glibc, Intel wrote
their own rte_memcpy(). Then DPDK developers, started to assume that rte_memcpy() is always best.

I expect that rte_memcpy() is able to do better than memcpy() for larger copies because it is
likely to use bigger vector instructions and check for alignment.
For small copies just doing the mov's directly is going to be as fast or faster.
In fact, lots of places in DPDK should
replace rte_memcpy() with simple structure assignment to preserve type safety.

This is somewhat historical data, it might be wrong. It would be worthwhile to have benchmarks
across different sizes (variable and fixed), different compilers, and different CPU's.
There might be surprising results.
Huichao Cai June 18, 2022, 11:01 a.m. UTC | #10
Hi,Stephen
Thank you very much for your reply!
>I would just replace all of the rte_memcpy with memcpy  
I will replace all of the rte_memcpy with memcpy.
>I expect that rte_memcpy() is able to do better than memcpy() for larger copies because it is
>likely to use bigger vector instructions and check for alignment.
>For small copies just doing the mov's directly is going to be as fast or faster.
>In fact, lots of places in DPDK should
>replace rte_memcpy() with simple structure assignment to preserve type safety.
I don't know the dividing line(the size of the data) between rte_memcpy and memcpy.
We simply test 1500 bytes of replication, memcpy seems to be faster, maybe our test is not accurate enough.
>This is somewhat historical data, it might be wrong. It would be worthwhile to have benchmarks
>across different sizes (variable and fixed), different compilers, and different CPU's.
>There might be surprising results.
So I hope this can go on and provide a more professional rte_memcpy manual.Thanks!
Huichao,Cai
diff mbox series

Patch

diff --git a/app/test/test_ipfrag.c b/app/test/test_ipfrag.c
index 1ced25a..610a86b 100644
--- a/app/test/test_ipfrag.c
+++ b/app/test/test_ipfrag.c
@@ -18,10 +18,50 @@ 
 #define NUM_MBUFS 128
 #define BURST 32
 
+uint8_t expected_first_frag_ipv4_opts_copied[] = {
+	0x07, 0x0b, 0x04, 0x00,
+	0x00, 0x00, 0x00, 0x00,
+	0x00, 0x00, 0x00, 0x83,
+	0x07, 0x04, 0xc0, 0xa8,
+	0xe3, 0x96, 0x00, 0x00,
+};
+
+uint8_t expected_sub_frag_ipv4_opts_copied[] = {
+	0x83, 0x07, 0x04, 0xc0,
+	0xa8, 0xe3, 0x96, 0x00,
+};
+
+uint8_t expected_first_frag_ipv4_opts_nocopied[] = {
+	0x07, 0x0b, 0x04, 0x00,
+	0x00, 0x00, 0x00, 0x00,
+	0x00, 0x00, 0x00, 0x00,
+};
+
+uint8_t expected_sub_frag_ipv4_opts_nocopied[0];
+
+struct test_opt_data {
+	bool is_first_frag;		 /**< offset is 0 */
+	bool opt_copied;		 /**< ip option copied flag */
+	uint16_t len;			 /**< option data len */
+	uint8_t data[RTE_IPV4_HDR_OPT_MAX_LEN]; /**< option data */
+};
+
 static struct rte_mempool *pkt_pool,
 			  *direct_pool,
 			  *indirect_pool;
 
+static inline void
+hex_to_str(uint8_t *hex, uint16_t len, char *str)
+{
+	int i;
+
+	for (i = 0; i < len; i++) {
+		sprintf(str, "%02x", hex[i]);
+		str += 2;
+	}
+	*str = 0;
+}
+
 static int
 setup_buf_pool(void)
 {
@@ -88,23 +128,67 @@  static void ut_teardown(void)
 {
 }
 
+static inline void
+test_get_ipv4_opt(bool is_first_frag, bool opt_copied,
+	struct test_opt_data *expected_opt)
+{
+	if (is_first_frag) {
+		if (opt_copied) {
+			expected_opt->len =
+				sizeof(expected_first_frag_ipv4_opts_copied);
+			rte_memcpy(expected_opt->data,
+				expected_first_frag_ipv4_opts_copied,
+				sizeof(expected_first_frag_ipv4_opts_copied));
+		} else {
+			expected_opt->len =
+				sizeof(expected_first_frag_ipv4_opts_nocopied);
+			rte_memcpy(expected_opt->data,
+				expected_first_frag_ipv4_opts_nocopied,
+				sizeof(expected_first_frag_ipv4_opts_nocopied));
+		}
+	} else {
+		if (opt_copied) {
+			expected_opt->len =
+				sizeof(expected_sub_frag_ipv4_opts_copied);
+			rte_memcpy(expected_opt->data,
+				expected_sub_frag_ipv4_opts_copied,
+				sizeof(expected_sub_frag_ipv4_opts_copied));
+		} else {
+			expected_opt->len =
+				sizeof(expected_sub_frag_ipv4_opts_nocopied);
+			rte_memcpy(expected_opt->data,
+				expected_sub_frag_ipv4_opts_nocopied,
+				sizeof(expected_sub_frag_ipv4_opts_nocopied));
+		}
+	}
+}
+
 static void
-v4_allocate_packet_of(struct rte_mbuf *b, int fill,
-		      size_t s, int df, uint8_t mf, uint16_t off,
-		      uint8_t ttl, uint8_t proto, uint16_t pktid)
+v4_allocate_packet_of(struct rte_mbuf *b, int fill, size_t s,
+	int df, uint8_t mf, uint16_t off, uint8_t ttl, uint8_t proto,
+	uint16_t pktid, bool have_opt, bool is_first_frag, bool opt_copied)
 {
 	/* Create a packet, 2k bytes long */
 	b->data_off = 0;
 	char *data = rte_pktmbuf_mtod(b, char *);
-	rte_be16_t fragment_offset = 0;	/**< fragmentation offset */
+	rte_be16_t fragment_offset = 0;	/* fragmentation offset */
+	uint16_t iph_len;
+	struct test_opt_data opt;
+
+	opt.len = 0;
+
+	if (have_opt)
+		test_get_ipv4_opt(is_first_frag, opt_copied, &opt);
 
-	memset(data, fill, sizeof(struct rte_ipv4_hdr) + s);
+	iph_len = sizeof(struct rte_ipv4_hdr) + opt.len;
+	memset(data, fill, iph_len + s);
 
 	struct rte_ipv4_hdr *hdr = (struct rte_ipv4_hdr *)data;
 
-	hdr->version_ihl = 0x45; /* standard IP header... */
+	hdr->version_ihl = 0x40; /* ipv4 */
+	hdr->version_ihl += (iph_len / 4);
 	hdr->type_of_service = 0;
-	b->pkt_len = s + sizeof(struct rte_ipv4_hdr);
+	b->pkt_len = s + iph_len;
 	b->data_len = b->pkt_len;
 	hdr->total_length = rte_cpu_to_be_16(b->pkt_len);
 	hdr->packet_id = rte_cpu_to_be_16(pktid);
@@ -131,6 +215,8 @@  static void ut_teardown(void)
 	hdr->hdr_checksum = 0;
 	hdr->src_addr = rte_cpu_to_be_32(0x8080808);
 	hdr->dst_addr = rte_cpu_to_be_32(0x8080404);
+
+	rte_memcpy(hdr + 1, opt.data, opt.len);
 }
 
 static void
@@ -187,6 +273,45 @@  static void ut_teardown(void)
 	}
 }
 
+static inline void
+test_get_frag_opt(struct rte_mbuf **mb, int32_t num,
+	struct test_opt_data *opt, int ipv, bool opt_copied)
+{
+	int32_t i;
+
+	for (i = 0; i < num; i++) {
+		if (ipv == 4) {
+			struct rte_ipv4_hdr *iph =
+			    rte_pktmbuf_mtod(mb[i], struct rte_ipv4_hdr *);
+			uint16_t header_len = (iph->version_ihl &
+				RTE_IPV4_HDR_IHL_MASK) *
+				RTE_IPV4_IHL_MULTIPLIER;
+			uint16_t opt_len = header_len -
+				sizeof(struct rte_ipv4_hdr);
+
+			opt->opt_copied = opt_copied;
+
+			if ((rte_be_to_cpu_16(iph->fragment_offset) &
+				    RTE_IPV4_HDR_OFFSET_MASK) == 0)
+				opt->is_first_frag = true;
+			else
+				opt->is_first_frag = false;
+
+			if (likely(opt_len <= RTE_IPV4_HDR_OPT_MAX_LEN)) {
+				char *iph_opt = rte_pktmbuf_mtod_offset(mb[i],
+				    char *, sizeof(struct rte_ipv4_hdr));
+				opt->len = opt_len;
+				rte_memcpy(opt->data, iph_opt, opt_len);
+			} else {
+				opt->len = RTE_IPV4_HDR_OPT_MAX_LEN;
+				memset(opt->data, RTE_IPV4_HDR_OPT_EOL,
+				    sizeof(opt->data));
+			}
+			opt++;
+		}
+	}
+}
+
 static int
 test_ip_frag(void)
 {
@@ -206,32 +331,52 @@  static void ut_teardown(void)
 		uint16_t pkt_id;
 		int      expected_frags;
 		uint16_t expected_fragment_offset[BURST];
+		bool have_opt;
+		bool is_first_frag;
+		bool opt_copied;
 	} tests[] = {
 		 {4, 1280, 1400, 0, 0, 0, 64, IPPROTO_ICMP, RND_ID,       2,
-		  {0x2000, 0x009D}},
+		  {0x2000, 0x009D}, false},
 		 {4, 1280, 1400, 0, 0, 0, 64, IPPROTO_ICMP, 0,            2,
-		  {0x2000, 0x009D}},
+		  {0x2000, 0x009D}, false},
 		 {4,  600, 1400, 0, 0, 0, 64, IPPROTO_ICMP, RND_ID,       3,
-		  {0x2000, 0x2048, 0x0090}},
+		  {0x2000, 0x2048, 0x0090}, false},
 		 {4, 4, 1400, 0, 0, 0, 64, IPPROTO_ICMP, RND_ID,    -EINVAL},
 		 {4, 600, 1400, 1, 0, 0, 64, IPPROTO_ICMP, RND_ID, -ENOTSUP},
 		 {4, 600, 1400, 0, 0, 0, 0, IPPROTO_ICMP, RND_ID,         3,
-		  {0x2000, 0x2048, 0x0090}},
+		  {0x2000, 0x2046, 0x008C}, true, true, true},
+		 /* The first fragment */
+		 {4, 68, 104, 0, 1, 0, 0, IPPROTO_ICMP, RND_ID,           5,
+		  {0x2000, 0x2003, 0x2006, 0x2009, 0x200C}, true, true, true},
+		 /* The middle fragment */
 		 {4, 68, 104, 0, 1, 13, 0, IPPROTO_ICMP, RND_ID,          3,
-		  {0x200D, 0x2013, 0x2019}},
-
+		  {0x200D, 0x2012, 0x2017}, true, false, true},
+		 /* The last fragment */
+		 {4, 68, 104, 0, 0, 26, 0, IPPROTO_ICMP, RND_ID,          3,
+		  {0x201A, 0x201F, 0x0024}, true, false, true},
+		 /* The first fragment */
+		 {4, 68, 104, 0, 1, 0, 0, IPPROTO_ICMP, RND_ID,           4,
+		  {0x2000, 0x2004, 0x2008, 0x200C}, true, true, false},
+		 /* The middle fragment */
+		 {4, 68, 104, 0, 1, 13, 0, IPPROTO_ICMP, RND_ID,          3,
+		  {0x200D, 0x2013, 0x2019}, true, false, false},
+		 /* The last fragment */
+		 {4, 68, 104, 0, 0, 26, 0, IPPROTO_ICMP, RND_ID,          3,
+		  {0x201A, 0x2020, 0x0026}, true, false, false},
 		 {6, 1280, 1400, 0, 0, 0, 64, IPPROTO_ICMP, RND_ID,       2,
-		  {0x0001, 0x04D0}},
+		  {0x0001, 0x04D0}, false},
 		 {6, 1300, 1400, 0, 0, 0, 64, IPPROTO_ICMP, RND_ID,       2,
-		  {0x0001, 0x04E0}},
+		  {0x0001, 0x04E0}, false},
 		 {6, 4, 1400, 0, 0, 0, 64, IPPROTO_ICMP, RND_ID,    -EINVAL},
 		 {6, 1300, 1400, 0, 0, 0, 0, IPPROTO_ICMP, RND_ID,        2,
-		  {0x0001, 0x04E0}},
+		  {0x0001, 0x04E0}, false},
 	};
 
 	for (i = 0; i < RTE_DIM(tests); i++) {
 		int32_t len = 0;
 		uint16_t fragment_offset[BURST];
+		struct test_opt_data opt_res[BURST];
+		struct test_opt_data opt_exp;
 		uint16_t pktid = tests[i].pkt_id;
 		struct rte_mbuf *pkts_out[BURST];
 		struct rte_mbuf *b = rte_pktmbuf_alloc(pkt_pool);
@@ -250,7 +395,10 @@  static void ut_teardown(void)
 					      tests[i].set_of,
 					      tests[i].ttl,
 					      tests[i].proto,
-					      pktid);
+					      pktid,
+					      tests[i].have_opt,
+					      tests[i].is_first_frag,
+					      tests[i].opt_copied);
 		} else if (tests[i].ipv == 6) {
 			v6_allocate_packet_of(b, 0x41414141,
 					      tests[i].pkt_size,
@@ -275,17 +423,20 @@  static void ut_teardown(void)
 		if (len > 0) {
 			test_get_offset(pkts_out, len,
 			    fragment_offset, tests[i].ipv);
+			if (tests[i].have_opt)
+				test_get_frag_opt(pkts_out, len, opt_res,
+					tests[i].ipv, tests[i].opt_copied);
 			test_free_fragments(pkts_out, len);
 		}
 
-		printf("%zd: checking %d with %d\n", i, len,
+		printf("[check frag number]%zd: checking %d with %d\n", i, len,
 		       tests[i].expected_frags);
 		RTE_TEST_ASSERT_EQUAL(len, tests[i].expected_frags,
 				      "Failed case %zd.\n", i);
 
 		if (len > 0) {
 			for (j = 0; j < (size_t)len; j++) {
-				printf("%zd-%zd: checking %d with %d\n",
+				printf("[check offset]%zd-%zd: checking %d with %d\n",
 				    i, j, fragment_offset[j],
 				    rte_cpu_to_be_16(
 					tests[i].expected_fragment_offset[j]));
@@ -294,6 +445,36 @@  static void ut_teardown(void)
 					tests[i].expected_fragment_offset[j]),
 				    "Failed case %zd.\n", i);
 			}
+
+			if (tests[i].have_opt && (tests[i].ipv == 4)) {
+				for (j = 0; j < (size_t)len; j++) {
+					char opt_res_str[2 *
+						RTE_IPV4_HDR_OPT_MAX_LEN + 1];
+					char opt_exp_str[2 *
+						RTE_IPV4_HDR_OPT_MAX_LEN + 1];
+
+					test_get_ipv4_opt(
+						opt_res[j].is_first_frag,
+						opt_res[j].opt_copied,
+						&opt_exp);
+					hex_to_str(opt_res[j].data,
+						opt_res[j].len,
+						opt_res_str);
+					hex_to_str(opt_exp.data,
+						opt_exp.len,
+						opt_exp_str);
+
+					printf(
+						"[check ipv4 option]%zd-%zd: checking (len:%u)%s with (len:%u)%s\n",
+						i, j,
+						opt_res[j].len, opt_res_str,
+						opt_exp.len, opt_exp_str);
+						RTE_TEST_ASSERT_SUCCESS(
+							strcmp(opt_res_str,
+								opt_exp_str),
+						"Failed case %zd.\n", i);
+				}
+			}
 		}
 
 	}
diff --git a/lib/ip_frag/rte_ipv4_fragmentation.c b/lib/ip_frag/rte_ipv4_fragmentation.c
index 2e7739d..a562424 100644
--- a/lib/ip_frag/rte_ipv4_fragmentation.c
+++ b/lib/ip_frag/rte_ipv4_fragmentation.c
@@ -22,6 +22,8 @@ 
 
 #define	IPV4_HDR_FO_ALIGN			(1 << RTE_IPV4_HDR_FO_SHIFT)
 
+#define IPV4_HDR_MAX_LEN			60
+
 static inline void __fill_ipv4hdr_frag(struct rte_ipv4_hdr *dst,
 		const struct rte_ipv4_hdr *src, uint16_t header_len,
 		uint16_t len, uint16_t fofs, uint16_t dofs, uint32_t mf)
@@ -41,6 +43,49 @@  static inline void __free_fragments(struct rte_mbuf *mb[], uint32_t num)
 		rte_pktmbuf_free(mb[i]);
 }
 
+static inline uint16_t __create_ipopt_frag_hdr(uint8_t *iph,
+	uint16_t ipopt_len, uint8_t *ipopt_frag_hdr)
+{
+	uint16_t len = ipopt_len;
+	struct rte_ipv4_hdr *iph_opt = (struct rte_ipv4_hdr *)ipopt_frag_hdr;
+
+	ipopt_len = 0;
+	rte_memcpy(ipopt_frag_hdr, iph, sizeof(struct rte_ipv4_hdr));
+	ipopt_frag_hdr += sizeof(struct rte_ipv4_hdr);
+
+	uint8_t *p_opt = iph + sizeof(struct rte_ipv4_hdr);
+
+	while (len > 0) {
+		if (unlikely(*p_opt == RTE_IPV4_HDR_OPT_NOP)) {
+			len--;
+			p_opt++;
+			continue;
+		} else if (unlikely(*p_opt == RTE_IPV4_HDR_OPT_EOL))
+			break;
+
+		if (unlikely(p_opt[1] < 2 || p_opt[1] > len))
+			break;
+
+		if (RTE_IPV4_HDR_OPT_COPIED(*p_opt)) {
+			rte_memcpy(ipopt_frag_hdr + ipopt_len,
+				p_opt, p_opt[1]);
+			ipopt_len += p_opt[1];
+		}
+
+		len -= p_opt[1];
+		p_opt += p_opt[1];
+	}
+
+	len = RTE_ALIGN_CEIL(ipopt_len, RTE_IPV4_IHL_MULTIPLIER);
+	memset(ipopt_frag_hdr + ipopt_len,
+		RTE_IPV4_HDR_OPT_EOL, len - ipopt_len);
+	ipopt_len = len;
+	iph_opt->ihl = (sizeof(struct rte_ipv4_hdr) + ipopt_len) /
+		RTE_IPV4_IHL_MULTIPLIER;
+
+	return ipopt_len;
+}
+
 /**
  * IPv4 fragmentation.
  *
@@ -76,6 +121,8 @@  static inline void __free_fragments(struct rte_mbuf *mb[], uint32_t num)
 	uint32_t more_in_segs;
 	uint16_t fragment_offset, flag_offset, frag_size, header_len;
 	uint16_t frag_bytes_remaining;
+	uint8_t ipopt_frag_hdr[IPV4_HDR_MAX_LEN];
+	uint16_t ipopt_len;
 
 	/*
 	 * Formal parameter checking.
@@ -118,6 +165,10 @@  static inline void __free_fragments(struct rte_mbuf *mb[], uint32_t num)
 	out_pkt_pos = 0;
 	fragment_offset = 0;
 
+	ipopt_len = header_len - sizeof(struct rte_ipv4_hdr);
+	if (unlikely(ipopt_len > RTE_IPV4_HDR_OPT_MAX_LEN))
+		return -EINVAL;
+
 	more_in_segs = 1;
 	while (likely(more_in_segs)) {
 		struct rte_mbuf *out_pkt = NULL, *out_seg_prev = NULL;
@@ -188,10 +239,21 @@  static inline void __free_fragments(struct rte_mbuf *mb[], uint32_t num)
 		    (uint16_t)out_pkt->pkt_len,
 		    flag_offset, fragment_offset, more_in_segs);
 
-		fragment_offset = (uint16_t)(fragment_offset +
-		    out_pkt->pkt_len - header_len);
-
-		out_pkt->l3_len = header_len;
+		if (unlikely((fragment_offset == 0) && (ipopt_len) &&
+			    ((flag_offset & RTE_IPV4_HDR_OFFSET_MASK) == 0))) {
+			ipopt_len = __create_ipopt_frag_hdr((uint8_t *)in_hdr,
+				ipopt_len, ipopt_frag_hdr);
+			fragment_offset = (uint16_t)(fragment_offset +
+				out_pkt->pkt_len - header_len);
+			out_pkt->l3_len = header_len;
+
+			header_len = sizeof(struct rte_ipv4_hdr) + ipopt_len;
+			in_hdr = (struct rte_ipv4_hdr *)ipopt_frag_hdr;
+		} else {
+			fragment_offset = (uint16_t)(fragment_offset +
+				out_pkt->pkt_len - header_len);
+			out_pkt->l3_len = header_len;
+		}
 
 		/* Write the fragment to the output list */
 		pkts_out[out_pkt_pos] = out_pkt;
diff --git a/lib/net/rte_ip.h b/lib/net/rte_ip.h
index c575250..2c3894b 100644
--- a/lib/net/rte_ip.h
+++ b/lib/net/rte_ip.h
@@ -97,6 +97,12 @@  struct rte_ipv4_hdr {
 
 #define	RTE_IPV4_HDR_OFFSET_UNITS	8
 
+/* IPv4 options */
+#define RTE_IPV4_HDR_OPT_EOL     0
+#define RTE_IPV4_HDR_OPT_NOP       1
+#define RTE_IPV4_HDR_OPT_COPIED(v) ((v) & 0x80)
+#define RTE_IPV4_HDR_OPT_MAX_LEN   40
+
 /*
  * IPv4 address types
  */