mbox series

[v5,00/32] replace use of rte_memcpy() with fixed size

Message ID 20240522033009.143100-1-stephen@networkplumber.org (mailing list archive)
Headers
Series replace use of rte_memcpy() with fixed size |

Message

Stephen Hemminger May 22, 2024, 3:27 a.m. UTC 
The DPDK has a lot of unnecessary usage of rte_memcpy.
This patch set replaces cases where rte_memcpy is used with a fixed
size constant size.

Typical example is:
	rte_memcpy(mac_addrs, mac.addr_bytes, RTE_ETHER_ADDR_LEN);
which can be replaced with:
	memcpy(mac_addrs, mac.addr_bytes, RTE_ETHER_ADDR_LEN);

This does not change the resulting binary on almost all architectures
because x86 version of intrisics and glibc are the same, and
other architectures were using __builtin_constant_p().

The main benefit is that analysis tools like fortify, Coverity, and ASAN
analyzers can check these memcpy's. A recent example is that
on Ubuntu 22.04 detected undefined use of memcpy such as:
	memcpy(dst, NULL, 0)

The first patch is a simple coccinelle script to do the replacement
and the rest are the results broken out by module. The script can be used
again to make sure more bad usage doesn't creep in with new drivers.

v5 - rebase and fix a couple of the SW drivers.
     rewording of commit messages

v4 - replace other unnecessary rte_memcpy in ethdev.
     replace memcpy with structure assignment where possible.
     fixup formatting in a couple places.

v3 - rebase and rerun coccinelle script on 24.03
   - consolidate patches by subtree
   - remove inclusion of rte_memcpy.h when no longer used.

Stephen Hemminger (32):
  cocci/rte_memcpy: add script to eliminate fixed size rte_memcpy
  eal: replace use of fixed size rte_memcpy
  ethdev: replace uses of rte_memcpy
  eventdev: replace use of fixed size rte_memcpy
  cryptodev: replace use of fixed size rte_memcpy
  ip_frag: replace use of fixed size rte_memcpy
  net: replace use of fixed size rte_memcpy
  lpm: replace use of fixed size rte_memcpy
  node: replace use of fixed size rte_memcpy
  pdcp: replace use of fixed size rte_memcpy
  pipeline: replace use of fixed size rte_memcpy
  rib: replace use of fixed size rte_memcpy
  security: replace use of fixed size rte_memcpy
  bus: remove unneeded rte_memcpy.h include
  raw: replace use of fixed size rte_memcpy
  baseband: replace use of fixed size rte_memcpy
  common: replace use of fixed size rte_memcpy
  crypto: replace use of fixed size rte_memcpy
  event: replace use of fixed size rte_memcpy
  mempool: replace use of fixed size rte_memcpy
  ml/cnxk: replace use of fixed size rte_memcpy
  app/test-pmd: replace use of fixed size rte_memcpy
  app/graph: replace use of fixed size rte_memcpy
  app/test-eventdev: replace use of fixed size rte_memcpy
  app/test: replace use of fixed size rte_memcpy
  app/test-pipeline: remove unused rte_memcpy.h include
  app/test-bbdev: remove unnecessary include of rte_memcpy.h
  examples: replace use of fixed size rte_memcpy
  net/null: replace use of fixed size memcpy
  net/tap: replace use of fixed size rte_memcpy
  net/pcap: replace use of fixed size rte_memcpy
  net/af_xdp:: replace use of fixed size rte_memcpy

 app/graph/neigh.c                             |   8 +-
 app/test-bbdev/test_bbdev.c                   |   1 -
 app/test-eventdev/test_pipeline_common.c      |  19 ++--
 app/test-pipeline/config.c                    |   1 -
 app/test-pipeline/init.c                      |   1 -
 app/test-pipeline/main.c                      |   1 -
 app/test-pipeline/runtime.c                   |   1 -
 app/test-pmd/cmdline.c                        |  48 ++++-----
 app/test-pmd/cmdline_flow.c                   |  24 ++---
 app/test-pmd/config.c                         |   8 +-
 app/test-pmd/csumonly.c                       |   1 -
 app/test-pmd/flowgen.c                        |   1 -
 app/test-pmd/iofwd.c                          |   1 -
 app/test-pmd/macfwd.c                         |   1 -
 app/test-pmd/macswap.c                        |   1 -
 app/test-pmd/noisy_vnf.c                      |   1 -
 app/test-pmd/rxonly.c                         |   1 -
 app/test-pmd/testpmd.c                        |   1 -
 app/test/commands.c                           |   1 -
 app/test/packet_burst_generator.c             |   4 +-
 app/test/test_crc.c                           |   5 +-
 app/test/test_cryptodev.c                     |  18 ++--
 app/test/test_cryptodev_asym.c                |   1 -
 app/test/test_cryptodev_security_pdcp.c       |   1 -
 app/test/test_efd.c                           |   1 -
 app/test/test_efd_perf.c                      |   1 -
 app/test/test_event_crypto_adapter.c          |  12 +--
 app/test/test_event_dma_adapter.c             |   4 +-
 app/test/test_eventdev.c                      |   1 -
 app/test/test_ipsec.c                         |   6 +-
 app/test/test_link_bonding_mode4.c            |   8 +-
 app/test/test_mbuf.c                          |   1 -
 app/test/test_member.c                        |   1 -
 app/test/test_member_perf.c                   |   1 -
 app/test/test_rawdev.c                        |   1 -
 app/test/test_security_inline_proto.c         |  36 +++----
 app/test/test_service_cores.c                 |   1 -
 app/test/virtual_pmd.c                        |   3 +-
 devtools/cocci/rte_memcpy.cocci               |  11 ++
 drivers/baseband/acc/rte_acc100_pmd.c         |  16 ++-
 drivers/baseband/acc/rte_vrb_pmd.c            |  21 ++--
 drivers/baseband/la12xx/bbdev_la12xx.c        |   4 +-
 drivers/bus/auxiliary/linux/auxiliary.c       |   1 -
 drivers/bus/fslmc/fslmc_bus.c                 |   1 -
 drivers/bus/fslmc/fslmc_vfio.c                |   1 -
 drivers/bus/fslmc/portal/dpaa2_hw_dpbp.c      |   1 -
 drivers/bus/fslmc/portal/dpaa2_hw_dpci.c      |   1 -
 drivers/bus/fslmc/portal/dpaa2_hw_dpio.c      |   1 -
 drivers/bus/pci/linux/pci.c                   |   1 -
 drivers/common/idpf/idpf_common_device.c      |   4 +-
 drivers/common/idpf/idpf_common_virtchnl.c    |  10 +-
 drivers/common/qat/qat_qp.c                   |  10 +-
 drivers/compress/qat/qat_comp.c               |   8 +-
 drivers/crypto/ccp/ccp_crypto.c               |  14 +--
 drivers/crypto/cnxk/cnxk_cryptodev_ops.c      |   2 +-
 drivers/crypto/cnxk/cnxk_se.h                 |   2 +-
 drivers/crypto/dpaa_sec/dpaa_sec.c            |   2 +-
 drivers/crypto/ipsec_mb/pmd_snow3g.c          |   4 +-
 drivers/crypto/qat/qat_sym_session.c          |  52 +++++----
 .../scheduler/rte_cryptodev_scheduler.c       |   6 +-
 drivers/crypto/scheduler/scheduler_failover.c |  12 +--
 drivers/event/cnxk/cnxk_eventdev_selftest.c   |   1 -
 drivers/event/cnxk/cnxk_tim_evdev.c           |   4 +-
 drivers/event/dlb2/dlb2.c                     |   4 +-
 drivers/event/dpaa/dpaa_eventdev.c            |   1 -
 drivers/event/dpaa2/dpaa2_eventdev.c          |   7 +-
 drivers/event/dpaa2/dpaa2_eventdev_selftest.c |   1 -
 drivers/event/dpaa2/dpaa2_hw_dpcon.c          |   1 -
 drivers/event/octeontx/ssovf_evdev_selftest.c |   1 -
 drivers/event/octeontx/timvf_evdev.c          |   4 +-
 drivers/mempool/dpaa/dpaa_mempool.c           |   4 +-
 drivers/mempool/dpaa2/dpaa2_hw_mempool.c      |   5 +-
 drivers/ml/cnxk/cn10k_ml_model.c              |   8 +-
 drivers/ml/cnxk/cn10k_ml_ops.c                |  11 +-
 drivers/ml/cnxk/cnxk_ml_ops.c                 |   2 +-
 drivers/ml/cnxk/mvtvm_ml_model.c              |   8 +-
 drivers/ml/cnxk/mvtvm_ml_ops.c                |   8 +-
 drivers/net/af_xdp/rte_eth_af_xdp.c           |   2 +-
 drivers/net/null/rte_eth_null.c               |   6 +-
 drivers/net/pcap/pcap_ethdev.c                |   2 +-
 drivers/net/pcap/pcap_osdep_freebsd.c         |   2 +-
 drivers/net/pcap/pcap_osdep_linux.c           |   2 +-
 drivers/net/tap/rte_eth_tap.c                 |  14 +--
 drivers/raw/ifpga/afu_pmd_he_hssi.c           |   3 +-
 drivers/raw/ifpga/afu_pmd_he_lpbk.c           |   3 +-
 drivers/raw/ifpga/afu_pmd_he_mem.c            |   3 +-
 drivers/raw/ifpga/afu_pmd_n3000.c             |   8 +-
 drivers/raw/ifpga/ifpga_rawdev.c              |  11 +-
 drivers/raw/skeleton/skeleton_rawdev.c        |   8 +-
 drivers/raw/skeleton/skeleton_rawdev_test.c   |   1 -
 examples/bbdev_app/main.c                     |   2 +-
 examples/bond/main.c                          |   1 -
 examples/ip_fragmentation/main.c              |   1 -
 examples/ip_reassembly/main.c                 |   1 -
 examples/ipv4_multicast/main.c                |   1 -
 examples/l2fwd-cat/cat.c                      |   4 +-
 examples/l2fwd-jobstats/main.c                |   1 -
 examples/l2fwd-keepalive/main.c               |   1 -
 examples/l2fwd-macsec/main.c                  |   1 -
 examples/l2fwd/main.c                         |   1 -
 examples/l3fwd-power/main.c                   |   1 -
 examples/l3fwd/main.c                         |   1 -
 examples/link_status_interrupt/main.c         |   1 -
 .../client_server_mp/mp_server/init.c         |   1 -
 .../client_server_mp/mp_server/main.c         |   1 -
 examples/multi_process/symmetric_mp/main.c    |   1 -
 examples/ptpclient/ptpclient.c                |  11 +-
 examples/qos_sched/app_thread.c               |   1 -
 examples/qos_sched/main.c                     |   1 -
 examples/server_node_efd/efd_server/init.c    |   1 -
 examples/server_node_efd/efd_server/main.c    |   1 -
 examples/vhost/main.c                         |   6 +-
 examples/vmdq/main.c                          |   7 +-
 examples/vmdq_dcb/main.c                      |  15 +--
 lib/cryptodev/rte_cryptodev.c                 |   2 +-
 lib/eal/common/eal_common_options.c           |   7 +-
 lib/ethdev/rte_ethdev.c                       |  12 +--
 lib/ethdev/rte_flow.c                         | 101 +++++++++---------
 lib/eventdev/rte_event_crypto_adapter.c       |   2 +-
 lib/eventdev/rte_event_dma_adapter.c          |   4 +-
 lib/eventdev/rte_event_timer_adapter.c        |   2 +-
 lib/fib/trie.c                                |   2 +-
 lib/ip_frag/rte_ipv6_fragmentation.c          |   4 +-
 lib/ip_frag/rte_ipv6_reassembly.c             |   6 +-
 lib/lpm/rte_lpm6.c                            |   3 +-
 lib/net/rte_ether.c                           |   2 +-
 lib/node/ip6_lookup.c                         |   8 +-
 lib/pdcp/pdcp_process.c                       |  36 +++----
 lib/pipeline/rte_table_action.c               |   8 +-
 lib/rib/rte_rib6.h                            |   5 +-
 lib/security/rte_security.c                   |   4 +-
 131 files changed, 368 insertions(+), 450 deletions(-)
 create mode 100644 devtools/cocci/rte_memcpy.cocci

Comments

Mattias Rönnblom May 26, 2024, 2:51 p.m. UTC | #1 
On 2024-05-22 05:27, Stephen Hemminger wrote:
> The DPDK has a lot of unnecessary usage of rte_memcpy.
> This patch set replaces cases where rte_memcpy is used with a fixed
> size constant size.
> 
> Typical example is:
> 	rte_memcpy(mac_addrs, mac.addr_bytes, RTE_ETHER_ADDR_LEN);
> which can be replaced with:
> 	memcpy(mac_addrs, mac.addr_bytes, RTE_ETHER_ADDR_LEN);
> 
> This does not change the resulting binary on almost all architectures
> because x86 version of intrisics and glibc are the same, and
> other architectures were using __builtin_constant_p().
> 
> The main benefit is that analysis tools like fortify, Coverity, and ASAN
> analyzers can check these memcpy's. A recent example is that
> on Ubuntu 22.04 detected undefined use of memcpy such as:
> 	memcpy(dst, NULL, 0)
> 

Would it be possible to instead have a build mode where rte_memcpy() is 
*always* delegating to memcpy(), and run these tools on that configuration?

It seems easier to just always using rte_memcpy() in DPDK code, then let 
the platform (not the programmer) choose whatever is most appropriate.

> The first patch is a simple coccinelle script to do the replacement
> and the rest are the results broken out by module. The script can be used
> again to make sure more bad usage doesn't creep in with new drivers.
> 
> v5 - rebase and fix a couple of the SW drivers.
>       rewording of commit messages
> 
> v4 - replace other unnecessary rte_memcpy in ethdev.
>       replace memcpy with structure assignment where possible.
>       fixup formatting in a couple places.
> 
> v3 - rebase and rerun coccinelle script on 24.03
>     - consolidate patches by subtree
>     - remove inclusion of rte_memcpy.h when no longer used.
> 
> Stephen Hemminger (32):
>    cocci/rte_memcpy: add script to eliminate fixed size rte_memcpy
>    eal: replace use of fixed size rte_memcpy
>    ethdev: replace uses of rte_memcpy
>    eventdev: replace use of fixed size rte_memcpy
>    cryptodev: replace use of fixed size rte_memcpy
>    ip_frag: replace use of fixed size rte_memcpy
>    net: replace use of fixed size rte_memcpy
>    lpm: replace use of fixed size rte_memcpy
>    node: replace use of fixed size rte_memcpy
>    pdcp: replace use of fixed size rte_memcpy
>    pipeline: replace use of fixed size rte_memcpy
>    rib: replace use of fixed size rte_memcpy
>    security: replace use of fixed size rte_memcpy
>    bus: remove unneeded rte_memcpy.h include
>    raw: replace use of fixed size rte_memcpy
>    baseband: replace use of fixed size rte_memcpy
>    common: replace use of fixed size rte_memcpy
>    crypto: replace use of fixed size rte_memcpy
>    event: replace use of fixed size rte_memcpy
>    mempool: replace use of fixed size rte_memcpy
>    ml/cnxk: replace use of fixed size rte_memcpy
>    app/test-pmd: replace use of fixed size rte_memcpy
>    app/graph: replace use of fixed size rte_memcpy
>    app/test-eventdev: replace use of fixed size rte_memcpy
>    app/test: replace use of fixed size rte_memcpy
>    app/test-pipeline: remove unused rte_memcpy.h include
>    app/test-bbdev: remove unnecessary include of rte_memcpy.h
>    examples: replace use of fixed size rte_memcpy
>    net/null: replace use of fixed size memcpy
>    net/tap: replace use of fixed size rte_memcpy
>    net/pcap: replace use of fixed size rte_memcpy
>    net/af_xdp:: replace use of fixed size rte_memcpy
> 
>   app/graph/neigh.c                             |   8 +-
>   app/test-bbdev/test_bbdev.c                   |   1 -
>   app/test-eventdev/test_pipeline_common.c      |  19 ++--
>   app/test-pipeline/config.c                    |   1 -
>   app/test-pipeline/init.c                      |   1 -
>   app/test-pipeline/main.c                      |   1 -
>   app/test-pipeline/runtime.c                   |   1 -
>   app/test-pmd/cmdline.c                        |  48 ++++-----
>   app/test-pmd/cmdline_flow.c                   |  24 ++---
>   app/test-pmd/config.c                         |   8 +-
>   app/test-pmd/csumonly.c                       |   1 -
>   app/test-pmd/flowgen.c                        |   1 -
>   app/test-pmd/iofwd.c                          |   1 -
>   app/test-pmd/macfwd.c                         |   1 -
>   app/test-pmd/macswap.c                        |   1 -
>   app/test-pmd/noisy_vnf.c                      |   1 -
>   app/test-pmd/rxonly.c                         |   1 -
>   app/test-pmd/testpmd.c                        |   1 -
>   app/test/commands.c                           |   1 -
>   app/test/packet_burst_generator.c             |   4 +-
>   app/test/test_crc.c                           |   5 +-
>   app/test/test_cryptodev.c                     |  18 ++--
>   app/test/test_cryptodev_asym.c                |   1 -
>   app/test/test_cryptodev_security_pdcp.c       |   1 -
>   app/test/test_efd.c                           |   1 -
>   app/test/test_efd_perf.c                      |   1 -
>   app/test/test_event_crypto_adapter.c          |  12 +--
>   app/test/test_event_dma_adapter.c             |   4 +-
>   app/test/test_eventdev.c                      |   1 -
>   app/test/test_ipsec.c                         |   6 +-
>   app/test/test_link_bonding_mode4.c            |   8 +-
>   app/test/test_mbuf.c                          |   1 -
>   app/test/test_member.c                        |   1 -
>   app/test/test_member_perf.c                   |   1 -
>   app/test/test_rawdev.c                        |   1 -
>   app/test/test_security_inline_proto.c         |  36 +++----
>   app/test/test_service_cores.c                 |   1 -
>   app/test/virtual_pmd.c                        |   3 +-
>   devtools/cocci/rte_memcpy.cocci               |  11 ++
>   drivers/baseband/acc/rte_acc100_pmd.c         |  16 ++-
>   drivers/baseband/acc/rte_vrb_pmd.c            |  21 ++--
>   drivers/baseband/la12xx/bbdev_la12xx.c        |   4 +-
>   drivers/bus/auxiliary/linux/auxiliary.c       |   1 -
>   drivers/bus/fslmc/fslmc_bus.c                 |   1 -
>   drivers/bus/fslmc/fslmc_vfio.c                |   1 -
>   drivers/bus/fslmc/portal/dpaa2_hw_dpbp.c      |   1 -
>   drivers/bus/fslmc/portal/dpaa2_hw_dpci.c      |   1 -
>   drivers/bus/fslmc/portal/dpaa2_hw_dpio.c      |   1 -
>   drivers/bus/pci/linux/pci.c                   |   1 -
>   drivers/common/idpf/idpf_common_device.c      |   4 +-
>   drivers/common/idpf/idpf_common_virtchnl.c    |  10 +-
>   drivers/common/qat/qat_qp.c                   |  10 +-
>   drivers/compress/qat/qat_comp.c               |   8 +-
>   drivers/crypto/ccp/ccp_crypto.c               |  14 +--
>   drivers/crypto/cnxk/cnxk_cryptodev_ops.c      |   2 +-
>   drivers/crypto/cnxk/cnxk_se.h                 |   2 +-
>   drivers/crypto/dpaa_sec/dpaa_sec.c            |   2 +-
>   drivers/crypto/ipsec_mb/pmd_snow3g.c          |   4 +-
>   drivers/crypto/qat/qat_sym_session.c          |  52 +++++----
>   .../scheduler/rte_cryptodev_scheduler.c       |   6 +-
>   drivers/crypto/scheduler/scheduler_failover.c |  12 +--
>   drivers/event/cnxk/cnxk_eventdev_selftest.c   |   1 -
>   drivers/event/cnxk/cnxk_tim_evdev.c           |   4 +-
>   drivers/event/dlb2/dlb2.c                     |   4 +-
>   drivers/event/dpaa/dpaa_eventdev.c            |   1 -
>   drivers/event/dpaa2/dpaa2_eventdev.c          |   7 +-
>   drivers/event/dpaa2/dpaa2_eventdev_selftest.c |   1 -
>   drivers/event/dpaa2/dpaa2_hw_dpcon.c          |   1 -
>   drivers/event/octeontx/ssovf_evdev_selftest.c |   1 -
>   drivers/event/octeontx/timvf_evdev.c          |   4 +-
>   drivers/mempool/dpaa/dpaa_mempool.c           |   4 +-
>   drivers/mempool/dpaa2/dpaa2_hw_mempool.c      |   5 +-
>   drivers/ml/cnxk/cn10k_ml_model.c              |   8 +-
>   drivers/ml/cnxk/cn10k_ml_ops.c                |  11 +-
>   drivers/ml/cnxk/cnxk_ml_ops.c                 |   2 +-
>   drivers/ml/cnxk/mvtvm_ml_model.c              |   8 +-
>   drivers/ml/cnxk/mvtvm_ml_ops.c                |   8 +-
>   drivers/net/af_xdp/rte_eth_af_xdp.c           |   2 +-
>   drivers/net/null/rte_eth_null.c               |   6 +-
>   drivers/net/pcap/pcap_ethdev.c                |   2 +-
>   drivers/net/pcap/pcap_osdep_freebsd.c         |   2 +-
>   drivers/net/pcap/pcap_osdep_linux.c           |   2 +-
>   drivers/net/tap/rte_eth_tap.c                 |  14 +--
>   drivers/raw/ifpga/afu_pmd_he_hssi.c           |   3 +-
>   drivers/raw/ifpga/afu_pmd_he_lpbk.c           |   3 +-
>   drivers/raw/ifpga/afu_pmd_he_mem.c            |   3 +-
>   drivers/raw/ifpga/afu_pmd_n3000.c             |   8 +-
>   drivers/raw/ifpga/ifpga_rawdev.c              |  11 +-
>   drivers/raw/skeleton/skeleton_rawdev.c        |   8 +-
>   drivers/raw/skeleton/skeleton_rawdev_test.c   |   1 -
>   examples/bbdev_app/main.c                     |   2 +-
>   examples/bond/main.c                          |   1 -
>   examples/ip_fragmentation/main.c              |   1 -
>   examples/ip_reassembly/main.c                 |   1 -
>   examples/ipv4_multicast/main.c                |   1 -
>   examples/l2fwd-cat/cat.c                      |   4 +-
>   examples/l2fwd-jobstats/main.c                |   1 -
>   examples/l2fwd-keepalive/main.c               |   1 -
>   examples/l2fwd-macsec/main.c                  |   1 -
>   examples/l2fwd/main.c                         |   1 -
>   examples/l3fwd-power/main.c                   |   1 -
>   examples/l3fwd/main.c                         |   1 -
>   examples/link_status_interrupt/main.c         |   1 -
>   .../client_server_mp/mp_server/init.c         |   1 -
>   .../client_server_mp/mp_server/main.c         |   1 -
>   examples/multi_process/symmetric_mp/main.c    |   1 -
>   examples/ptpclient/ptpclient.c                |  11 +-
>   examples/qos_sched/app_thread.c               |   1 -
>   examples/qos_sched/main.c                     |   1 -
>   examples/server_node_efd/efd_server/init.c    |   1 -
>   examples/server_node_efd/efd_server/main.c    |   1 -
>   examples/vhost/main.c                         |   6 +-
>   examples/vmdq/main.c                          |   7 +-
>   examples/vmdq_dcb/main.c                      |  15 +--
>   lib/cryptodev/rte_cryptodev.c                 |   2 +-
>   lib/eal/common/eal_common_options.c           |   7 +-
>   lib/ethdev/rte_ethdev.c                       |  12 +--
>   lib/ethdev/rte_flow.c                         | 101 +++++++++---------
>   lib/eventdev/rte_event_crypto_adapter.c       |   2 +-
>   lib/eventdev/rte_event_dma_adapter.c          |   4 +-
>   lib/eventdev/rte_event_timer_adapter.c        |   2 +-
>   lib/fib/trie.c                                |   2 +-
>   lib/ip_frag/rte_ipv6_fragmentation.c          |   4 +-
>   lib/ip_frag/rte_ipv6_reassembly.c             |   6 +-
>   lib/lpm/rte_lpm6.c                            |   3 +-
>   lib/net/rte_ether.c                           |   2 +-
>   lib/node/ip6_lookup.c                         |   8 +-
>   lib/pdcp/pdcp_process.c                       |  36 +++----
>   lib/pipeline/rte_table_action.c               |   8 +-
>   lib/rib/rte_rib6.h                            |   5 +-
>   lib/security/rte_security.c                   |   4 +-
>   131 files changed, 368 insertions(+), 450 deletions(-)
>   create mode 100644 devtools/cocci/rte_memcpy.cocci
>
Stephen Hemminger May 26, 2024, 11:32 p.m. UTC | #2 
On Sun, 26 May 2024 16:51:52 +0200
Mattias Rönnblom <hofors@lysator.liu.se> wrote:

> > This does not change the resulting binary on almost all architectures
> > because x86 version of intrisics and glibc are the same, and
> > other architectures were using __builtin_constant_p().
> > 
> > The main benefit is that analysis tools like fortify, Coverity, and ASAN
> > analyzers can check these memcpy's. A recent example is that
> > on Ubuntu 22.04 detected undefined use of memcpy such as:
> > 	memcpy(dst, NULL, 0)
> >   
> 
> Would it be possible to instead have a build mode where rte_memcpy() is 
> *always* delegating to memcpy(), and run these tools on that configuration?
> 
> It seems easier to just always using rte_memcpy() in DPDK code, then let 
> the platform (not the programmer) choose whatever is most appropriate.

I would prefer that rte_memcpy be laid to rest and go away.
It never had a reason to be there.
Mattias Rönnblom May 27, 2024, 6:06 a.m. UTC | #3 
On 2024-05-27 01:32, Stephen Hemminger wrote:
> On Sun, 26 May 2024 16:51:52 +0200
> Mattias Rönnblom <hofors@lysator.liu.se> wrote:
> 
>>> This does not change the resulting binary on almost all architectures
>>> because x86 version of intrisics and glibc are the same, and
>>> other architectures were using __builtin_constant_p().
>>>
>>> The main benefit is that analysis tools like fortify, Coverity, and ASAN
>>> analyzers can check these memcpy's. A recent example is that
>>> on Ubuntu 22.04 detected undefined use of memcpy such as:
>>> 	memcpy(dst, NULL, 0)
>>>    
>>
>> Would it be possible to instead have a build mode where rte_memcpy() is
>> *always* delegating to memcpy(), and run these tools on that configuration?
>>
>> It seems easier to just always using rte_memcpy() in DPDK code, then let
>> the platform (not the programmer) choose whatever is most appropriate.
> 
> I would prefer that rte_memcpy be laid to rest and go away.

Sounds good, but is this piecemeal approach the way to do it. Why ask 
DPDK programmers to memorize rules about when to use memcpy() and when 
not to, when that can be coded into the rte_memcpy() implementation.

Worst case, when people have learned those rules, we'll change them. 
"Use rte_memcpy() only for large copies," or whatever it might be.

Use rte_memcpy() across the board until memcpy() is a complete 
replacement, for every use case, for every CPU/compiler/OS.

Applications using DPDK will benefit from any heuristics encoded into 
rte_memcpy.h. The won't run coccinelle script.

> It never had a reason to be there.

Sure, it did. Historically it has had better performance, up to at least 
fairly recently that seemed still be the case. I've patched DPDK to 
replace rte_memcpy() with memcpy() several times, and run with 
real-world apps. The result is always the same.

The compiler is in a better position than DPDK to implement memcpy(), so 
eventually the compiler will win.
Morten Brørup May 27, 2024, 6:38 a.m. UTC | #4 
> I would prefer that rte_memcpy be laid to rest and go away.
> It never had a reason to be there.

DPDK still contains remains of hand crafted code dating back from a time where compilers (read: the oldest compilers supported by DPDK) didn't produce efficient code.
Rte_memcpy() is a prime example of this.
There are other examples, e.g. the use of Duff's Device directly in C source code.

This code was written a decade ago, and modern compilers are now much better at producing efficient code.

Testing would be able to verify the theory that modern compilers (read: the oldest compilers supported by DPDK) produce code that is as efficient as the hand crafted code.
If so, the hand crafted code has outlived its purpose, and should be considered obsolete cruft and be completely removed.

Until we are certain that removing it doesn't degrade performance, we should keep it where it makes a difference (in the fast path).

Special thanks goes to Stephen for his efforts in cleaning up DPDK!