diff mbox series

vhost: fix OOB access for invalid vid

Message ID 20230227105927.3643421-1-david.marchand@redhat.com (mailing list archive)
State Accepted, archived
Delegated to: Maxime Coquelin
Headers
Series vhost: fix OOB access for invalid vid |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK
ci/intel-Testing success Testing PASS
ci/intel-Performance success Performance Testing PASS
ci/github-robot: build success github build: passed
ci/iol-mellanox-Performance success Performance Testing PASS
ci/iol-broadcom-Performance success Performance Testing PASS
ci/iol-broadcom-Functional success Functional Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/iol-intel-Functional success Functional Testing PASS
ci/iol-aarch64-unit-testing success Testing PASS
ci/iol-abi-testing success Testing PASS
ci/iol-testing success Testing PASS
ci/iol-x86_64-unit-testing success Testing PASS
ci/iol-x86_64-compile-testing success Testing PASS
ci/iol-aarch64-compile-testing success Testing PASS
ci/loongarch-compilation success Compilation OK
ci/loongarch-unit-testing success Unit Testing PASS

Commit Message

David Marchand Feb. 27, 2023, 10:59 a.m. UTC 
  The net/vhost pmd currently provides a -1 vid when disabling interrupt
after a virtio port got disconnected.

This can be caught when running with ASan.

First, start dpdk-l3fwd-power in interrupt mode with a net/vhost port.

$ ./build-clang/examples/dpdk-l3fwd-power -l0,1 --in-memory \
	-a 0000:00:00.0 \
	--vdev net_vhost0,iface=plop.sock,client=1\
	-- \
	-p 0x1 \
	--interrupt-only \
	--config '(0,0,1)' \
	--parse-ptype 0

Then start testpmd with virtio-user.

$ ./build-clang/app/dpdk-testpmd -l0,2 --single-file-segment --in-memory \
	-a 0000:00:00.0 \
	--vdev net_virtio_user0,path=plop.sock,server=1 \
	-- \
	-i

Finally stop testpmd.
ASan then splats in dpdk-l3fwd-power:

=================================================================
==3641005==ERROR: AddressSanitizer: global-buffer-overflow on address
	0x000005ed0778 at pc 0x000001270f81 bp 0x7fddbd2eee20
	sp 0x7fddbd2eee18
READ of size 8 at 0x000005ed0778 thread T2
    #0 0x1270f80 in get_device .../lib/vhost/vhost.h:801:27
    #1 0x1270f80 in rte_vhost_get_vhost_vring .../lib/vhost/vhost.c:951:8
    #2 0x3ac95cb in eth_rxq_intr_disable
	.../drivers/net/vhost/rte_eth_vhost.c:647:8
    #3 0x170e0bf in rte_eth_dev_rx_intr_disable
	.../lib/ethdev/rte_ethdev.c:5443:25
    #4 0xf72ba7 in turn_on_off_intr .../examples/l3fwd-power/main.c:881:4
    #5 0xf71045 in main_intr_loop .../examples/l3fwd-power/main.c:1061:6
    #6 0x17f9292 in eal_thread_loop
	.../lib/eal/common/eal_common_thread.c:210:9
    #7 0x18373f5 in eal_worker_thread_loop .../lib/eal/linux/eal.c:915:2
    #8 0x7fddc16ae12c in start_thread (/lib64/libc.so.6+0x8b12c)
	(BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
    #9 0x7fddc172fbbf in __GI___clone3 (/lib64/libc.so.6+0x10cbbf)
	(BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)

0x000005ed0778 is located 8 bytes to the left of global variable
	'vhost_devices' defined in '.../lib/vhost/vhost.c:24'
	(0x5ed0780) of size 8192
0x000005ed0778 is located 20 bytes to the right of global variable
	'vhost_config_log_level' defined in '.../lib/vhost/vhost.c:2174'
	(0x5ed0760) of size 4
SUMMARY: AddressSanitizer: global-buffer-overflow
	.../lib/vhost/vhost.h:801:27 in get_device
Shadow bytes around the buggy address:
  0x000080bd2090: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
  0x000080bd20a0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
  0x000080bd20b0: f9 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9
  0x000080bd20c0: 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 04 f9 f9 f9
  0x000080bd20d0: 00 00 00 00 00 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
=>0x000080bd20e0: 00 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 04 f9 f9[f9]
  0x000080bd20f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x000080bd2100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x000080bd2110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x000080bd2120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x000080bd2130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
Thread T2 created by T0 here:
    #0 0xe98996 in __interceptor_pthread_create
	(.examples/dpdk-l3fwd-power+0xe98996)
	(BuildId: d0b984a3b0287b9e0f301b73426fa921aeecca3a)
    #1 0x1836767 in eal_worker_thread_create .../lib/eal/linux/eal.c:952:6
    #2 0x1834b83 in rte_eal_init .../lib/eal/linux/eal.c:1257:9
    #3 0xf68902 in main .../examples/l3fwd-power/main.c:2496:8
    #4 0x7fddc164a50f in __libc_start_call_main (/lib64/libc.so.6+0x2750f)
	(BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)

==3641005==ABORTING

More generally, any application passing an incorrect vid would trigger
such an OOB access.

Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
---
Note: even after this patch, reconnecting the virtio-user port with
dpdk-l3fwd-power in interrupt mode still seems broken, as the net/vhost
pmd keeps complaining about an issue with rx interrupt fds.

---
 lib/vhost/vhost.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

Comments

Maxime Coquelin Feb. 27, 2023, 4:19 p.m. UTC | #1 
On 2/27/23 11:59, David Marchand wrote:
> The net/vhost pmd currently provides a -1 vid when disabling interrupt
> after a virtio port got disconnected.
> 
> This can be caught when running with ASan.
> 
> First, start dpdk-l3fwd-power in interrupt mode with a net/vhost port.
> 
> $ ./build-clang/examples/dpdk-l3fwd-power -l0,1 --in-memory \
> 	-a 0000:00:00.0 \
> 	--vdev net_vhost0,iface=plop.sock,client=1\
> 	-- \
> 	-p 0x1 \
> 	--interrupt-only \
> 	--config '(0,0,1)' \
> 	--parse-ptype 0
> 
> Then start testpmd with virtio-user.
> 
> $ ./build-clang/app/dpdk-testpmd -l0,2 --single-file-segment --in-memory \
> 	-a 0000:00:00.0 \
> 	--vdev net_virtio_user0,path=plop.sock,server=1 \
> 	-- \
> 	-i
> 
> Finally stop testpmd.
> ASan then splats in dpdk-l3fwd-power:
> 
> =================================================================
> ==3641005==ERROR: AddressSanitizer: global-buffer-overflow on address
> 	0x000005ed0778 at pc 0x000001270f81 bp 0x7fddbd2eee20
> 	sp 0x7fddbd2eee18
> READ of size 8 at 0x000005ed0778 thread T2
>      #0 0x1270f80 in get_device .../lib/vhost/vhost.h:801:27
>      #1 0x1270f80 in rte_vhost_get_vhost_vring .../lib/vhost/vhost.c:951:8
>      #2 0x3ac95cb in eth_rxq_intr_disable
> 	.../drivers/net/vhost/rte_eth_vhost.c:647:8
>      #3 0x170e0bf in rte_eth_dev_rx_intr_disable
> 	.../lib/ethdev/rte_ethdev.c:5443:25
>      #4 0xf72ba7 in turn_on_off_intr .../examples/l3fwd-power/main.c:881:4
>      #5 0xf71045 in main_intr_loop .../examples/l3fwd-power/main.c:1061:6
>      #6 0x17f9292 in eal_thread_loop
> 	.../lib/eal/common/eal_common_thread.c:210:9
>      #7 0x18373f5 in eal_worker_thread_loop .../lib/eal/linux/eal.c:915:2
>      #8 0x7fddc16ae12c in start_thread (/lib64/libc.so.6+0x8b12c)
> 	(BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
>      #9 0x7fddc172fbbf in __GI___clone3 (/lib64/libc.so.6+0x10cbbf)
> 	(BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
> 
> 0x000005ed0778 is located 8 bytes to the left of global variable
> 	'vhost_devices' defined in '.../lib/vhost/vhost.c:24'
> 	(0x5ed0780) of size 8192
> 0x000005ed0778 is located 20 bytes to the right of global variable
> 	'vhost_config_log_level' defined in '.../lib/vhost/vhost.c:2174'
> 	(0x5ed0760) of size 4
> SUMMARY: AddressSanitizer: global-buffer-overflow
> 	.../lib/vhost/vhost.h:801:27 in get_device
> Shadow bytes around the buggy address:
>    0x000080bd2090: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
>    0x000080bd20a0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
>    0x000080bd20b0: f9 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9
>    0x000080bd20c0: 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 04 f9 f9 f9
>    0x000080bd20d0: 00 00 00 00 00 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
> =>0x000080bd20e0: 00 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 04 f9 f9[f9]
>    0x000080bd20f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>    0x000080bd2100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>    0x000080bd2110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>    0x000080bd2120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>    0x000080bd2130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> Shadow byte legend (one shadow byte represents 8 application bytes):
>    Addressable:           00
>    Partially addressable: 01 02 03 04 05 06 07
>    Heap left redzone:       fa
>    Freed heap region:       fd
>    Stack left redzone:      f1
>    Stack mid redzone:       f2
>    Stack right redzone:     f3
>    Stack after return:      f5
>    Stack use after scope:   f8
>    Global redzone:          f9
>    Global init order:       f6
>    Poisoned by user:        f7
>    Container overflow:      fc
>    Array cookie:            ac
>    Intra object redzone:    bb
>    ASan internal:           fe
>    Left alloca redzone:     ca
>    Right alloca redzone:    cb
> Thread T2 created by T0 here:
>      #0 0xe98996 in __interceptor_pthread_create
> 	(.examples/dpdk-l3fwd-power+0xe98996)
> 	(BuildId: d0b984a3b0287b9e0f301b73426fa921aeecca3a)
>      #1 0x1836767 in eal_worker_thread_create .../lib/eal/linux/eal.c:952:6
>      #2 0x1834b83 in rte_eal_init .../lib/eal/linux/eal.c:1257:9
>      #3 0xf68902 in main .../examples/l3fwd-power/main.c:2496:8
>      #4 0x7fddc164a50f in __libc_start_call_main (/lib64/libc.so.6+0x2750f)
> 	(BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
> 
> ==3641005==ABORTING
> 
> More generally, any application passing an incorrect vid would trigger
> such an OOB access.
> 
> Cc: stable@dpdk.org
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> Note: even after this patch, reconnecting the virtio-user port with
> dpdk-l3fwd-power in interrupt mode still seems broken, as the net/vhost
> pmd keeps complaining about an issue with rx interrupt fds.
> 
> ---
>   lib/vhost/vhost.h | 5 ++++-
>   1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
> index 5750f0c005..d9e97280fa 100644
> --- a/lib/vhost/vhost.h
> +++ b/lib/vhost/vhost.h
> @@ -798,7 +798,10 @@ hva_to_gpa(struct virtio_net *dev, uint64_t vva, uint64_t len)
>   static __rte_always_inline struct virtio_net *
>   get_device(int vid)
>   {
> -	struct virtio_net *dev = vhost_devices[vid];
> +	struct virtio_net *dev = NULL;
> +
> +	if (vid >= 0 && vid < RTE_MAX_VHOST_DEVICE)

Maybe use likely() to help the compiler?

> +		dev = vhost_devices[vid];
>   
>   	if (unlikely(!dev)) {
>   		VHOST_LOG_CONFIG("device", ERR, "(%d) device not found.\n", vid);
David Marchand Feb. 28, 2023, 8:27 a.m. UTC | #2 
On Mon, Feb 27, 2023 at 5:19 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
> On 2/27/23 11:59, David Marchand wrote:
> > The net/vhost pmd currently provides a -1 vid when disabling interrupt
> > after a virtio port got disconnected.
> >
> > This can be caught when running with ASan.
> >
> > First, start dpdk-l3fwd-power in interrupt mode with a net/vhost port.
> >
> > $ ./build-clang/examples/dpdk-l3fwd-power -l0,1 --in-memory \
> >       -a 0000:00:00.0 \
> >       --vdev net_vhost0,iface=plop.sock,client=1\
> >       -- \
> >       -p 0x1 \
> >       --interrupt-only \
> >       --config '(0,0,1)' \
> >       --parse-ptype 0
> >
> > Then start testpmd with virtio-user.
> >
> > $ ./build-clang/app/dpdk-testpmd -l0,2 --single-file-segment --in-memory \
> >       -a 0000:00:00.0 \
> >       --vdev net_virtio_user0,path=plop.sock,server=1 \
> >       -- \
> >       -i
> >
> > Finally stop testpmd.
> > ASan then splats in dpdk-l3fwd-power:
> >
> > =================================================================
> > ==3641005==ERROR: AddressSanitizer: global-buffer-overflow on address
> >       0x000005ed0778 at pc 0x000001270f81 bp 0x7fddbd2eee20
> >       sp 0x7fddbd2eee18
> > READ of size 8 at 0x000005ed0778 thread T2
> >      #0 0x1270f80 in get_device .../lib/vhost/vhost.h:801:27
> >      #1 0x1270f80 in rte_vhost_get_vhost_vring .../lib/vhost/vhost.c:951:8
> >      #2 0x3ac95cb in eth_rxq_intr_disable
> >       .../drivers/net/vhost/rte_eth_vhost.c:647:8
> >      #3 0x170e0bf in rte_eth_dev_rx_intr_disable
> >       .../lib/ethdev/rte_ethdev.c:5443:25
> >      #4 0xf72ba7 in turn_on_off_intr .../examples/l3fwd-power/main.c:881:4
> >      #5 0xf71045 in main_intr_loop .../examples/l3fwd-power/main.c:1061:6
> >      #6 0x17f9292 in eal_thread_loop
> >       .../lib/eal/common/eal_common_thread.c:210:9
> >      #7 0x18373f5 in eal_worker_thread_loop .../lib/eal/linux/eal.c:915:2
> >      #8 0x7fddc16ae12c in start_thread (/lib64/libc.so.6+0x8b12c)
> >       (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
> >      #9 0x7fddc172fbbf in __GI___clone3 (/lib64/libc.so.6+0x10cbbf)
> >       (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
> >
> > 0x000005ed0778 is located 8 bytes to the left of global variable
> >       'vhost_devices' defined in '.../lib/vhost/vhost.c:24'
> >       (0x5ed0780) of size 8192
> > 0x000005ed0778 is located 20 bytes to the right of global variable
> >       'vhost_config_log_level' defined in '.../lib/vhost/vhost.c:2174'
> >       (0x5ed0760) of size 4
> > SUMMARY: AddressSanitizer: global-buffer-overflow
> >       .../lib/vhost/vhost.h:801:27 in get_device
> > Shadow bytes around the buggy address:
> >    0x000080bd2090: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
> >    0x000080bd20a0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
> >    0x000080bd20b0: f9 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9
> >    0x000080bd20c0: 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 04 f9 f9 f9
> >    0x000080bd20d0: 00 00 00 00 00 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
> > =>0x000080bd20e0: 00 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 04 f9 f9[f9]
> >    0x000080bd20f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >    0x000080bd2100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >    0x000080bd2110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >    0x000080bd2120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> >    0x000080bd2130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > Shadow byte legend (one shadow byte represents 8 application bytes):
> >    Addressable:           00
> >    Partially addressable: 01 02 03 04 05 06 07
> >    Heap left redzone:       fa
> >    Freed heap region:       fd
> >    Stack left redzone:      f1
> >    Stack mid redzone:       f2
> >    Stack right redzone:     f3
> >    Stack after return:      f5
> >    Stack use after scope:   f8
> >    Global redzone:          f9
> >    Global init order:       f6
> >    Poisoned by user:        f7
> >    Container overflow:      fc
> >    Array cookie:            ac
> >    Intra object redzone:    bb
> >    ASan internal:           fe
> >    Left alloca redzone:     ca
> >    Right alloca redzone:    cb
> > Thread T2 created by T0 here:
> >      #0 0xe98996 in __interceptor_pthread_create
> >       (.examples/dpdk-l3fwd-power+0xe98996)
> >       (BuildId: d0b984a3b0287b9e0f301b73426fa921aeecca3a)
> >      #1 0x1836767 in eal_worker_thread_create .../lib/eal/linux/eal.c:952:6
> >      #2 0x1834b83 in rte_eal_init .../lib/eal/linux/eal.c:1257:9
> >      #3 0xf68902 in main .../examples/l3fwd-power/main.c:2496:8
> >      #4 0x7fddc164a50f in __libc_start_call_main (/lib64/libc.so.6+0x2750f)
> >       (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
> >
> > ==3641005==ABORTING
> >
> > More generally, any application passing an incorrect vid would trigger
> > such an OOB access.
> >
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > ---
> > Note: even after this patch, reconnecting the virtio-user port with
> > dpdk-l3fwd-power in interrupt mode still seems broken, as the net/vhost
> > pmd keeps complaining about an issue with rx interrupt fds.
> >
> > ---
> >   lib/vhost/vhost.h | 5 ++++-
> >   1 file changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
> > index 5750f0c005..d9e97280fa 100644
> > --- a/lib/vhost/vhost.h
> > +++ b/lib/vhost/vhost.h
> > @@ -798,7 +798,10 @@ hva_to_gpa(struct virtio_net *dev, uint64_t vva, uint64_t len)
> >   static __rte_always_inline struct virtio_net *
> >   get_device(int vid)
> >   {
> > -     struct virtio_net *dev = vhost_devices[vid];
> > +     struct virtio_net *dev = NULL;
> > +
> > +     if (vid >= 0 && vid < RTE_MAX_VHOST_DEVICE)
>
> Maybe use likely() to help the compiler?

I see no difference in x86 generated code with gcc 12 but I don't mind
adding it (or you can add it while applying ? :-)).


>
> > +             dev = vhost_devices[vid];
> >
> >       if (unlikely(!dev)) {
> >               VHOST_LOG_CONFIG("device", ERR, "(%d) device not found.\n", vid);
Maxime Coquelin Feb. 28, 2023, 9:24 a.m. UTC | #3 
On 2/28/23 09:27, David Marchand wrote:
> On Mon, Feb 27, 2023 at 5:19 PM Maxime Coquelin
> <maxime.coquelin@redhat.com> wrote:
>> On 2/27/23 11:59, David Marchand wrote:
>>> The net/vhost pmd currently provides a -1 vid when disabling interrupt
>>> after a virtio port got disconnected.
>>>
>>> This can be caught when running with ASan.
>>>
>>> First, start dpdk-l3fwd-power in interrupt mode with a net/vhost port.
>>>
>>> $ ./build-clang/examples/dpdk-l3fwd-power -l0,1 --in-memory \
>>>        -a 0000:00:00.0 \
>>>        --vdev net_vhost0,iface=plop.sock,client=1\
>>>        -- \
>>>        -p 0x1 \
>>>        --interrupt-only \
>>>        --config '(0,0,1)' \
>>>        --parse-ptype 0
>>>
>>> Then start testpmd with virtio-user.
>>>
>>> $ ./build-clang/app/dpdk-testpmd -l0,2 --single-file-segment --in-memory \
>>>        -a 0000:00:00.0 \
>>>        --vdev net_virtio_user0,path=plop.sock,server=1 \
>>>        -- \
>>>        -i
>>>
>>> Finally stop testpmd.
>>> ASan then splats in dpdk-l3fwd-power:
>>>
>>> =================================================================
>>> ==3641005==ERROR: AddressSanitizer: global-buffer-overflow on address
>>>        0x000005ed0778 at pc 0x000001270f81 bp 0x7fddbd2eee20
>>>        sp 0x7fddbd2eee18
>>> READ of size 8 at 0x000005ed0778 thread T2
>>>       #0 0x1270f80 in get_device .../lib/vhost/vhost.h:801:27
>>>       #1 0x1270f80 in rte_vhost_get_vhost_vring .../lib/vhost/vhost.c:951:8
>>>       #2 0x3ac95cb in eth_rxq_intr_disable
>>>        .../drivers/net/vhost/rte_eth_vhost.c:647:8
>>>       #3 0x170e0bf in rte_eth_dev_rx_intr_disable
>>>        .../lib/ethdev/rte_ethdev.c:5443:25
>>>       #4 0xf72ba7 in turn_on_off_intr .../examples/l3fwd-power/main.c:881:4
>>>       #5 0xf71045 in main_intr_loop .../examples/l3fwd-power/main.c:1061:6
>>>       #6 0x17f9292 in eal_thread_loop
>>>        .../lib/eal/common/eal_common_thread.c:210:9
>>>       #7 0x18373f5 in eal_worker_thread_loop .../lib/eal/linux/eal.c:915:2
>>>       #8 0x7fddc16ae12c in start_thread (/lib64/libc.so.6+0x8b12c)
>>>        (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
>>>       #9 0x7fddc172fbbf in __GI___clone3 (/lib64/libc.so.6+0x10cbbf)
>>>        (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
>>>
>>> 0x000005ed0778 is located 8 bytes to the left of global variable
>>>        'vhost_devices' defined in '.../lib/vhost/vhost.c:24'
>>>        (0x5ed0780) of size 8192
>>> 0x000005ed0778 is located 20 bytes to the right of global variable
>>>        'vhost_config_log_level' defined in '.../lib/vhost/vhost.c:2174'
>>>        (0x5ed0760) of size 4
>>> SUMMARY: AddressSanitizer: global-buffer-overflow
>>>        .../lib/vhost/vhost.h:801:27 in get_device
>>> Shadow bytes around the buggy address:
>>>     0x000080bd2090: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
>>>     0x000080bd20a0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
>>>     0x000080bd20b0: f9 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9
>>>     0x000080bd20c0: 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 04 f9 f9 f9
>>>     0x000080bd20d0: 00 00 00 00 00 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
>>> =>0x000080bd20e0: 00 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 04 f9 f9[f9]
>>>     0x000080bd20f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>>     0x000080bd2100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>>     0x000080bd2110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>>     0x000080bd2120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>>     0x000080bd2130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> Shadow byte legend (one shadow byte represents 8 application bytes):
>>>     Addressable:           00
>>>     Partially addressable: 01 02 03 04 05 06 07
>>>     Heap left redzone:       fa
>>>     Freed heap region:       fd
>>>     Stack left redzone:      f1
>>>     Stack mid redzone:       f2
>>>     Stack right redzone:     f3
>>>     Stack after return:      f5
>>>     Stack use after scope:   f8
>>>     Global redzone:          f9
>>>     Global init order:       f6
>>>     Poisoned by user:        f7
>>>     Container overflow:      fc
>>>     Array cookie:            ac
>>>     Intra object redzone:    bb
>>>     ASan internal:           fe
>>>     Left alloca redzone:     ca
>>>     Right alloca redzone:    cb
>>> Thread T2 created by T0 here:
>>>       #0 0xe98996 in __interceptor_pthread_create
>>>        (.examples/dpdk-l3fwd-power+0xe98996)
>>>        (BuildId: d0b984a3b0287b9e0f301b73426fa921aeecca3a)
>>>       #1 0x1836767 in eal_worker_thread_create .../lib/eal/linux/eal.c:952:6
>>>       #2 0x1834b83 in rte_eal_init .../lib/eal/linux/eal.c:1257:9
>>>       #3 0xf68902 in main .../examples/l3fwd-power/main.c:2496:8
>>>       #4 0x7fddc164a50f in __libc_start_call_main (/lib64/libc.so.6+0x2750f)
>>>        (BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
>>>
>>> ==3641005==ABORTING
>>>
>>> More generally, any application passing an incorrect vid would trigger
>>> such an OOB access.
>>>
>>> Cc: stable@dpdk.org
>>>
>>> Signed-off-by: David Marchand <david.marchand@redhat.com>
>>> ---
>>> Note: even after this patch, reconnecting the virtio-user port with
>>> dpdk-l3fwd-power in interrupt mode still seems broken, as the net/vhost
>>> pmd keeps complaining about an issue with rx interrupt fds.
>>>
>>> ---
>>>    lib/vhost/vhost.h | 5 ++++-
>>>    1 file changed, 4 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
>>> index 5750f0c005..d9e97280fa 100644
>>> --- a/lib/vhost/vhost.h
>>> +++ b/lib/vhost/vhost.h
>>> @@ -798,7 +798,10 @@ hva_to_gpa(struct virtio_net *dev, uint64_t vva, uint64_t len)
>>>    static __rte_always_inline struct virtio_net *
>>>    get_device(int vid)
>>>    {
>>> -     struct virtio_net *dev = vhost_devices[vid];
>>> +     struct virtio_net *dev = NULL;
>>> +
>>> +     if (vid >= 0 && vid < RTE_MAX_VHOST_DEVICE)
>>
>> Maybe use likely() to help the compiler?
> 
> I see no difference in x86 generated code with gcc 12 but I don't mind
> adding it (or you can add it while applying ? :-)).

I can add it while applying.

Thanks!
Maxime

> 
> 
>>
>>> +             dev = vhost_devices[vid];
>>>
>>>        if (unlikely(!dev)) {
>>>                VHOST_LOG_CONFIG("device", ERR, "(%d) device not found.\n", vid);
> 
>
Maxime Coquelin March 6, 2023, 2:07 p.m. UTC | #4 
On 2/27/23 11:59, David Marchand wrote:
> The net/vhost pmd currently provides a -1 vid when disabling interrupt
> after a virtio port got disconnected.
> 
> This can be caught when running with ASan.
> 
> First, start dpdk-l3fwd-power in interrupt mode with a net/vhost port.
> 
> $ ./build-clang/examples/dpdk-l3fwd-power -l0,1 --in-memory \
> 	-a 0000:00:00.0 \
> 	--vdev net_vhost0,iface=plop.sock,client=1\
> 	-- \
> 	-p 0x1 \
> 	--interrupt-only \
> 	--config '(0,0,1)' \
> 	--parse-ptype 0
> 
> Then start testpmd with virtio-user.
> 
> $ ./build-clang/app/dpdk-testpmd -l0,2 --single-file-segment --in-memory \
> 	-a 0000:00:00.0 \
> 	--vdev net_virtio_user0,path=plop.sock,server=1 \
> 	-- \
> 	-i
> 
> Finally stop testpmd.
> ASan then splats in dpdk-l3fwd-power:
> 
> =================================================================
> ==3641005==ERROR: AddressSanitizer: global-buffer-overflow on address
> 	0x000005ed0778 at pc 0x000001270f81 bp 0x7fddbd2eee20
> 	sp 0x7fddbd2eee18
> READ of size 8 at 0x000005ed0778 thread T2
>      #0 0x1270f80 in get_device .../lib/vhost/vhost.h:801:27
>      #1 0x1270f80 in rte_vhost_get_vhost_vring .../lib/vhost/vhost.c:951:8
>      #2 0x3ac95cb in eth_rxq_intr_disable
> 	.../drivers/net/vhost/rte_eth_vhost.c:647:8
>      #3 0x170e0bf in rte_eth_dev_rx_intr_disable
> 	.../lib/ethdev/rte_ethdev.c:5443:25
>      #4 0xf72ba7 in turn_on_off_intr .../examples/l3fwd-power/main.c:881:4
>      #5 0xf71045 in main_intr_loop .../examples/l3fwd-power/main.c:1061:6
>      #6 0x17f9292 in eal_thread_loop
> 	.../lib/eal/common/eal_common_thread.c:210:9
>      #7 0x18373f5 in eal_worker_thread_loop .../lib/eal/linux/eal.c:915:2
>      #8 0x7fddc16ae12c in start_thread (/lib64/libc.so.6+0x8b12c)
> 	(BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
>      #9 0x7fddc172fbbf in __GI___clone3 (/lib64/libc.so.6+0x10cbbf)
> 	(BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
> 
> 0x000005ed0778 is located 8 bytes to the left of global variable
> 	'vhost_devices' defined in '.../lib/vhost/vhost.c:24'
> 	(0x5ed0780) of size 8192
> 0x000005ed0778 is located 20 bytes to the right of global variable
> 	'vhost_config_log_level' defined in '.../lib/vhost/vhost.c:2174'
> 	(0x5ed0760) of size 4
> SUMMARY: AddressSanitizer: global-buffer-overflow
> 	.../lib/vhost/vhost.h:801:27 in get_device
> Shadow bytes around the buggy address:
>    0x000080bd2090: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
>    0x000080bd20a0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
>    0x000080bd20b0: f9 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9
>    0x000080bd20c0: 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 04 f9 f9 f9
>    0x000080bd20d0: 00 00 00 00 00 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
> =>0x000080bd20e0: 00 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 04 f9 f9[f9]
>    0x000080bd20f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>    0x000080bd2100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>    0x000080bd2110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>    0x000080bd2120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>    0x000080bd2130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> Shadow byte legend (one shadow byte represents 8 application bytes):
>    Addressable:           00
>    Partially addressable: 01 02 03 04 05 06 07
>    Heap left redzone:       fa
>    Freed heap region:       fd
>    Stack left redzone:      f1
>    Stack mid redzone:       f2
>    Stack right redzone:     f3
>    Stack after return:      f5
>    Stack use after scope:   f8
>    Global redzone:          f9
>    Global init order:       f6
>    Poisoned by user:        f7
>    Container overflow:      fc
>    Array cookie:            ac
>    Intra object redzone:    bb
>    ASan internal:           fe
>    Left alloca redzone:     ca
>    Right alloca redzone:    cb
> Thread T2 created by T0 here:
>      #0 0xe98996 in __interceptor_pthread_create
> 	(.examples/dpdk-l3fwd-power+0xe98996)
> 	(BuildId: d0b984a3b0287b9e0f301b73426fa921aeecca3a)
>      #1 0x1836767 in eal_worker_thread_create .../lib/eal/linux/eal.c:952:6
>      #2 0x1834b83 in rte_eal_init .../lib/eal/linux/eal.c:1257:9
>      #3 0xf68902 in main .../examples/l3fwd-power/main.c:2496:8
>      #4 0x7fddc164a50f in __libc_start_call_main (/lib64/libc.so.6+0x2750f)
> 	(BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
> 
> ==3641005==ABORTING
> 
> More generally, any application passing an incorrect vid would trigger
> such an OOB access.
> 
> Cc: stable@dpdk.org
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> Note: even after this patch, reconnecting the virtio-user port with
> dpdk-l3fwd-power in interrupt mode still seems broken, as the net/vhost
> pmd keeps complaining about an issue with rx interrupt fds.
> 
> ---
>   lib/vhost/vhost.h | 5 ++++-
>   1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
> index 5750f0c005..d9e97280fa 100644
> --- a/lib/vhost/vhost.h
> +++ b/lib/vhost/vhost.h
> @@ -798,7 +798,10 @@ hva_to_gpa(struct virtio_net *dev, uint64_t vva, uint64_t len)
>   static __rte_always_inline struct virtio_net *
>   get_device(int vid)
>   {
> -	struct virtio_net *dev = vhost_devices[vid];
> +	struct virtio_net *dev = NULL;
> +
> +	if (vid >= 0 && vid < RTE_MAX_VHOST_DEVICE)
> +		dev = vhost_devices[vid];
>   
>   	if (unlikely(!dev)) {
>   		VHOST_LOG_CONFIG("device", ERR, "(%d) device not found.\n", vid);

Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Thanks,
Maxime
Maxime Coquelin March 6, 2023, 2:22 p.m. UTC | #5 
On 2/27/23 11:59, David Marchand wrote:
> The net/vhost pmd currently provides a -1 vid when disabling interrupt
> after a virtio port got disconnected.
> 
> This can be caught when running with ASan.
> 
> First, start dpdk-l3fwd-power in interrupt mode with a net/vhost port.
> 
> $ ./build-clang/examples/dpdk-l3fwd-power -l0,1 --in-memory \
> 	-a 0000:00:00.0 \
> 	--vdev net_vhost0,iface=plop.sock,client=1\
> 	-- \
> 	-p 0x1 \
> 	--interrupt-only \
> 	--config '(0,0,1)' \
> 	--parse-ptype 0
> 
> Then start testpmd with virtio-user.
> 
> $ ./build-clang/app/dpdk-testpmd -l0,2 --single-file-segment --in-memory \
> 	-a 0000:00:00.0 \
> 	--vdev net_virtio_user0,path=plop.sock,server=1 \
> 	-- \
> 	-i
> 
> Finally stop testpmd.
> ASan then splats in dpdk-l3fwd-power:
> 
> =================================================================
> ==3641005==ERROR: AddressSanitizer: global-buffer-overflow on address
> 	0x000005ed0778 at pc 0x000001270f81 bp 0x7fddbd2eee20
> 	sp 0x7fddbd2eee18
> READ of size 8 at 0x000005ed0778 thread T2
>      #0 0x1270f80 in get_device .../lib/vhost/vhost.h:801:27
>      #1 0x1270f80 in rte_vhost_get_vhost_vring .../lib/vhost/vhost.c:951:8
>      #2 0x3ac95cb in eth_rxq_intr_disable
> 	.../drivers/net/vhost/rte_eth_vhost.c:647:8
>      #3 0x170e0bf in rte_eth_dev_rx_intr_disable
> 	.../lib/ethdev/rte_ethdev.c:5443:25
>      #4 0xf72ba7 in turn_on_off_intr .../examples/l3fwd-power/main.c:881:4
>      #5 0xf71045 in main_intr_loop .../examples/l3fwd-power/main.c:1061:6
>      #6 0x17f9292 in eal_thread_loop
> 	.../lib/eal/common/eal_common_thread.c:210:9
>      #7 0x18373f5 in eal_worker_thread_loop .../lib/eal/linux/eal.c:915:2
>      #8 0x7fddc16ae12c in start_thread (/lib64/libc.so.6+0x8b12c)
> 	(BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
>      #9 0x7fddc172fbbf in __GI___clone3 (/lib64/libc.so.6+0x10cbbf)
> 	(BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
> 
> 0x000005ed0778 is located 8 bytes to the left of global variable
> 	'vhost_devices' defined in '.../lib/vhost/vhost.c:24'
> 	(0x5ed0780) of size 8192
> 0x000005ed0778 is located 20 bytes to the right of global variable
> 	'vhost_config_log_level' defined in '.../lib/vhost/vhost.c:2174'
> 	(0x5ed0760) of size 4
> SUMMARY: AddressSanitizer: global-buffer-overflow
> 	.../lib/vhost/vhost.h:801:27 in get_device
> Shadow bytes around the buggy address:
>    0x000080bd2090: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
>    0x000080bd20a0: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
>    0x000080bd20b0: f9 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9 00 f9 f9 f9
>    0x000080bd20c0: 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 04 f9 f9 f9
>    0x000080bd20d0: 00 00 00 00 00 f9 f9 f9 f9 f9 f9 f9 00 00 00 00
> =>0x000080bd20e0: 00 f9 f9 f9 f9 f9 f9 f9 04 f9 f9 f9 04 f9 f9[f9]
>    0x000080bd20f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>    0x000080bd2100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>    0x000080bd2110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>    0x000080bd2120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>    0x000080bd2130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> Shadow byte legend (one shadow byte represents 8 application bytes):
>    Addressable:           00
>    Partially addressable: 01 02 03 04 05 06 07
>    Heap left redzone:       fa
>    Freed heap region:       fd
>    Stack left redzone:      f1
>    Stack mid redzone:       f2
>    Stack right redzone:     f3
>    Stack after return:      f5
>    Stack use after scope:   f8
>    Global redzone:          f9
>    Global init order:       f6
>    Poisoned by user:        f7
>    Container overflow:      fc
>    Array cookie:            ac
>    Intra object redzone:    bb
>    ASan internal:           fe
>    Left alloca redzone:     ca
>    Right alloca redzone:    cb
> Thread T2 created by T0 here:
>      #0 0xe98996 in __interceptor_pthread_create
> 	(.examples/dpdk-l3fwd-power+0xe98996)
> 	(BuildId: d0b984a3b0287b9e0f301b73426fa921aeecca3a)
>      #1 0x1836767 in eal_worker_thread_create .../lib/eal/linux/eal.c:952:6
>      #2 0x1834b83 in rte_eal_init .../lib/eal/linux/eal.c:1257:9
>      #3 0xf68902 in main .../examples/l3fwd-power/main.c:2496:8
>      #4 0x7fddc164a50f in __libc_start_call_main (/lib64/libc.so.6+0x2750f)
> 	(BuildId: 81daba31ee66dbd63efdc4252a872949d874d136)
> 
> ==3641005==ABORTING
> 
> More generally, any application passing an incorrect vid would trigger
> such an OOB access.
> 
> Cc: stable@dpdk.org
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> ---
> Note: even after this patch, reconnecting the virtio-user port with
> dpdk-l3fwd-power in interrupt mode still seems broken, as the net/vhost
> pmd keeps complaining about an issue with rx interrupt fds.
> 
> ---
>   lib/vhost/vhost.h | 5 ++++-
>   1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
> index 5750f0c005..d9e97280fa 100644
> --- a/lib/vhost/vhost.h
> +++ b/lib/vhost/vhost.h
> @@ -798,7 +798,10 @@ hva_to_gpa(struct virtio_net *dev, uint64_t vva, uint64_t len)
>   static __rte_always_inline struct virtio_net *
>   get_device(int vid)
>   {
> -	struct virtio_net *dev = vhost_devices[vid];
> +	struct virtio_net *dev = NULL;
> +
> +	if (vid >= 0 && vid < RTE_MAX_VHOST_DEVICE)
> +		dev = vhost_devices[vid];
>   
>   	if (unlikely(!dev)) {
>   		VHOST_LOG_CONFIG("device", ERR, "(%d) device not found.\n", vid);

Applied to dpdk-next-virtio/main.

Thanks,
Maxime
diff mbox series

Patch

diff --git a/lib/vhost/vhost.h b/lib/vhost/vhost.h
index 5750f0c005..d9e97280fa 100644
--- a/lib/vhost/vhost.h
+++ b/lib/vhost/vhost.h
@@ -798,7 +798,10 @@  hva_to_gpa(struct virtio_net *dev, uint64_t vva, uint64_t len)
 static __rte_always_inline struct virtio_net *
 get_device(int vid)
 {
-	struct virtio_net *dev = vhost_devices[vid];
+	struct virtio_net *dev = NULL;
+
+	if (vid >= 0 && vid < RTE_MAX_VHOST_DEVICE)
+		dev = vhost_devices[vid];
 
 	if (unlikely(!dev)) {
 		VHOST_LOG_CONFIG("device", ERR, "(%d) device not found.\n", vid);