[RFC,0/1] integrate dmadev in vhost

Since dmadev is introduced in 21.11, to avoid the overhead of vhost DMA
abstraction layer and simplify application logics, this patch integrates
dmadev in vhost.

To enable the flexibility of using DMA devices in different function
modules, not limited in vhost, vhost doesn't manage DMA devices.
Applications, like OVS, need to manage and configure DMA devices and
tell vhost what DMA device to use in every dataplane function call.

In addition, vhost supports M:N mapping between vrings and DMA virtual
channels. Specifically, one vring can use multiple different DMA channels
and one DMA channel can be shared by multiple vrings at the same time.
The reason of enabling one vring to use multiple DMA channels is that
it's possible that more than one dataplane threads enqueue packets to
the same vring with their own DMA virtual channels. Besides, the number
of DMA devices is limited. For the purpose of scaling, it's necessary to
support sharing DMA channels among vrings.

As only enqueue path is enabled DMA acceleration, the new dataplane
functions are like:
1). rte_vhost_submit_enqueue_burst(vid, queue_id, pkts, count, dma_id,
    dma_vchan):
    Get descriptors and submit copies to DMA virtual channel for the
    packets that need to be send to VM.
 
2). rte_vhost_poll_enqueue_completed(vid, queue_id, pkts, count, dma_id,
    dma_vchan):
    Check completed DMA copies from the given DMA virtual channel and
    write back corresponding descriptors to vring.

OVS needs to call rte_vhost_poll_enqueue_completed to clean in-flight
copies on previous call and it can be called inside rxq_recv function,
so that it doesn't require big change in OVS datapath. For example:
netdev_dpdk_vhost_rxq_recv() {
	...
	qid = rxq->queue_id * VIRTIO_QNUM + VIRTIO_RXQ;
	rte_vhost_poll_enqueue_completed(vid, qid, ...);
}

Jiayu Hu (1):
  vhost: integrate dmadev in asynchronous datapath

 doc/guides/prog_guide/vhost_lib.rst |  63 ++++----
 examples/vhost/ioat.c               | 218 ----------------------------
 examples/vhost/ioat.h               |  63 --------
 examples/vhost/main.c               | 144 +++++++++++++++---
 examples/vhost/main.h               |  12 ++
 examples/vhost/meson.build          |   6 +-
 lib/vhost/meson.build               |   3 +-
 lib/vhost/rte_vhost_async.h         |  73 +++-------
 lib/vhost/vhost.c                   |  37 ++---
 lib/vhost/vhost.h                   |  45 +++++-
 lib/vhost/virtio_net.c              | 198 ++++++++++++++++++++-----
 11 files changed, 410 insertions(+), 452 deletions(-)
 delete mode 100644 examples/vhost/ioat.c
 delete mode 100644 examples/vhost/ioat.h

Hi Jiayu

I notice that examples/vhost rely on VMDQ, Could the examples/vhost provide
options that do not depend on VMDQ?  In this way, many network adapters can
be used.

Thanks.


On 2021/11/22 18:54, Jiayu Hu wrote:
> Since dmadev is introduced in 21.11, to avoid the overhead of vhost DMA
> abstraction layer and simplify application logics, this patch integrates
> dmadev in vhost.
> 
> To enable the flexibility of using DMA devices in different function
> modules, not limited in vhost, vhost doesn't manage DMA devices.
> Applications, like OVS, need to manage and configure DMA devices and
> tell vhost what DMA device to use in every dataplane function call.
> 
> In addition, vhost supports M:N mapping between vrings and DMA virtual
> channels. Specifically, one vring can use multiple different DMA channels
> and one DMA channel can be shared by multiple vrings at the same time.
> The reason of enabling one vring to use multiple DMA channels is that
> it's possible that more than one dataplane threads enqueue packets to
> the same vring with their own DMA virtual channels. Besides, the number
> of DMA devices is limited. For the purpose of scaling, it's necessary to
> support sharing DMA channels among vrings.
> 

...