[v9,0/2] support both PIO and MMIO BAR for legacy device in virtio PMD

  virtio PMD assumes legacy device only supports PIO BAR resource. This is wrong.
As we need to create lots of devices, as PIO resource on x86 is very limited, 
we expose MMIO(memory IO) BAR.

Kernel supports both PIO and MMIO BAR for legacy virtio-pci device, and for all
other pci devices. This patchset handles different type of BAR in the similar way.

In previous implementation, under igb_uio driver we get PIO address from igb_uio
sysfs entry; with uio_pci_generic, we get PIO address from /proc/ioports for x86,
and for other ARCHs, we get PIO address from standard PCI sysfs entry.
For PIO/MMIO RW, there is different path for different drivers and arch.


All of the above is too much twisted.
This patchset unifies the way to get both PIO and MMIO address for different driver
and ARCHs, all from standard resource attr under pci sysfs. This is most generic.

We distinguish PIO and MMIO by their address like how kernel does. It is ugly but works.

v2 changes:
    - add more explanation in the commit message

v3 changes:
    - fix patch format issues

v4 changes:
    - fixes for RTE_KDRV_UIO_GENERIC -> RTE_PCI_KDRV_UIO_GENERIC

v5 changes:
    - split into three seperate patches

v6 changes:
    - change to DEBUG level for IO bar detection in pci_uio_ioport_map
    - rework the code in iobar branch
    - fixes commit message format issue
    - temporarily remove the 3rd patch for vfio path, leave it for future discusssion
    - rework against virtio_pmd_rework_v2

v7 changes:
    - fix compilation issues of in/out instruction on non X86 archs

v8 changes:
    - change the word fix to refactor in patch 1's commit message

v9 changes:
    - keep pause version in in/out instructions

huawei.xhw (2):
  bus/pci: use PCI standard sysfs entry to get PIO address
  bus/pci: support MMIO in PCI ioport accessors

 drivers/bus/pci/linux/pci.c     |  81 ----------------
 drivers/bus/pci/linux/pci_uio.c | 202 +++++++++++++++++++++++++++++-----------
 2 files changed, 150 insertions(+), 133 deletions(-)
  

On Thu, 04 Mar 2021 01:46:50 +0800
"谢华伟(此时此刻）" <huawei.xhw@alibaba-inc.com> wrote:

> virtio PMD assumes legacy device only supports PIO BAR resource. This is wrong.
> As we need to create lots of devices, as PIO resource on x86 is very limited, 
> we expose MMIO(memory IO) BAR.
> 
> Kernel supports both PIO and MMIO BAR for legacy virtio-pci device, and for all
> other pci devices. This patchset handles different type of BAR in the similar way.
> 
> In previous implementation, under igb_uio driver we get PIO address from igb_uio
> sysfs entry; with uio_pci_generic, we get PIO address from /proc/ioports for x86,
> and for other ARCHs, we get PIO address from standard PCI sysfs entry.
> For PIO/MMIO RW, there is different path for different drivers and arch.

Just to add some background. At the time virtio for DPDK was developed,
the kernel only supported legacy mode, and it required I/O ports on x86.

One concern is that, you should make sure these patches still work on
the oldest releases of Linux kernel that DPDK supports. For upstream
kernel that should be 4.4 kernel (oldest currently maintained LTS).
The Linux system requirements doc file needs update!

For distributions, the oldest version would be probably be RHEL 7.
  

On 2021/3/4 2:24, Stephen Hemminger wrote:
> On Thu, 04 Mar 2021 01:46:50 +0800
> "谢华伟(此时此刻）"<huawei.xhw@alibaba-inc.com>  wrote:
>
>> virtio PMD assumes legacy device only supports PIO BAR resource. This is wrong.
>> As we need to create lots of devices, as PIO resource on x86 is very limited,
>> we expose MMIO(memory IO) BAR.
>>
>> Kernel supports both PIO and MMIO BAR for legacy virtio-pci device, and for all
>> other pci devices. This patchset handles different type of BAR in the similar way.
>>
>> In previous implementation, under igb_uio driver we get PIO address from igb_uio
>> sysfs entry; with uio_pci_generic, we get PIO address from /proc/ioports for x86,
>> and for other ARCHs, we get PIO address from standard PCI sysfs entry.
>> For PIO/MMIO RW, there is different path for different drivers and arch.
> Just to add some background. At the time virtio for DPDK was developed,
> the kernel only supported legacy mode, and it required I/O ports on x86.

Hi Stephen:

Do you mean QEMU?  I think QEMU exports PIO by default.

For kernel, I checked 3.10 code at hand, virtio legacy driver doesn't 
assume I/O port. It uses generic API pci_iomap and ioread/write.

>
> One concern is that, you should make sure these patches still work on
> the oldest releases of Linux kernel that DPDK supports. For upstream
> kernel that should be 4.4 kernel (oldest currently maintained LTS).
> The Linux system requirements doc file needs update!
>
> For distributions, the oldest version would be probably be RHEL 7.

PIO is verified on Redhat 7.9 (3.10.0-1160.11.1.el7.x86_64).

As this patch uses the generic kernel exported interface, which all 
other PMDs have been using, i think it doesn't break compatibility.