diff mbox series

vdpa/ifc: add live migration for block device

Message ID 1671004060-62997-1-git-send-email-andy.pei@intel.com (mailing list archive)
State Accepted, archived
Delegated to: Maxime Coquelin
Headers
Series vdpa/ifc: add live migration for block device |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/iol-broadcom-Functional fail Functional Testing issues
ci/iol-mellanox-Performance success Performance Testing PASS
ci/iol-broadcom-Performance success Performance Testing PASS
ci/iol-intel-Functional success Functional Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/github-robot: build success github build: passed
ci/iol-aarch64-unit-testing success Testing PASS
ci/loongarch-compilation success Compilation OK
ci/loongarch-unit-testing success Unit Testing PASS
ci/iol-x86_64-compile-testing success Testing PASS
ci/iol-testing success Testing PASS
ci/iol-x86_64-unit-testing success Testing PASS
ci/iol-aarch64-compile-testing success Testing PASS
ci/Intel-compilation success Compilation OK
ci/intel-Testing success Testing PASS

Commit Message

Pei, Andy Dec. 14, 2022, 7:47 a.m. UTC 
  When we use "sw-live-migration=1" in arguments, we run in SW assisted 
live migration mode.

In SW assisted live migration mode, driver will stop the device and
setup a mediated virtio ring to relay the communication between the
virtio driver and the VDPA device.

For block devices, we have to make sure that each IO is completed.
When we terminate the above mediated virtio ring, we stop 
notifying queue about new IOs, wait for all the in-flight IOs to 
be completed.

Signed-off-by: Andy Pei <andy.pei@intel.com>
---
 drivers/vdpa/ifc/ifcvf_vdpa.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

Comments

Maxime Coquelin Jan. 2, 2023, 3:14 p.m. UTC | #1 
On 12/14/22 08:47, Andy Pei wrote:
> When we use "sw-live-migration=1" in arguments, we run in SW assisted
> live migration mode.
> 
> In SW assisted live migration mode, driver will stop the device and
> setup a mediated virtio ring to relay the communication between the
> virtio driver and the VDPA device.
> 
> For block devices, we have to make sure that each IO is completed.
> When we terminate the above mediated virtio ring, we stop
> notifying queue about new IOs, wait for all the in-flight IOs to
> be completed.
> 
> Signed-off-by: Andy Pei <andy.pei@intel.com>
> ---
>   drivers/vdpa/ifc/ifcvf_vdpa.c | 22 ++++++++++++++++++++++
>   1 file changed, 22 insertions(+)
> 

Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Thanks,
Maxime
Maxime Coquelin Feb. 3, 2023, 2:57 p.m. UTC | #2 
On 12/14/22 08:47, Andy Pei wrote:
> When we use "sw-live-migration=1" in arguments, we run in SW assisted
> live migration mode.
> 
> In SW assisted live migration mode, driver will stop the device and
> setup a mediated virtio ring to relay the communication between the
> virtio driver and the VDPA device.
> 
> For block devices, we have to make sure that each IO is completed.
> When we terminate the above mediated virtio ring, we stop
> notifying queue about new IOs, wait for all the in-flight IOs to
> be completed.
> 
> Signed-off-by: Andy Pei <andy.pei@intel.com>
> ---
>   drivers/vdpa/ifc/ifcvf_vdpa.c | 22 ++++++++++++++++++++++
>   1 file changed, 22 insertions(+)
> 

Applied to dpdk-next-virtio/main.

Thanks,
Maxime
diff mbox series

Patch

diff --git a/drivers/vdpa/ifc/ifcvf_vdpa.c b/drivers/vdpa/ifc/ifcvf_vdpa.c
index 49d68ad..17838ac 100644
--- a/drivers/vdpa/ifc/ifcvf_vdpa.c
+++ b/drivers/vdpa/ifc/ifcvf_vdpa.c
@@ -865,8 +865,30 @@  struct rte_vdpa_dev_info {
 	struct ifcvf_hw *hw = &internal->hw;
 	uint64_t m_vring_iova = IFCVF_MEDIATED_VRING;
 	uint64_t size, len;
+	u32 ring_state = 0;
 
 	vid = internal->vid;
+
+	/* to make sure no packet is lost for blk device
+	 * do not stop until last_avail_idx == last_used_idx
+	 */
+	if (internal->hw.device_type == IFCVF_BLK) {
+		for (i = 0; i < hw->nr_vring; i++) {
+			do {
+				if (hw->lm_cfg != NULL)
+					ring_state = *(u32 *)(hw->lm_cfg +
+						IFCVF_LM_RING_STATE_OFFSET +
+						i * IFCVF_LM_CFG_SIZE);
+				hw->vring[i].last_avail_idx =
+					(u16)(ring_state & IFCVF_16_BIT_MASK);
+				hw->vring[i].last_used_idx =
+					(u16)(ring_state >> 16);
+				usleep(10);
+			} while (hw->vring[i].last_avail_idx !=
+				hw->vring[i].last_used_idx);
+		}
+	}
+
 	ifcvf_stop_hw(hw);
 
 	for (i = 0; i < hw->nr_vring; i++) {