[dpdk-dev,v2] vhost: added user callbacks for socket open/close
Checks
Commit Message
When user receives destroy_device signal, he does not know *why* that
event happened. He does not differ between socket shutdown and virtio
processing pause. User could completely delete device during transition
from BIOS to kernel, causing freeze or possibly kernel panic. Instead
of changing new_device/destroy_device callbacks and breaking the ABI,
a set of new functions new_connection/destroy_connection has been added.
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
---
v2: also updated vhost_lib.rst
doc/guides/prog_guide/vhost_lib.rst | 15 +++++++++++++--
lib/librte_vhost/rte_vhost.h | 5 ++++-
lib/librte_vhost/socket.c | 23 +++++++++++++++++++----
3 files changed, 36 insertions(+), 7 deletions(-)
Comments
Hi Dariusz,
On Tue, Aug 22, 2017 at 06:24:52PM +0200, Dariusz Stojaczyk wrote:
>When user receives destroy_device signal, he does not know *why* that
>event happened. He does not differ between socket shutdown and virtio
>processing pause. User could completely delete device during transition
>from BIOS to kernel, causing freeze or possibly kernel panic. Instead
>of changing new_device/destroy_device callbacks and breaking the ABI,
>a set of new functions new_connection/destroy_connection has been added.
>
>Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
>---
>v2: also updated vhost_lib.rst
> doc/guides/prog_guide/vhost_lib.rst | 15 +++++++++++++--
> lib/librte_vhost/rte_vhost.h | 5 ++++-
> lib/librte_vhost/socket.c | 23 +++++++++++++++++++----
> 3 files changed, 36 insertions(+), 7 deletions(-)
thanks for adding documentation!
I'm still not sure I understand the use case. So just for my
understanding: users need to distinct between "the device is going
away temporarily, keep the connection" and "we're shutting down for good", is that it?
Maybe it's just me or maybe it means you could explain your example in
the commit message a bit more. I think the code looks sane, so
Reviewed-by: Jens Freimann <jfreimann@redhat.com>
Oh, and you should put the maintainers on Cc to get a faster review.
regards,
Jens
Hi Jens,
> I'm still not sure I understand the use case. So just for my
> understanding: users need to distinct between "the device is going away
> temporarily, keep the connection" and "we're shutting down for good", is
> that it?
Yes, exactly.
> Maybe it's just me or maybe it means you could explain your example in the
> commit message a bit more.
Ok. How about the following commit message instead:
```
rte_vhost: added user callbacks for socket open/close
Added new callbacks to notify about socket connection status.
As destroy_device is used for virtqueue processing *pause* as
well as connection close, the user has no distinction between those.
Consider the following scenario:
rte_vhost: received SET_VRING_BASE message,
calling destroy_device() as usual
user: end-user asks to remove the device (together with socket file),
OK, device is not *in use* - that's NOT the behavior we want
calling rte_vhost_driver_unregister() etc.
Instead of changing new_device/destroy_device callbacks and breaking
the ABI, a set of new functions new_connection/destroy_connection
has been added.
```
> Oh, and you should put the maintainers on Cc to get a faster review.
Thanks, I will!
Regards,
D.
On Tue, Aug 29, 2017 at 06:08:45AM +0000, Stojaczyk, DariuszX wrote:
>Hi Jens,
>
>> I'm still not sure I understand the use case. So just for my
>> understanding: users need to distinct between "the device is going away
>> temporarily, keep the connection" and "we're shutting down for good", is
>> that it?
>
>Yes, exactly.
>
>> Maybe it's just me or maybe it means you could explain your example in the
>> commit message a bit more.
>
>Ok. How about the following commit message instead:
>```
>rte_vhost: added user callbacks for socket open/close
>
>Added new callbacks to notify about socket connection status.
>As destroy_device is used for virtqueue processing *pause* as
>well as connection close, the user has no distinction between those.
>
>Consider the following scenario:
>rte_vhost: received SET_VRING_BASE message,
> calling destroy_device() as usual
>
>user: end-user asks to remove the device (together with socket file),
> OK, device is not *in use* - that's NOT the behavior we want
> calling rte_vhost_driver_unregister() etc.
>
>Instead of changing new_device/destroy_device callbacks and breaking
>the ABI, a set of new functions new_connection/destroy_connection
>has been added.
>```
Sounds good to me. Thanks!
regards,
Jens
@@ -129,8 +129,7 @@ The following is an overview of some key Vhost API functions:
* ``destroy_device(int vid)``
- This callback is invoked when a virtio device shuts down (or when the
- vhost connection is broken).
+ This callback is invoked when a virtio device is paused or shut down.
* ``vring_state_changed(int vid, uint16_t queue_id, int enable)``
@@ -143,6 +142,18 @@ The following is an overview of some key Vhost API functions:
``VHOST_F_LOG_ALL`` will be set/cleared at the start/end of live
migration, respectively.
+ * ``new_connection(int vid)``
+
+ This callback is invoked on new vhost-user socket connection. If DPDK
+ acts as the server the device should not be deleted before
+ ``destroy_connection`` callback is received.
+
+ * ``destroy_connection(int vid)``
+
+ This callback is invoked when vhost-user socket connection is closed.
+ It indicates that device with id ``vid`` is no longer in use and can be
+ safely deleted.
+
* ``rte_vhost_driver_disable/enable_features(path, features))``
This function disables/enables some features. For example, it can be used to
@@ -107,7 +107,10 @@ struct vhost_device_ops {
*/
int (*features_changed)(int vid, uint64_t features);
- void *reserved[4]; /**< Reserved for future extension */
+ int (*new_connection)(int vid); /**< Connect to socket. */
+ void (*destroy_connection)(int vid); /**< Disconnect from socket */
+
+ void *reserved[2]; /**< Reserved for future extension */
};
/**
@@ -230,24 +230,36 @@ vhost_user_add_connection(int fd, struct vhost_user_socket *vsocket)
RTE_LOG(INFO, VHOST_CONFIG, "new device, handle is %d\n", vid);
+ if (vsocket->notify_ops->new_connection) {
+ ret = vsocket->notify_ops->new_connection(vid);
+ if (ret < 0) {
+ RTE_LOG(ERR, VHOST_CONFIG,
+ "failed to add vhost user connection with fd %d\n",
+ fd);
+ goto err;
+ }
+ }
+
conn->connfd = fd;
conn->vsocket = vsocket;
conn->vid = vid;
ret = fdset_add(&vhost_user.fdset, fd, vhost_user_read_cb,
NULL, conn);
if (ret < 0) {
- conn->connfd = -1;
- free(conn);
- close(fd);
RTE_LOG(ERR, VHOST_CONFIG,
"failed to add fd %d into vhost server fdset\n",
fd);
- return;
+ goto err;
}
pthread_mutex_lock(&vsocket->conn_mutex);
TAILQ_INSERT_TAIL(&vsocket->conn_list, conn, next);
pthread_mutex_unlock(&vsocket->conn_mutex);
+ return;
+
+err:
+ free(conn);
+ close(fd);
}
/* call back when there is new vhost-user connection from client */
@@ -277,6 +289,9 @@ vhost_user_read_cb(int connfd, void *dat, int *remove)
*remove = 1;
vhost_destroy_device(conn->vid);
+ if (vsocket->notify_ops->destroy_connection)
+ vsocket->notify_ops->destroy_connection(conn->vid);
+
pthread_mutex_lock(&vsocket->conn_mutex);
TAILQ_REMOVE(&vsocket->conn_list, conn, next);
pthread_mutex_unlock(&vsocket->conn_mutex);