Hi all,
This the RFC I'm preparing to send to upstream QEMU for initial RFC review. A couple of limitations: I've not updated the protocol with the new msg_token field yet. We only support a single device per bus (dev_num = 0). The kernel driver only works as a module, when building it into the kernel it panics.
-------------------
This adds virtio-msg, a new virtio transport. Virtio-msg works by exchanging messages over a bus and doesn't rely on trapping and emulating making it a good fit for a number of applications such as AMP, real-time and safety applications.
Together with the new transport, this series adds a PCI device that implements an AMP setup much like it would look if two SoC's would use virtio-msg across a PCI link.
The virtio-msg spec: https://github.com/Linaro/virtio-msg-spec/
Linux with virtio-msg: https://github.com/edgarigl/linux/tree/edgari/virtio-msg-6.17
To try it, first build Linux with the following as modules: CONFIG_VIRTIO_MSG=m CONFIG_VIRTIO_MSG_AMP=m CONFIG_VIRTIO_MSG_AMP_PCI=m
Boot linux in QEMU with a virtio-msg-amp-pci device, in this example with a virtio-net device attached to it (x86/q35 machine): -device virtio-msg-amp-pci -device virtio-net-device,netdev=n1,bus=/q35-pcihost/pcie.0/virtio-msg-amp-pci/vmsg.0 -netdev user,id=nc
Modprobe: modprobe virtio_msg_transport.ko modprobe virtio_msg_amp.ko modprobe virtio_msg_amp_pci.ko
You now should see the virtio device.
Cheers, Edgar
Edgar E. Iglesias (4): virtio: Introduce notify_queue virtio: Add virtio_queue_get_rings virtio: Add the virtio-msg transport virtio-msg-bus: amp-pci: Add generic AMP PCI device
hw/misc/Kconfig | 7 + hw/misc/meson.build | 1 + hw/misc/virtio-msg-amp-pci.c | 324 ++++++++++++ hw/virtio/Kconfig | 4 + hw/virtio/meson.build | 5 + hw/virtio/virtio-msg-bus.c | 89 ++++ hw/virtio/virtio-msg.c | 596 ++++++++++++++++++++++ hw/virtio/virtio.c | 23 + include/hw/virtio/virtio-bus.h | 1 + include/hw/virtio/virtio-msg-bus.h | 95 ++++ include/hw/virtio/virtio-msg-prot.h | 747 ++++++++++++++++++++++++++++ include/hw/virtio/virtio-msg.h | 45 ++ include/hw/virtio/virtio.h | 2 + 13 files changed, 1939 insertions(+) create mode 100644 hw/misc/virtio-msg-amp-pci.c create mode 100644 hw/virtio/virtio-msg-bus.c create mode 100644 hw/virtio/virtio-msg.c create mode 100644 include/hw/virtio/virtio-msg-bus.h create mode 100644 include/hw/virtio/virtio-msg-prot.h create mode 100644 include/hw/virtio/virtio-msg.h
Signed-off-by: Edgar E. Iglesias edgar.iglesias@amd.com --- hw/virtio/virtio.c | 7 +++++++ include/hw/virtio/virtio-bus.h | 1 + 2 files changed, 8 insertions(+)
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 153ee0a0cf..8a53fb5f93 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -2700,12 +2700,19 @@ static void virtio_irq(VirtQueue *vq)
void virtio_notify(VirtIODevice *vdev, VirtQueue *vq) { + BusState *qbus = qdev_get_parent_bus(DEVICE(vdev)); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + WITH_RCU_READ_LOCK_GUARD() { if (!virtio_should_notify(vdev, vq)) { return; } }
+ if (k->notify_queue) { + k->notify_queue(qbus->parent, virtio_get_queue_index(vq)); + } + trace_virtio_notify(vdev, vq); virtio_irq(vq); } diff --git a/include/hw/virtio/virtio-bus.h b/include/hw/virtio/virtio-bus.h index 7ab8c9dab0..043dbeb4cf 100644 --- a/include/hw/virtio/virtio-bus.h +++ b/include/hw/virtio/virtio-bus.h @@ -39,6 +39,7 @@ DECLARE_OBJ_CHECKERS(VirtioBusState, VirtioBusClass, struct VirtioBusClass { /* This is what a VirtioBus must implement */ BusClass parent; + void (*notify_queue)(DeviceState *d, uint16_t index); void (*notify)(DeviceState *d, uint16_t vector); void (*save_config)(DeviceState *d, QEMUFile *f); void (*save_queue)(DeviceState *d, int n, QEMUFile *f);
Signed-off-by: Edgar E. Iglesias edgar.iglesias@amd.com --- hw/virtio/virtio.c | 16 ++++++++++++++++ include/hw/virtio/virtio.h | 2 ++ 2 files changed, 18 insertions(+)
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 8a53fb5f93..fe7c635390 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -2379,6 +2379,22 @@ void virtio_queue_set_rings(VirtIODevice *vdev, int n, hwaddr desc, virtio_init_region_cache(vdev, n); }
+void virtio_queue_get_rings(VirtIODevice *vdev, int n, hwaddr *desc, + hwaddr *avail, hwaddr *used) +{ + assert(vdev->vq[n].vring.num); + + if (desc) { + *desc = vdev->vq[n].vring.desc; + } + if (avail) { + *avail = vdev->vq[n].vring.avail; + } + if (used) { + *used = vdev->vq[n].vring.used; + } +} + void virtio_queue_set_num(VirtIODevice *vdev, int n, int num) { /* Don't allow guest to flip queue between existent and diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index d97529c3f1..8bceb115a3 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -361,6 +361,8 @@ int virtio_queue_get_max_num(VirtIODevice *vdev, int n); int virtio_get_num_queues(VirtIODevice *vdev); void virtio_queue_set_rings(VirtIODevice *vdev, int n, hwaddr desc, hwaddr avail, hwaddr used); +void virtio_queue_get_rings(VirtIODevice *vdev, int n, hwaddr *desc, + hwaddr *avail, hwaddr *used); void virtio_queue_update_rings(VirtIODevice *vdev, int n); void virtio_init_region_cache(VirtIODevice *vdev, int n); void virtio_queue_set_align(VirtIODevice *vdev, int n, int align);
Add the new virtio-msg transport together with a framework for virtio-msg buses.
Signed-off-by: Edgar E. Iglesias edgar.iglesias@amd.com --- hw/virtio/Kconfig | 4 + hw/virtio/meson.build | 5 + hw/virtio/virtio-msg-bus.c | 89 ++++ hw/virtio/virtio-msg.c | 596 ++++++++++++++++++++++ include/hw/virtio/virtio-msg-bus.h | 95 ++++ include/hw/virtio/virtio-msg-prot.h | 747 ++++++++++++++++++++++++++++ include/hw/virtio/virtio-msg.h | 45 ++ 7 files changed, 1581 insertions(+) create mode 100644 hw/virtio/virtio-msg-bus.c create mode 100644 hw/virtio/virtio-msg.c create mode 100644 include/hw/virtio/virtio-msg-bus.h create mode 100644 include/hw/virtio/virtio-msg-prot.h create mode 100644 include/hw/virtio/virtio-msg.h
diff --git a/hw/virtio/Kconfig b/hw/virtio/Kconfig index 10f5c53ac0..d2d8c475b2 100644 --- a/hw/virtio/Kconfig +++ b/hw/virtio/Kconfig @@ -26,6 +26,10 @@ config VIRTIO_MMIO bool select VIRTIO
+config VIRTIO_MSG + bool + select VIRTIO + config VIRTIO_CCW bool select VIRTIO diff --git a/hw/virtio/meson.build b/hw/virtio/meson.build index 48b9fedfa5..9624296977 100644 --- a/hw/virtio/meson.build +++ b/hw/virtio/meson.build @@ -13,6 +13,11 @@ specific_virtio_ss = ss.source_set() specific_virtio_ss.add(files('virtio.c')) specific_virtio_ss.add(files('virtio-qmp.c'))
+specific_virtio_ss.add(when: 'CONFIG_VIRTIO_MSG', if_true: files( + 'virtio-msg.c', + 'virtio-msg-bus.c', +)) + if have_vhost system_virtio_ss.add(files('vhost.c')) system_virtio_ss.add(files('vhost-backend.c', 'vhost-iova-tree.c')) diff --git a/hw/virtio/virtio-msg-bus.c b/hw/virtio/virtio-msg-bus.c new file mode 100644 index 0000000000..e907fd64ec --- /dev/null +++ b/hw/virtio/virtio-msg-bus.c @@ -0,0 +1,89 @@ +/* + * VirtIO MSG bus. + * + * Copyright (c) 2024 Advanced Micro Devices, Inc. + * Written by Edgar E. Iglesias edgar.iglesias@amd.com + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "qemu/units.h" +#include "hw/virtio/virtio-msg-bus.h" + +bool virtio_msg_bus_connect(BusState *bus, + const VirtIOMSGBusPort *port, + void *opaque) +{ + VirtIOMSGBusDevice *bd = virtio_msg_bus_get_device(bus); + + if (!bd) { + /* Nothing connected to this virtio-msg device. Ignore. */ + return false; + } + + bd->peer = port; + bd->opaque = opaque; + return true; +} + +void virtio_msg_bus_process(VirtIOMSGBusDevice *bd) +{ + VirtIOMSGBusDeviceClass *bdc; + bdc = VIRTIO_MSG_BUS_DEVICE_CLASS(object_get_class(OBJECT(bd))); + + bdc->process(bd); +} + +int virtio_msg_bus_send(BusState *bus, VirtIOMSG *msg_req) +{ + VirtIOMSGBusDevice *bd = virtio_msg_bus_get_device(bus); + VirtIOMSGBusDeviceClass *bdc; + int r = VIRTIO_MSG_NO_ERROR; + + bdc = VIRTIO_MSG_BUS_DEVICE_CLASS(object_get_class(OBJECT(bd))); + + if (bdc->send) { + r = bdc->send(bd, msg_req); + } + return r; +} + +static void virtio_msg_bus_class_init(ObjectClass *klass, const void *data) +{ + BusClass *bc = BUS_CLASS(klass); + + bc->max_dev = 1; +} + +static const TypeInfo virtio_msg_bus_info = { + .name = TYPE_VIRTIO_MSG_BUS, + .parent = TYPE_BUS, + .instance_size = sizeof(BusState), + .class_init = virtio_msg_bus_class_init, +}; + +static void virtio_msg_bus_device_class_init(ObjectClass *klass, + const void *data) +{ + DeviceClass *k = DEVICE_CLASS(klass); + + k->bus_type = TYPE_VIRTIO_MSG_BUS; +} + +static const TypeInfo virtio_msg_bus_device_type_info = { + .name = TYPE_VIRTIO_MSG_BUS_DEVICE, + .parent = TYPE_DEVICE, + .instance_size = sizeof(VirtIOMSGBusDevice), + .abstract = true, + .class_size = sizeof(VirtIOMSGBusDeviceClass), + .class_init = virtio_msg_bus_device_class_init, +}; + +static void virtio_msg_bus_register_types(void) +{ + type_register_static(&virtio_msg_bus_info); + type_register_static(&virtio_msg_bus_device_type_info); +} + +type_init(virtio_msg_bus_register_types) diff --git a/hw/virtio/virtio-msg.c b/hw/virtio/virtio-msg.c new file mode 100644 index 0000000000..e578dcaabd --- /dev/null +++ b/hw/virtio/virtio-msg.c @@ -0,0 +1,596 @@ +/* + * Virtio MSG bindings + * + * Copyright (c) 2024 Advanced Micro Devices, Inc. + * Written by Edgar E. Iglesias edgar.iglesias@amd.com. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "qemu/units.h" +#include "qapi/error.h" +#include "hw/irq.h" +#include "hw/qdev-properties.h" +#include "hw/virtio/virtio.h" +#include "hw/virtio/virtio-msg-bus.h" +#include "hw/virtio/virtio-msg.h" +#include "qemu/error-report.h" +#include "qemu/log.h" +#include "trace.h" + +static bool virtio_msg_bad(VirtIOMSGProxy *s, VirtIOMSG *msg) +{ + bool drop = false; + size_t min_size; + unsigned int n; + + min_size = virtio_msg_header_size(); + switch (msg->msg_id) { + case VIRTIO_MSG_GET_DEVICE_STATUS: + case VIRTIO_MSG_DEVICE_INFO: + break; + case VIRTIO_MSG_GET_FEATURES: + min_size += sizeof msg->get_features; + break; + case VIRTIO_MSG_SET_FEATURES: + n = msg->set_features.num; + + /* We expect at least one feature block. */ + if (n == 0 || n > VIRTIO_MSG_MAX_FEATURE_NUM) { + drop = true; + break; + } + + min_size += sizeof msg->set_features + n * 4; + break; + case VIRTIO_MSG_GET_CONFIG: + min_size += sizeof msg->get_config; + break; + case VIRTIO_MSG_SET_CONFIG: + if (msg->set_config.size > VIRTIO_MSG_MAX_CONFIG_BYTES) { + drop = true; + break; + } + + min_size += sizeof msg->set_config + msg->set_config.size; + break; + case VIRTIO_MSG_SET_DEVICE_STATUS: + min_size += sizeof msg->set_device_status; + break; + case VIRTIO_MSG_GET_VQUEUE: + min_size += sizeof msg->get_vqueue; + break; + case VIRTIO_MSG_SET_VQUEUE: + min_size += sizeof msg->set_vqueue; + break; + case VIRTIO_MSG_RESET_VQUEUE: + min_size += sizeof msg->reset_vqueue; + break; + case VIRTIO_MSG_EVENT_AVAIL: + min_size += sizeof msg->event_avail; + break; + default: + /* Unexpected message. */ + drop = true; + break; + } + + /* Accept large messages allowing future backwards compatible extensions. */ + if (drop || + msg->msg_size < min_size || msg->msg_size > VIRTIO_MSG_MAX_SIZE) { + return true; + } + + return false; +} + +static void virtio_msg_device_info(VirtIOMSGProxy *s, + VirtIOMSG *msg) +{ + VirtIODevice *vdev = virtio_bus_get_device(&s->bus); + uint32_t device_id = 0; + VirtIOMSG msg_resp; + + if (vdev) { + device_id = vdev->device_id; + } else { + error_report("%s: No virtio device on bus %s!", + __func__, BUS(&s->bus)->name); + } + + virtio_msg_pack_get_device_info_resp(&msg_resp, device_id, + VIRTIO_MSG_VENDOR_ID, + /* Feature bits */ + 64, + vdev->config_len, + VIRTIO_QUEUE_MAX, + 0, 0); + virtio_msg_bus_send(&s->msg_bus, &msg_resp); +} + +static void virtio_msg_get_features(VirtIOMSGProxy *s, + VirtIOMSG *msg) +{ + VirtIODevice *vdev = virtio_bus_get_device(&s->bus); + VirtioDeviceClass *vdc = VIRTIO_DEVICE_GET_CLASS(vdev); + VirtIOMSG msg_resp; + uint32_t index = msg->get_features.index; + uint32_t f[VIRTIO_MSG_MAX_FEATURE_NUM] = { 0 }; + uint32_t num = msg->get_features.num; + uint64_t features; + + features = vdev->host_features & ~vdc->legacy_features; + + /* We only have 64 feature bits. If driver asks for more, return zeros */ + if (index < 2) { + features >>= index * 32; + f[0] = features; + f[1] = features >> 32; + } + + /* If index is out of bounds, we respond with num=0, f=0. */ + virtio_msg_pack_get_features_resp(&msg_resp, index, num, f); + virtio_msg_bus_send(&s->msg_bus, &msg_resp); +} + +static void virtio_msg_set_features(VirtIOMSGProxy *s, + VirtIOMSG *msg) +{ + VirtIOMSG msg_resp; + unsigned int i; + + for (i = 0; i < msg->set_features.num; i++) { + unsigned int feature_index = i + msg->set_features.index; + + /* We only support up to 64bits */ + if (feature_index >= 2) { + break; + } + + s->guest_features = deposit64(s->guest_features, + feature_index * 32, 32, + msg->set_features.b32[i]); + } + + virtio_msg_pack_set_features_resp(&msg_resp); + virtio_msg_bus_send(&s->msg_bus, &msg_resp); +} + +static void virtio_msg_soft_reset(VirtIOMSGProxy *s) +{ + virtio_bus_reset(&s->bus); + s->guest_features = 0; +} + +static void virtio_msg_set_device_status(VirtIOMSGProxy *s, + VirtIOMSG *msg) +{ + VirtIODevice *vdev = virtio_bus_get_device(&s->bus); + uint32_t status = msg->set_device_status.status; + VirtIOMSG msg_resp; + + if (!(status & VIRTIO_CONFIG_S_DRIVER_OK)) { + virtio_bus_stop_ioeventfd(&s->bus); + } + + if (status & VIRTIO_CONFIG_S_FEATURES_OK) { + virtio_set_features(vdev, s->guest_features); + } + + virtio_set_status(vdev, status); + assert(vdev->status == status); + + if (status & VIRTIO_CONFIG_S_DRIVER_OK) { + virtio_bus_start_ioeventfd(&s->bus); + } + + if (status == 0) { + virtio_msg_soft_reset(s); + } + + virtio_msg_pack_set_device_status_resp(&msg_resp, vdev->status); + virtio_msg_bus_send(&s->msg_bus, &msg_resp); +} + +static void virtio_msg_get_device_status(VirtIOMSGProxy *s, + VirtIOMSG *msg) +{ + VirtIODevice *vdev = virtio_bus_get_device(&s->bus); + VirtIOMSG msg_resp; + + virtio_msg_pack_get_device_status_resp(&msg_resp, vdev->status); + virtio_msg_bus_send(&s->msg_bus, &msg_resp); +} + +static void virtio_msg_get_config(VirtIOMSGProxy *s, + VirtIOMSG *msg) +{ + VirtIODevice *vdev = virtio_bus_get_device(&s->bus); + uint32_t size = msg->get_config.size; + uint32_t offset = msg->get_config.offset; + uint8_t data[VIRTIO_MSG_MAX_CONFIG_BYTES]; + VirtIOMSG msg_resp; + unsigned int i; + + if (size > VIRTIO_MSG_MAX_CONFIG_BYTES) { + return; + } + + for (i = 0; i < size; i++) { + data[i] = virtio_config_modern_readb(vdev, offset + i); + } + + virtio_msg_pack_get_config_resp(&msg_resp, size, offset, + vdev->generation, data); + virtio_msg_bus_send(&s->msg_bus, &msg_resp); +} + +static void virtio_msg_set_config(VirtIOMSGProxy *s, + VirtIOMSG *msg) +{ + VirtIODevice *vdev = virtio_bus_get_device(&s->bus); + uint32_t offset = msg->set_config.offset; + uint32_t size = msg->set_config.size; + uint8_t *data = msg->set_config.data; + VirtIOMSG msg_resp; + unsigned int i; + + for (i = 0; i < size; i++) { + virtio_config_modern_writeb(vdev, offset + i, data[i]); + } + + virtio_msg_pack_set_config_resp(&msg_resp, size, offset, + vdev->generation, data); + virtio_msg_bus_send(&s->msg_bus, &msg_resp); +} + +static void virtio_msg_get_vqueue(VirtIOMSGProxy *s, + VirtIOMSG *msg) +{ + VirtIODevice *vdev = virtio_bus_get_device(&s->bus); + uint32_t max_size = VIRTQUEUE_MAX_SIZE; + uint32_t index = msg->get_vqueue.index; + hwaddr desc, avail, used; + VirtIOMSG msg_resp; + uint32_t size; + + if (index < VIRTIO_QUEUE_MAX) { + size = virtio_queue_get_num(vdev, index); + if (!size) { + max_size = 0; + } + + virtio_queue_get_rings(vdev, index, &desc, &avail, &used); + virtio_msg_pack_get_vqueue_resp(&msg_resp, index, max_size, size, + desc, avail, used); + } else { + /* OOB index, respond with all zeroes. */ + virtio_msg_pack_get_vqueue_resp(&msg_resp, index, 0, 0, 0, 0, 0); + } + + virtio_msg_bus_send(&s->msg_bus, &msg_resp); +} + +static void virtio_msg_set_vqueue(VirtIOMSGProxy *s, VirtIOMSG *msg) +{ + VirtIODevice *vdev = virtio_bus_get_device(&s->bus); + uint32_t index = msg->set_vqueue.index; + VirtIOMSG msg_resp; + + if (index >= VIRTIO_QUEUE_MAX) { + /* OOB index, ignore. */ + return; + } + + virtio_queue_set_num(vdev, index, msg->set_vqueue.size); + virtio_queue_set_rings(vdev, index, + msg->set_vqueue.descriptor_addr, + msg->set_vqueue.driver_addr, + msg->set_vqueue.device_addr); + virtio_queue_enable(vdev, index); + + virtio_msg_pack_set_vqueue_resp(&msg_resp); + virtio_msg_bus_send(&s->msg_bus, &msg_resp); +} + +static void virtio_msg_reset_vqueue(VirtIOMSGProxy *s, VirtIOMSG *msg) +{ + VirtIODevice *vdev = virtio_bus_get_device(&s->bus); + VirtIOMSG msg_resp; + + virtio_queue_reset(vdev, msg->reset_vqueue.index); + + virtio_msg_pack_reset_vqueue_resp(&msg_resp); + virtio_msg_bus_send(&s->msg_bus, &msg_resp); +} + +static void virtio_msg_event_avail(VirtIOMSGProxy *s, + VirtIOMSG *msg) +{ + VirtIODevice *vdev = virtio_bus_get_device(&s->bus); + uint16_t vq_idx = msg->event_avail.index; + + if (!(vdev->status & VIRTIO_CONFIG_S_DRIVER_OK)) { + VirtIOMSG msg_ev; + + virtio_error(vdev, "Notification while driver not OK?"); + virtio_msg_pack_event_config(&msg_ev, vdev->status, vdev->generation, + 0, 0, NULL); + virtio_msg_bus_send(&s->msg_bus, &msg_ev); + return; + } + + if (vq_idx >= VIRTIO_QUEUE_MAX) { + virtio_error(vdev, "Notification to bad VQ!"); + return; + } + + if (!virtio_queue_get_num(vdev, vq_idx)) { + virtio_error(vdev, "Notification to unconfigured VQ!"); + return; + } + + /* NOTIFICATION_DATA doesn't exist in QEMU 8.2.7. if (0) it */ + if (virtio_vdev_has_feature(vdev, VIRTIO_F_NOTIFICATION_DATA) && 0) { + VirtQueue *vq = virtio_get_queue(vdev, vq_idx); + uint32_t next_offset_wrap = msg->event_avail.next_offset_wrap; + uint16_t qsize = virtio_queue_get_num(vdev, vq_idx); + uint32_t offset = next_offset_wrap & 0x7fffffff; + bool wrap = next_offset_wrap & 0x80000000; + uint16_t data; + + if (offset > 0x7fff || offset >= qsize) { + virtio_error(vdev, "Next offset to large!"); + /* Bail out without notification??? */ + return; + } + + data = wrap << 15; + data |= offset & 0x7fff; + + virtio_queue_set_shadow_avail_idx(vq, data); + } + virtio_queue_notify(vdev, msg->event_avail.index); +} + +typedef void (*VirtIOMSGHandler)(VirtIOMSGProxy *s, + VirtIOMSG *msg); + +static const VirtIOMSGHandler msg_handlers[] = { + [VIRTIO_MSG_DEVICE_INFO] = virtio_msg_device_info, + [VIRTIO_MSG_GET_FEATURES] = virtio_msg_get_features, + [VIRTIO_MSG_SET_FEATURES] = virtio_msg_set_features, + [VIRTIO_MSG_GET_DEVICE_STATUS] = virtio_msg_get_device_status, + [VIRTIO_MSG_SET_DEVICE_STATUS] = virtio_msg_set_device_status, + [VIRTIO_MSG_GET_CONFIG] = virtio_msg_get_config, + [VIRTIO_MSG_SET_CONFIG] = virtio_msg_set_config, + [VIRTIO_MSG_GET_VQUEUE] = virtio_msg_get_vqueue, + [VIRTIO_MSG_SET_VQUEUE] = virtio_msg_set_vqueue, + [VIRTIO_MSG_RESET_VQUEUE] = virtio_msg_reset_vqueue, + [VIRTIO_MSG_EVENT_AVAIL] = virtio_msg_event_avail, +}; + +static int virtio_msg_receive_msg(VirtIOMSGBusDevice *bd, VirtIOMSG *msg) +{ + VirtIOMSGProxy *s = VIRTIO_MSG(bd->opaque); + VirtIOMSGHandler handler; + + /* virtio_msg_print(msg); */ + if (msg->msg_id > ARRAY_SIZE(msg_handlers)) { + return VIRTIO_MSG_ERROR_UNSUPPORTED_MESSAGE_ID; + } + + handler = msg_handlers[msg->msg_id]; + assert((msg->type & VIRTIO_MSG_TYPE_RESPONSE) == 0); + + if (virtio_msg_bad(s, msg)) { + /* Drop bad messages. */ + return VIRTIO_MSG_ERROR_BAD_MESSAGE; + } + + if (handler) { + handler(s, msg); + } + + return VIRTIO_MSG_NO_ERROR; +} + +static const VirtIOMSGBusPort virtio_msg_port = { + .receive = virtio_msg_receive_msg, + .is_driver = false +}; + +static void virtio_msg_notify_queue(DeviceState *opaque, uint16_t index) +{ + VirtIOMSGProxy *s = VIRTIO_MSG(opaque); + VirtIODevice *vdev = virtio_bus_get_device(&s->bus); + VirtIOMSG msg; + + if (!vdev || !virtio_msg_bus_connected(&s->msg_bus)) { + return; + } + + virtio_msg_pack_event_used(&msg, index); + virtio_msg_bus_send(&s->msg_bus, &msg); +} + +static void virtio_msg_notify(DeviceState *opaque, uint16_t vector) +{ + VirtIOMSGProxy *s = VIRTIO_MSG(opaque); + VirtIODevice *vdev = virtio_bus_get_device(&s->bus); + VirtIOMSG msg; + + if (!virtio_msg_bus_connected(&s->msg_bus)) { + return; + } + + /* Check if we're notifying for VQ or CONFIG updates. */ + if (vdev->isr & 2) { + virtio_msg_pack_event_config(&msg, vdev->status, vdev->generation, + 0, 0, NULL); + virtio_msg_bus_send(&s->msg_bus, &msg); + } +} + +static const VMStateDescription vmstate_virtio_msg_state_sub = { + .name = "virtio_msg_device", + .version_id = 1, + .minimum_version_id = 1, + .fields = (VMStateField[]) { + VMSTATE_UINT64(guest_features, VirtIOMSGProxy), + VMSTATE_END_OF_LIST() + } +}; + +static const VMStateDescription vmstate_virtio_msg = { + .name = "virtio_msg_proxy_backend", + .version_id = 1, + .minimum_version_id = 1, + .fields = (const VMStateField[]) { + VMSTATE_END_OF_LIST() + }, + .subsections = (const VMStateDescription * []) { + &vmstate_virtio_msg_state_sub, + NULL + } +}; + +static void virtio_msg_save_extra_state(DeviceState *opaque, QEMUFile *f) +{ + VirtIOMSGProxy *s = VIRTIO_MSG(opaque); + + vmstate_save_state(f, &vmstate_virtio_msg, s, NULL, &error_fatal); +} + +static int virtio_msg_load_extra_state(DeviceState *opaque, QEMUFile *f) +{ + VirtIOMSGProxy *s = VIRTIO_MSG(opaque); + + return vmstate_load_state(f, &vmstate_virtio_msg, s, 1, &error_fatal); +} + +static bool virtio_msg_has_extra_state(DeviceState *opaque) +{ + return true; +} + +static void virtio_msg_reset_hold(Object *obj, ResetType type) +{ + VirtIOMSGProxy *s = VIRTIO_MSG(obj); + VirtIODevice *vdev = virtio_bus_get_device(&s->bus); + + virtio_msg_soft_reset(s); + + /* Only connect transports with virtio-devs. */ + if (vdev) { + bool r; + + r = virtio_msg_bus_connect(&s->msg_bus, &virtio_msg_port, s); + if (!r) { + error_report("%s: No bus connected!", + object_get_canonical_path(obj)); + exit(EXIT_FAILURE); + } + } +} + +static void virtio_msg_pre_plugged(DeviceState *d, Error **errp) +{ + VirtIOMSGProxy *s = VIRTIO_MSG(d); + VirtIODevice *vdev = virtio_bus_get_device(&s->bus); + + virtio_add_feature(&vdev->host_features, VIRTIO_F_VERSION_1); +} + +static AddressSpace *virtio_msg_get_dma_as(DeviceState *d) +{ + VirtIOMSGProxy *s = VIRTIO_MSG(d); + AddressSpace *as; + + as = virtio_msg_bus_get_remote_as(&s->msg_bus); + return as; +} + +static void virtio_msg_realize(DeviceState *d, Error **errp) +{ + VirtIOMSGProxy *s = VIRTIO_MSG(d); + + qbus_init(&s->bus, sizeof(s->bus), + TYPE_VIRTIO_MSG_PROXY_BUS, d, NULL); + qbus_init(&s->msg_bus, sizeof(s->msg_bus), + TYPE_VIRTIO_MSG_BUS, d, NULL); +} + +static void virtio_msg_class_init(ObjectClass *klass, const void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + ResettableClass *rc = RESETTABLE_CLASS(klass); + + dc->realize = virtio_msg_realize; + dc->bus_type = TYPE_BUS; + dc->user_creatable = true; + rc->phases.hold = virtio_msg_reset_hold; + + set_bit(DEVICE_CATEGORY_MISC, dc->categories); +} + +static const TypeInfo virtio_msg_types[] = { + { + .name = TYPE_VIRTIO_MSG, + .parent = TYPE_DEVICE, + .instance_size = sizeof(VirtIOMSGProxy), + .class_init = virtio_msg_class_init, + }, +}; +DEFINE_TYPES(virtio_msg_types); + +static char *virtio_msg_bus_get_dev_path(DeviceState *dev) +{ + BusState *virtio_msg_bus; + VirtIOMSGProxy *virtio_msg_proxy; + char *proxy_path; + + virtio_msg_bus = qdev_get_parent_bus(dev); + virtio_msg_proxy = VIRTIO_MSG(virtio_msg_bus->parent); + proxy_path = qdev_get_dev_path(DEVICE(virtio_msg_proxy)); + + if (!proxy_path) { + const char *path = object_get_canonical_path(OBJECT(virtio_msg_proxy)); + proxy_path = strdup(path); + } + return proxy_path; +} + +static void virtio_msg_bus_class_init(ObjectClass *klass, const void *data) +{ + BusClass *bus_class = BUS_CLASS(klass); + VirtioBusClass *k = VIRTIO_BUS_CLASS(klass); + + k->notify_queue = virtio_msg_notify_queue; + k->notify = virtio_msg_notify; + k->save_extra_state = virtio_msg_save_extra_state; + k->load_extra_state = virtio_msg_load_extra_state; + k->has_extra_state = virtio_msg_has_extra_state; + k->pre_plugged = virtio_msg_pre_plugged; + k->has_variable_vring_alignment = true; + k->get_dma_as = virtio_msg_get_dma_as; + bus_class->max_dev = 1; + bus_class->get_dev_path = virtio_msg_bus_get_dev_path; +} + +static const TypeInfo virtio_msg_bus_types[] = { + { + .name = TYPE_VIRTIO_MSG_PROXY_BUS, + .parent = TYPE_VIRTIO_BUS, + .instance_size = sizeof(VirtioBusState), + .class_init = virtio_msg_bus_class_init, + }, + { + .name = TYPE_VIRTIO_MSG_TP_BUS, + .parent = TYPE_BUS, + .instance_size = sizeof(BusState), + }, +}; + +DEFINE_TYPES(virtio_msg_bus_types); diff --git a/include/hw/virtio/virtio-msg-bus.h b/include/hw/virtio/virtio-msg-bus.h new file mode 100644 index 0000000000..12ef8030f4 --- /dev/null +++ b/include/hw/virtio/virtio-msg-bus.h @@ -0,0 +1,95 @@ +/* + * VirtIO MSG bus. + * + * Copyright (c) 2024 Advanced Micro Devices, Inc. + * Written by Edgar E. Iglesias edgar.iglesias@amd.com + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef QEMU_VIRTIO_MSG_BUS_H +#define QEMU_VIRTIO_MSG_BUS_H + +#include "qom/object.h" +#include "system/dma.h" +#include "hw/qdev-core.h" +#include "hw/virtio/virtio-msg-prot.h" + +#define TYPE_VIRTIO_MSG_BUS "virtio-msg-bus" +DECLARE_INSTANCE_CHECKER(BusState, VIRTIO_MSG_BUS, + TYPE_VIRTIO_MSG_BUS) + + +#define TYPE_VIRTIO_MSG_BUS_DEVICE "virtio-msg-bus-device" +OBJECT_DECLARE_TYPE(VirtIOMSGBusDevice, VirtIOMSGBusDeviceClass, + VIRTIO_MSG_BUS_DEVICE) + +typedef struct VirtIOMSGBusPort { + int (*receive)(VirtIOMSGBusDevice *bd, VirtIOMSG *msg); + bool is_driver; +} VirtIOMSGBusPort; + +struct VirtIOMSGBusDeviceClass { + /*< private >*/ + DeviceClass parent_class; + + DeviceRealize parent_realize; + DeviceUnrealize parent_unrealize; + + /* + * Ask the bus to receive and process all messages that + * are readily available. The bus will call the registered + * VirtIOMSGBusPort.receive() function for each message. + * + * Will return immediately if no messages are available. + */ + void (*process)(VirtIOMSGBusDevice *bd); + + /* + * Called by the transport to send a message. + */ + int (*send)(VirtIOMSGBusDevice *bd, VirtIOMSG *msg_req); +}; + +typedef struct VirtIOMSGBusDevice { + DeviceState parent; + + const VirtIOMSGBusPort *peer; + void *opaque; +} VirtIOMSGBusDevice; + +static inline VirtIOMSGBusDevice *virtio_msg_bus_get_device(BusState *qbus) +{ + BusChild *kid = QTAILQ_FIRST(&qbus->children); + DeviceState *qdev = kid ? kid->child : NULL; + + return (VirtIOMSGBusDevice *)qdev; +} + +static inline bool virtio_msg_bus_connected(BusState *bus) +{ + VirtIOMSGBusDevice *bd = virtio_msg_bus_get_device(bus); + + return bd && bd->peer != NULL; +} + +void virtio_msg_bus_process(VirtIOMSGBusDevice *bd); + +bool virtio_msg_bus_connect(BusState *bus, + const VirtIOMSGBusPort *port, + void *opaque); + +static inline void +virtio_msg_bus_receive(VirtIOMSGBusDevice *bd, VirtIOMSG *msg) +{ + virtio_msg_unpack(msg); + bd->peer->receive(bd, msg); +} + +int virtio_msg_bus_send(BusState *bus, VirtIOMSG *msg_req); + +static inline AddressSpace *virtio_msg_bus_get_remote_as(BusState *bus) +{ + return &address_space_memory; +} +#endif diff --git a/include/hw/virtio/virtio-msg-prot.h b/include/hw/virtio/virtio-msg-prot.h new file mode 100644 index 0000000000..495bb69913 --- /dev/null +++ b/include/hw/virtio/virtio-msg-prot.h @@ -0,0 +1,747 @@ +/* + * Virtio MSG - Message packing/unpacking functions. + * + * Copyright (c) 2024 Advanced Micro Devices, Inc. + * Written by Edgar E. Iglesias edgar.iglesias@amd.com. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef QEMU_VIRTIO_MSG_H +#define QEMU_VIRTIO_MSG_H + +#include <stdint.h> +#include "standard-headers/linux/virtio_config.h" + +#define VIRTIO_MSG_VENDOR_ID 0x554D4551 /* 'QEMU' */ + +enum { + VIRTIO_MSG_NO_ERROR = 0, + VIRTIO_MSG_ERROR_RETRY, + VIRTIO_MSG_ERROR_TIMEOUT, + VIRTIO_MSG_ERROR_UNSUPPORTED_MESSAGE_ID, + VIRTIO_MSG_ERROR_BAD_MESSAGE, + VIRTIO_MSG_ERROR_MEMORY, /* General memory error. */ +}; + +enum { + VIRTIO_MSG_DEVICE_INFO = 0x02, + VIRTIO_MSG_GET_FEATURES = 0x03, + VIRTIO_MSG_SET_FEATURES = 0x04, + VIRTIO_MSG_GET_CONFIG = 0x05, + VIRTIO_MSG_SET_CONFIG = 0x06, + VIRTIO_MSG_GET_DEVICE_STATUS = 0x07, + VIRTIO_MSG_SET_DEVICE_STATUS = 0x08, + VIRTIO_MSG_GET_VQUEUE = 0x09, + VIRTIO_MSG_SET_VQUEUE = 0x0a, + VIRTIO_MSG_RESET_VQUEUE = 0x0b, + VIRTIO_MSG_GET_SHM = 0x0c, /* Not yet supported */ + VIRTIO_MSG_EVENT_CONFIG = 0x40, + VIRTIO_MSG_EVENT_AVAIL = 0x41, + VIRTIO_MSG_EVENT_USED = 0x42, + + VIRTIO_MSG_MAX = VIRTIO_MSG_EVENT_USED, +}; + +#define VIRTIO_MSG_MAX_SIZE 44 + +#define VIRTIO_MSG_TYPE_RESPONSE (1 << 0) +#define VIRTIO_MSG_TYPE_BUS (1 << 1) + +typedef struct VirtIOMSG { + uint8_t type; + uint8_t msg_id; + uint16_t dev_num; + uint16_t msg_size; + + union { + uint8_t payload_u8[38]; + + struct { + uint32_t device_id; + uint32_t vendor_id; + uint32_t num_feature_bits; + uint32_t config_size; + uint32_t max_vqs; + uint16_t admin_vq_idx; + uint16_t admin_vq_count; + } QEMU_PACKED get_device_info_resp; + struct { + uint32_t index; + uint32_t num; + } QEMU_PACKED get_features; + struct { + uint32_t index; + uint32_t num; + uint32_t b32[]; + } QEMU_PACKED get_features_resp; + struct { + uint32_t index; + uint32_t num; + uint32_t b32[]; + } QEMU_PACKED set_features; + struct { + uint32_t offset; + uint32_t size; + } QEMU_PACKED get_config; + struct { + uint32_t generation; + uint32_t offset; + uint32_t size; + uint8_t data[]; + } QEMU_PACKED get_config_resp; + struct { + uint32_t generation; + uint32_t offset; + uint32_t size; + uint8_t data[]; + } QEMU_PACKED set_config; + struct { + uint32_t generation; + uint32_t offset; + uint32_t size; + uint8_t data[]; + } QEMU_PACKED set_config_resp; + struct { + uint32_t status; + } QEMU_PACKED get_device_status_resp; + struct { + uint32_t status; + } QEMU_PACKED set_device_status; + struct { + uint32_t status; + } QEMU_PACKED set_device_status_resp; + struct { + uint32_t index; + } QEMU_PACKED get_vqueue; + struct { + uint32_t index; + uint32_t max_size; + uint32_t size; + uint64_t descriptor_addr; + uint64_t driver_addr; + uint64_t device_addr; + } QEMU_PACKED get_vqueue_resp; + struct { + uint32_t index; + uint32_t unused; + uint32_t size; + uint64_t descriptor_addr; + uint64_t driver_addr; + uint64_t device_addr; + } QEMU_PACKED set_vqueue; + struct { + uint32_t index; + } QEMU_PACKED reset_vqueue; + struct { + uint32_t status; + uint32_t generation; + uint32_t offset; + uint32_t size; + uint8_t config_value[]; + } QEMU_PACKED event_config; + struct { + uint32_t index; + uint32_t next_offset_wrap; + } QEMU_PACKED event_avail; + struct { + uint32_t index; + } QEMU_PACKED event_used; + }; +} QEMU_PACKED VirtIOMSG; + +/* Maximum number of 32b feature-blocks in a single message. */ +#define VIRTIO_MSG_MAX_FEATURE_NUM \ + ((VIRTIO_MSG_MAX_SIZE - offsetof(VirtIOMSG, get_features_resp.b32)) / 4) + +/* Maximum amount of config-data in a single message, in bytes. */ +#define VIRTIO_MSG_MAX_CONFIG_BYTES \ + (VIRTIO_MSG_MAX_SIZE - offsetof(VirtIOMSG, set_config.data)) + +#define LE_TO_CPU(v) \ +{ \ + if (sizeof(v) == 2) { \ + v = le16_to_cpu(v); \ + } else if (sizeof(v) == 4) { \ + v = le32_to_cpu(v); \ + } else if (sizeof(v) == 8) { \ + v = le64_to_cpu(v); \ + } else { \ + g_assert_not_reached(); \ + } \ +} + +/** + * virtio_msg_unpack_resp: Unpacks a wire virtio message responses into + * a host version + * @msg: the virtio message to unpack + * + * See virtio_msg_unpack(). + */ +static inline void virtio_msg_unpack_resp(VirtIOMSG *msg) +{ + LE_TO_CPU(msg->dev_num); + int i; + + switch (msg->msg_id) { + case VIRTIO_MSG_DEVICE_INFO: + LE_TO_CPU(msg->get_device_info_resp.device_id); + LE_TO_CPU(msg->get_device_info_resp.vendor_id); + LE_TO_CPU(msg->get_device_info_resp.num_feature_bits); + LE_TO_CPU(msg->get_device_info_resp.config_size); + LE_TO_CPU(msg->get_device_info_resp.max_vqs); + LE_TO_CPU(msg->get_device_info_resp.admin_vq_idx); + LE_TO_CPU(msg->get_device_info_resp.admin_vq_count); + break; + case VIRTIO_MSG_GET_FEATURES: + LE_TO_CPU(msg->get_features_resp.index); + LE_TO_CPU(msg->get_features_resp.num); + for (i = 0; i < VIRTIO_MSG_MAX_FEATURE_NUM && + i < msg->get_features_resp.num; i++) { + LE_TO_CPU(msg->get_features_resp.b32[i]); + } + break; + case VIRTIO_MSG_GET_DEVICE_STATUS: + LE_TO_CPU(msg->get_device_status_resp.status); + break; + case VIRTIO_MSG_GET_CONFIG: + LE_TO_CPU(msg->get_config_resp.generation); + LE_TO_CPU(msg->get_config_resp.offset); + LE_TO_CPU(msg->get_config_resp.size); + break; + case VIRTIO_MSG_SET_CONFIG: + LE_TO_CPU(msg->set_config_resp.generation); + LE_TO_CPU(msg->set_config_resp.offset); + LE_TO_CPU(msg->set_config_resp.size); + break; + case VIRTIO_MSG_GET_VQUEUE: + LE_TO_CPU(msg->get_vqueue_resp.index); + LE_TO_CPU(msg->get_vqueue_resp.max_size); + LE_TO_CPU(msg->get_vqueue_resp.size); + LE_TO_CPU(msg->get_vqueue_resp.descriptor_addr); + LE_TO_CPU(msg->get_vqueue_resp.driver_addr); + LE_TO_CPU(msg->get_vqueue_resp.device_addr); + break; + default: + break; + } +} + +/** + * virtio_msg_unpack: Unpacks a wire virtio message into a host version + * @msg: the virtio message to unpack + * + * Virtio messages arriving on the virtio message bus have a specific + * format (little-endian, packet encoding, etc). To simplify for the + * the rest of the implementation, we have packers and unpackers that + * convert the wire messages into host versions. This includes endianess + * conversion and potentially future decoding and expansion. + * + * At the moment, we only do endian conversion, virtio_msg_unpack() should + * get completely eliminated by the compiler when targetting little-endian + * hosts. + */ +static inline void virtio_msg_unpack(VirtIOMSG *msg) +{ + int i; + + if (msg->type & VIRTIO_MSG_TYPE_RESPONSE) { + virtio_msg_unpack_resp(msg); + return; + } + + LE_TO_CPU(msg->dev_num); + + switch (msg->msg_id) { + case VIRTIO_MSG_GET_FEATURES: + LE_TO_CPU(msg->get_features.index); + LE_TO_CPU(msg->get_features.num); + break; + case VIRTIO_MSG_SET_FEATURES: + LE_TO_CPU(msg->set_features.index); + LE_TO_CPU(msg->set_features.num); + for (i = 0; i < VIRTIO_MSG_MAX_FEATURE_NUM && + i < msg->set_features.num; i++) { + LE_TO_CPU(msg->set_features.b32[i]); + } + break; + case VIRTIO_MSG_SET_DEVICE_STATUS: + LE_TO_CPU(msg->set_device_status.status); + break; + case VIRTIO_MSG_GET_CONFIG: + LE_TO_CPU(msg->get_config.offset); + LE_TO_CPU(msg->get_config.size); + break; + case VIRTIO_MSG_SET_CONFIG: + LE_TO_CPU(msg->set_config.generation); + LE_TO_CPU(msg->set_config.offset); + LE_TO_CPU(msg->set_config.size); + break; + case VIRTIO_MSG_GET_VQUEUE: + LE_TO_CPU(msg->get_vqueue.index); + break; + case VIRTIO_MSG_SET_VQUEUE: + LE_TO_CPU(msg->set_vqueue.index); + LE_TO_CPU(msg->set_vqueue.size); + LE_TO_CPU(msg->set_vqueue.descriptor_addr); + LE_TO_CPU(msg->set_vqueue.driver_addr); + LE_TO_CPU(msg->set_vqueue.device_addr); + break; + case VIRTIO_MSG_RESET_VQUEUE: + LE_TO_CPU(msg->reset_vqueue.index); + break; + case VIRTIO_MSG_EVENT_CONFIG: + LE_TO_CPU(msg->event_config.status); + LE_TO_CPU(msg->event_config.generation); + LE_TO_CPU(msg->event_config.offset); + LE_TO_CPU(msg->event_config.size); + break; + case VIRTIO_MSG_EVENT_AVAIL: + LE_TO_CPU(msg->event_avail.index); + LE_TO_CPU(msg->event_avail.next_offset_wrap); + break; + case VIRTIO_MSG_EVENT_USED: + LE_TO_CPU(msg->event_used.index); + break; + default: + break; + } +} + +static inline size_t virtio_msg_header_size(void) +{ + return offsetof(VirtIOMSG, payload_u8); +} + +static inline void virtio_msg_pack_header(VirtIOMSG *msg, + uint8_t msg_id, + uint8_t type, + uint16_t dev_num, + uint16_t payload_size) +{ + uint16_t msg_size = virtio_msg_header_size() + payload_size; + + msg->type = type; + msg->msg_id = msg_id; + msg->dev_num = cpu_to_le16(dev_num); + msg->msg_size = cpu_to_le16(msg_size); + + /* Keep things predictable. */ + memset(msg->payload_u8, 0, sizeof msg->payload_u8); +} + +static inline void virtio_msg_pack_get_device_info(VirtIOMSG *msg) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_DEVICE_INFO, 0, 0, 0); +} + +static inline void virtio_msg_pack_get_device_info_resp(VirtIOMSG *msg, + uint32_t dev_num, + uint32_t vendor_id, + uint32_t num_feature_bits, + uint32_t config_size, + uint32_t max_vqs, + uint16_t admin_vq_idx, + uint16_t admin_vq_count) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_DEVICE_INFO, + VIRTIO_MSG_TYPE_RESPONSE, 0, + sizeof msg->get_device_info_resp); + + msg->get_device_info_resp.device_id = cpu_to_le32(dev_num); + msg->get_device_info_resp.vendor_id = cpu_to_le32(vendor_id); + msg->get_device_info_resp.num_feature_bits = cpu_to_le32(num_feature_bits); + msg->get_device_info_resp.config_size = cpu_to_le32(config_size); + msg->get_device_info_resp.max_vqs = cpu_to_le32(max_vqs); + msg->get_device_info_resp.admin_vq_idx = cpu_to_le16(admin_vq_idx); + msg->get_device_info_resp.admin_vq_count = cpu_to_le16(admin_vq_count); +} + +static inline void virtio_msg_pack_get_features(VirtIOMSG *msg, + uint32_t index, + uint32_t num) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_GET_FEATURES, 0, 0, + sizeof msg->get_features); + + msg->get_features.index = cpu_to_le32(index); + msg->get_features.num = cpu_to_le32(num); +} + +static inline void virtio_msg_pack_get_features_resp(VirtIOMSG *msg, + uint32_t index, + uint32_t num, + uint32_t *f) +{ + unsigned int i; + + virtio_msg_pack_header(msg, VIRTIO_MSG_GET_FEATURES, + VIRTIO_MSG_TYPE_RESPONSE, 0, + sizeof msg->get_features_resp + num * sizeof(*f)); + + msg->get_features_resp.index = cpu_to_le32(index); + msg->get_features_resp.num = cpu_to_le32(num); + + assert(num <= VIRTIO_MSG_MAX_FEATURE_NUM); + + for (i = 0; i < num && i < VIRTIO_MSG_MAX_FEATURE_NUM; i++) { + msg->get_features_resp.b32[i] = cpu_to_le32(f[i]); + } +} + +static inline void virtio_msg_pack_set_features(VirtIOMSG *msg, + uint32_t index, + uint32_t num, + uint32_t *f) +{ + unsigned int i; + + virtio_msg_pack_header(msg, VIRTIO_MSG_SET_FEATURES, 0, 0, + sizeof msg->set_features + num * sizeof(*f)); + + msg->set_features.index = cpu_to_le32(index); + msg->set_features.num = cpu_to_le32(num); + + assert(num <= VIRTIO_MSG_MAX_FEATURE_NUM); + + for (i = 0; i < num && i < VIRTIO_MSG_MAX_FEATURE_NUM; i++) { + msg->set_features.b32[i] = cpu_to_le32(f[i]); + } +} + +static inline void virtio_msg_pack_set_features_resp(VirtIOMSG *msg) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_SET_FEATURES, + VIRTIO_MSG_TYPE_RESPONSE, 0, 0); +} + +static inline void virtio_msg_pack_set_device_status(VirtIOMSG *msg, + uint32_t status) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_SET_DEVICE_STATUS, 0, 0, + sizeof msg->set_device_status); + + msg->set_device_status.status = cpu_to_le32(status); +} + +static inline void virtio_msg_pack_set_device_status_resp(VirtIOMSG *msg, + uint32_t status) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_SET_DEVICE_STATUS, + VIRTIO_MSG_TYPE_RESPONSE, 0, + sizeof msg->set_device_status_resp); + + msg->set_device_status_resp.status = cpu_to_le32(status); +} + +static inline void virtio_msg_pack_get_device_status(VirtIOMSG *msg) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_GET_DEVICE_STATUS, 0, 0, 0); +} + +static inline void virtio_msg_pack_get_device_status_resp(VirtIOMSG *msg, + uint32_t status) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_GET_DEVICE_STATUS, + VIRTIO_MSG_TYPE_RESPONSE, 0, + sizeof msg->get_device_status_resp); + + msg->get_device_status_resp.status = cpu_to_le32(status); +} + +static inline void virtio_msg_pack_get_config(VirtIOMSG *msg, + uint32_t size, + uint32_t offset) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_GET_CONFIG, 0, 0, + sizeof msg->get_config); + + msg->get_config.offset = cpu_to_le32(offset); + msg->get_config.size = cpu_to_le32(size); +} + +static inline void virtio_msg_pack_get_config_resp(VirtIOMSG *msg, + uint32_t size, + uint32_t offset, + uint32_t generation, + uint8_t data[]) +{ + assert(size <= VIRTIO_MSG_MAX_CONFIG_BYTES); + + virtio_msg_pack_header(msg, VIRTIO_MSG_GET_CONFIG, + VIRTIO_MSG_TYPE_RESPONSE, 0, + sizeof msg->get_config_resp + size); + + msg->get_config_resp.generation = cpu_to_le32(generation); + msg->get_config_resp.offset = cpu_to_le32(offset); + msg->get_config_resp.size = cpu_to_le32(size); + + memcpy(&msg->get_config_resp.data, data, size); +} + +static inline void virtio_msg_pack_set_config(VirtIOMSG *msg, + uint32_t size, + uint32_t offset, + uint32_t generation, + uint8_t data[]) +{ + assert(size <= VIRTIO_MSG_MAX_CONFIG_BYTES); + + virtio_msg_pack_header(msg, VIRTIO_MSG_SET_CONFIG, 0, 0, + sizeof msg->set_config + size); + + msg->set_config.offset = cpu_to_le32(offset); + msg->set_config.size = cpu_to_le32(size); + msg->set_config.generation = cpu_to_le32(generation); + + memcpy(&msg->set_config.data, data, size); +} + +static inline void virtio_msg_pack_set_config_resp(VirtIOMSG *msg, + uint32_t size, + uint32_t offset, + uint32_t generation, + uint8_t data[]) +{ + assert(size <= VIRTIO_MSG_MAX_CONFIG_BYTES); + + virtio_msg_pack_header(msg, VIRTIO_MSG_SET_CONFIG, + VIRTIO_MSG_TYPE_RESPONSE, 0, + sizeof msg->set_config_resp + size); + + msg->set_config_resp.offset = cpu_to_le32(offset); + msg->set_config_resp.size = cpu_to_le32(size); + msg->set_config_resp.generation = cpu_to_le32(generation); + + memcpy(&msg->set_config_resp.data, data, size); +} + +static inline void virtio_msg_pack_get_vqueue(VirtIOMSG *msg, + uint32_t index) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_GET_VQUEUE, 0, 0, + sizeof msg->get_vqueue); + + msg->get_vqueue.index = cpu_to_le32(index); +} + +static inline void virtio_msg_pack_get_vqueue_resp(VirtIOMSG *msg, + uint32_t index, + uint32_t max_size, + uint32_t size, + uint64_t descriptor_addr, + uint64_t driver_addr, + uint64_t device_addr) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_GET_VQUEUE, + VIRTIO_MSG_TYPE_RESPONSE, 0, + sizeof msg->get_vqueue_resp); + + msg->get_vqueue_resp.index = cpu_to_le32(index); + msg->get_vqueue_resp.max_size = cpu_to_le32(max_size); + msg->get_vqueue_resp.size = cpu_to_le32(size); + msg->get_vqueue_resp.descriptor_addr = cpu_to_le64(descriptor_addr); + msg->get_vqueue_resp.driver_addr = cpu_to_le64(driver_addr); + msg->get_vqueue_resp.device_addr = cpu_to_le64(device_addr); +} + +static inline void virtio_msg_pack_reset_vqueue(VirtIOMSG *msg, uint32_t index) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_RESET_VQUEUE, 0, 0, + sizeof msg->reset_vqueue); + + msg->reset_vqueue.index = cpu_to_le32(index); +} + +static inline void virtio_msg_pack_reset_vqueue_resp(VirtIOMSG *msg) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_RESET_VQUEUE, + VIRTIO_MSG_TYPE_RESPONSE, 0, 0); +} + +static inline void virtio_msg_pack_set_vqueue(VirtIOMSG *msg, + uint32_t index, + uint32_t size, + uint64_t descriptor_addr, + uint64_t driver_addr, + uint64_t device_addr) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_SET_VQUEUE, 0, 0, + sizeof msg->set_vqueue); + + msg->set_vqueue.index = cpu_to_le32(index); + msg->set_vqueue.unused = 0; + msg->set_vqueue.size = cpu_to_le32(size); + msg->set_vqueue.descriptor_addr = cpu_to_le64(descriptor_addr); + msg->set_vqueue.driver_addr = cpu_to_le64(driver_addr); + msg->set_vqueue.device_addr = cpu_to_le64(device_addr); +} + +static inline void virtio_msg_pack_set_vqueue_resp(VirtIOMSG *msg) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_SET_VQUEUE, + VIRTIO_MSG_TYPE_RESPONSE, 0, 0); +} + +static inline void virtio_msg_pack_event_avail(VirtIOMSG *msg, + uint32_t index, + uint32_t next_offset, + bool next_wrap) +{ + uint32_t next_ow; + + virtio_msg_pack_header(msg, VIRTIO_MSG_EVENT_AVAIL, 0, 0, + sizeof msg->event_avail); + + /* next_offset is 31b wide. */ + assert((next_offset & 0x80000000U) == 0); + + /* Pack the next_offset_wrap field. */ + next_ow = next_wrap << 31; + next_ow |= next_offset; + + msg->event_avail.index = cpu_to_le32(index); + msg->event_avail.next_offset_wrap = cpu_to_le32(next_ow); +} + +static inline void virtio_msg_pack_event_used(VirtIOMSG *msg, uint32_t index) +{ + virtio_msg_pack_header(msg, VIRTIO_MSG_EVENT_USED, 0, 0, + sizeof msg->event_used); + + msg->event_used.index = cpu_to_le32(index); +} + +static inline void virtio_msg_pack_event_config(VirtIOMSG *msg, + uint32_t status, + uint32_t generation, + uint32_t offset, + uint32_t size, + uint8_t *value) +{ + unsigned int max_size; + + virtio_msg_pack_header(msg, VIRTIO_MSG_EVENT_CONFIG, 0, 0, + sizeof msg->event_config); + + msg->event_config.status = cpu_to_le32(status); + msg->event_config.generation = cpu_to_le32(generation); + msg->event_config.offset = cpu_to_le32(offset); + msg->event_config.size = cpu_to_le32(size); + + max_size = VIRTIO_MSG_MAX_SIZE; + max_size -= offsetof(VirtIOMSG, event_config.config_value); + assert(size <= max_size); + + if (size > 0 && size <= max_size) { + memcpy(&msg->event_config.config_value[0], value, size); + } +} + +/* + * Return true if msg_resp is a response for msg_req. + */ +static inline bool virtio_msg_is_resp(VirtIOMSG *msg_req, VirtIOMSG *msg_resp) +{ + if (msg_resp->dev_num == msg_req->dev_num && + msg_resp->msg_id == msg_req->msg_id && + msg_resp->type & VIRTIO_MSG_TYPE_RESPONSE) { + return true; + } + return false; +} + +static inline const char *virtio_msg_id_to_str(unsigned int type) +{ +#define VIRTIO_MSG_TYPE2STR(x) [VIRTIO_MSG_ ## x] = stringify(x) + static const char *type2str[VIRTIO_MSG_MAX + 1] = { + VIRTIO_MSG_TYPE2STR(DEVICE_INFO), + VIRTIO_MSG_TYPE2STR(GET_FEATURES), + VIRTIO_MSG_TYPE2STR(SET_FEATURES), + VIRTIO_MSG_TYPE2STR(GET_CONFIG), + VIRTIO_MSG_TYPE2STR(SET_CONFIG), + VIRTIO_MSG_TYPE2STR(GET_DEVICE_STATUS), + VIRTIO_MSG_TYPE2STR(SET_DEVICE_STATUS), + VIRTIO_MSG_TYPE2STR(GET_VQUEUE), + VIRTIO_MSG_TYPE2STR(SET_VQUEUE), + VIRTIO_MSG_TYPE2STR(RESET_VQUEUE), + VIRTIO_MSG_TYPE2STR(EVENT_CONFIG), + VIRTIO_MSG_TYPE2STR(EVENT_AVAIL), + VIRTIO_MSG_TYPE2STR(EVENT_USED), + }; + + return type2str[type]; +} + +static inline void virtio_msg_print_status(uint32_t status) +{ + printf("status %x", status); + + if (status & VIRTIO_CONFIG_S_ACKNOWLEDGE) { + printf(" ACKNOWLEDGE"); + } + if (status & VIRTIO_CONFIG_S_DRIVER) { + printf(" DRIVER"); + } + if (status & VIRTIO_CONFIG_S_DRIVER_OK) { + printf(" DRIVER_OK"); + } + if (status & VIRTIO_CONFIG_S_FEATURES_OK) { + printf(" FEATURES_OK"); + } + if (status & VIRTIO_CONFIG_S_NEEDS_RESET) { + printf(" NEEDS_RESET"); + } + if (status & VIRTIO_CONFIG_S_FAILED) { + printf(" FAILED"); + } + + printf("\n"); +} + +static inline void virtio_msg_print(VirtIOMSG *msg) +{ + bool resp = msg->type & VIRTIO_MSG_TYPE_RESPONSE; + size_t payload_size; + int i; + + assert(msg); + printf("virtio-msg: id %s 0x%x type 0x%x dev_num 0x%x msg_size 0x%x\n", + virtio_msg_id_to_str(msg->msg_id), msg->msg_id, + msg->type, msg->dev_num, msg->msg_size); + + payload_size = msg->msg_size - offsetof(VirtIOMSG, payload_u8); + if (payload_size > ARRAY_SIZE(msg->payload_u8)) { + printf("Size overflow! %zu > %zu\n", + payload_size, ARRAY_SIZE(msg->payload_u8)); + payload_size = ARRAY_SIZE(msg->payload_u8); + } + + for (i = 0; i < payload_size; i++) { + printf("%2.2x ", msg->payload_u8[i]); + if (((i + 1) % 16) == 0) { + printf("\n"); + } + } + + switch (msg->msg_id) { + case VIRTIO_MSG_GET_DEVICE_STATUS: + if (resp) { + virtio_msg_print_status(msg->get_device_status_resp.status); + } + break; + case VIRTIO_MSG_SET_DEVICE_STATUS: + virtio_msg_print_status(msg->set_device_status.status); + break; + case VIRTIO_MSG_SET_VQUEUE: + printf("set-vqueue: index=%d size=%d desc-addr=%lx " + "driver-addr=%lx device-addr=%lx\n", + msg->set_vqueue.index, msg->set_vqueue.size, + msg->set_vqueue.descriptor_addr, + msg->set_vqueue.driver_addr, + msg->set_vqueue.device_addr); + break; + } + printf("\n"); +} +#endif diff --git a/include/hw/virtio/virtio-msg.h b/include/hw/virtio/virtio-msg.h new file mode 100644 index 0000000000..99a72e93f4 --- /dev/null +++ b/include/hw/virtio/virtio-msg.h @@ -0,0 +1,45 @@ +/* + * Virtio MSG bindings + * + * Copyright (c) 2024 Advanced Micro Devices, Inc. + * Written by Edgar E. Iglesias edgar.iglesias@amd.com. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef HW_VIRTIO_MSG_PROXY_BACKEND_H +#define HW_VIRTIO_MSG_PROXY_BACKEND_H + +#include "hw/sysbus.h" +#include "hw/virtio/virtio-bus.h" + +#define TYPE_VIRTIO_MSG_PROXY_BUS "virtio-msg-proxy-bus" +/* This is reusing the VirtioBusState typedef from TYPE_VIRTIO_BUS */ +DECLARE_OBJ_CHECKERS(VirtioBusState, VirtioBusClass, + VIRTIO_MSG_PROXY_BUS, TYPE_VIRTIO_MSG_PROXY_BUS) + +#define TYPE_VIRTIO_MSG "virtio-msg" +OBJECT_DECLARE_SIMPLE_TYPE(VirtIOMSGProxy, VIRTIO_MSG) + +/* This is a BUS to hold VirtIOMSG transports */ +#define TYPE_VIRTIO_MSG_TP_BUS "virtio-msg-tp-bus" + +struct VirtIOMSGProxy { + SysBusDevice parent_obj; + + AddressSpace dma_as; + AddressSpace *bus_as; + IOMMUMemoryRegion mr_iommu; + MemoryRegion *mr_bus; + + /* virtio-bus */ + VirtioBusState bus; + /* virtio-msg-bus. */ + BusState msg_bus; + + bool iommu_enabled; + + /* Fields only used for non-legacy (v2) devices */ + uint64_t guest_features; +}; +#endif
Signed-off-by: Edgar E. Iglesias edgar.iglesias@amd.com --- hw/misc/Kconfig | 7 + hw/misc/meson.build | 1 + hw/misc/virtio-msg-amp-pci.c | 324 +++++++++++++++++++++++++++++++++++ 3 files changed, 332 insertions(+) create mode 100644 hw/misc/virtio-msg-amp-pci.c
diff --git a/hw/misc/Kconfig b/hw/misc/Kconfig index 4e35657468..53124f5c9b 100644 --- a/hw/misc/Kconfig +++ b/hw/misc/Kconfig @@ -25,6 +25,13 @@ config PCI_TESTDEV default y if TEST_DEVICES depends on PCI
+config VIRTIO_MSG_AMP_PCI + bool + default y if PCI_DEVICES + depends on PCI + select VIRTIO + select VIRTIO_MSG + config EDU bool default y if TEST_DEVICES diff --git a/hw/misc/meson.build b/hw/misc/meson.build index b1d8d8e5d2..80d4886808 100644 --- a/hw/misc/meson.build +++ b/hw/misc/meson.build @@ -8,6 +8,7 @@ system_ss.add(when: 'CONFIG_UNIMP', if_true: files('unimp.c')) system_ss.add(when: 'CONFIG_EMPTY_SLOT', if_true: files('empty_slot.c')) system_ss.add(when: 'CONFIG_LED', if_true: files('led.c')) system_ss.add(when: 'CONFIG_PVPANIC_COMMON', if_true: files('pvpanic.c')) +system_ss.add(when: 'CONFIG_VIRTIO_MSG_AMP_PCI', if_true: files('virtio-msg-amp-pci.c'))
# ARM devices system_ss.add(when: 'CONFIG_PL310', if_true: files('arm_l2x0.c')) diff --git a/hw/misc/virtio-msg-amp-pci.c b/hw/misc/virtio-msg-amp-pci.c new file mode 100644 index 0000000000..6926da848c --- /dev/null +++ b/hw/misc/virtio-msg-amp-pci.c @@ -0,0 +1,324 @@ +/* + * Model of a virtio-msg AMP capable PCI device. + * + * Copyright (C) 2025 Advanced Micro Devices, Inc. + * Written by Edgar E. Iglesias edgar.iglesias@amd.com + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "qemu/units.h" +#include "qemu/log.h" + +#include "migration/vmstate.h" +#include "hw/qdev-properties.h" +#include "hw/pci/pci_device.h" +#include "hw/pci/msix.h" +#include "hw/sysbus.h" +#include "hw/register.h" + +#include "hw/virtio/virtio-msg.h" +#include "hw/virtio/virtio-msg-bus.h" +#include "hw/virtio/spsc_queue.h" + +#define TYPE_VMSG_AMP_PCI "virtio-msg-amp-pci" +OBJECT_DECLARE_SIMPLE_TYPE(VmsgAmpPciState, VMSG_AMP_PCI) + +#define TYPE_VMSG_BUS_AMP_PCI "virtio-msg-bus-amp-pci" +OBJECT_DECLARE_SIMPLE_TYPE(VmsgBusAmpPciState, VMSG_BUS_AMP_PCI) +#define VMSG_BUS_AMP_PCI_GET_PARENT_CLASS(obj) \ + OBJECT_GET_PARENT_CLASS(obj, TYPE_VMSG_BUS_AMP_PCI) + +REG32(VERSION, 0x00) +REG32(FEATURES, 0x04) +REG32(NOTIFY, 0x20) + +#define MAX_FIFOS 1 + +typedef struct VmsgBusAmpPciState { + VirtIOMSGBusDevice parent; + PCIDevice *pcidev; + unsigned int queue_index; + + struct { + void *va; + spsc_queue driver; + spsc_queue device; + unsigned int mapcount; + } shm; +} VmsgBusAmpPciState; + +typedef struct VmsgAmpPciState { + PCIDevice dev; + MemoryRegion mr_mmio; + MemoryRegion mr_ram; + + struct fifo_bus { + VmsgBusAmpPciState dev; + VirtIOMSGProxy proxy; + BusState bus; + } fifo[MAX_FIFOS]; + + struct { + uint32_t num_fifos; + } cfg; +} VmsgAmpPciState; + +static void vmsg_bus_amp_pci_process(VirtIOMSGBusDevice *bd); + +static uint64_t vmsg_read(void *opaque, hwaddr addr, unsigned int size) +{ + uint64_t r = 0; + + assert(size == 4); + + switch (addr) { + case A_VERSION: + /* v0.1 */ + r = 0x0001; + break; + case A_FEATURES: + /* No features bit yet. */ + break; + default: + break; + } + + return r; +} + +static void vmsg_write(void *opaque, hwaddr addr, uint64_t val, + unsigned int size) +{ + VmsgAmpPciState *s = VMSG_AMP_PCI(opaque); + VmsgBusAmpPciState *dev; + unsigned int q; + + assert(size == 4); + + if (addr >= A_NOTIFY) { + q = (addr - A_NOTIFY) / 4; + dev = &s->fifo[q].dev; + + vmsg_bus_amp_pci_process(VIRTIO_MSG_BUS_DEVICE(dev)); + } else { + qemu_log_mask(LOG_GUEST_ERROR, + "%s: write to read-only reg 0x%" HWADDR_PRIx "\n", + __func__, addr); + } +} + +static const MemoryRegionOps vmsg_pci_ops = { + .read = vmsg_read, + .write = vmsg_write, + .endianness = DEVICE_LITTLE_ENDIAN, + .impl = { + .min_access_size = 4, + .max_access_size = 4, + }, +}; + +static void vmsg_create_bus(VmsgAmpPciState *s, unsigned int i) +{ + DeviceState *dev = DEVICE(s); + Object *o = OBJECT(s); + struct fifo_bus *fifo = &s->fifo[i]; + g_autofree char *name = g_strdup_printf("vmsg.%d", i); + + qbus_init(&fifo->bus, sizeof(fifo->bus), TYPE_SYSTEM_BUS, dev, name); + + /* Create the proxy. */ + object_initialize_child(o, "proxy[*]", &fifo->proxy, TYPE_VIRTIO_MSG); + qdev_realize(DEVICE(&fifo->proxy), &fifo->bus, &error_fatal); + + object_initialize_child(o, "dev[*]", &fifo->dev, + TYPE_VMSG_BUS_AMP_PCI); + qdev_realize(DEVICE(&fifo->dev), &fifo->proxy.msg_bus, &error_fatal); + + msix_vector_use(PCI_DEVICE(s), i); + + /* Caches for quick lookup. */ + fifo->dev.queue_index = i; + fifo->dev.pcidev = PCI_DEVICE(s); +} + +static void vmsg_amp_pci_realizefn(PCIDevice *dev, Error **errp) +{ + VmsgAmpPciState *s = VMSG_AMP_PCI(dev); + int i; + + if (!s->cfg.num_fifos || s->cfg.num_fifos > MAX_FIFOS) { + error_setg(errp, "Unsupported number of FIFOs (%u)", s->cfg.num_fifos); + } + + memory_region_init_io(&s->mr_mmio, OBJECT(s), &vmsg_pci_ops, s, + TYPE_VMSG_AMP_PCI, 16 * KiB); + + /* 16KB per FIFO. */ + memory_region_init_ram(&s->mr_ram, OBJECT(s), "ram", + s->cfg.num_fifos * 16 * KiB, &error_fatal); + + pci_register_bar(dev, 0, PCI_BASE_ADDRESS_SPACE_MEMORY, &s->mr_mmio); + pci_register_bar(dev, 1, PCI_BASE_ADDRESS_SPACE_MEMORY | + PCI_BASE_ADDRESS_MEM_PREFETCH, + &s->mr_ram); + + msix_init_exclusive_bar(PCI_DEVICE(s), s->cfg.num_fifos, 2, &error_fatal); + for (i = 0; i < s->cfg.num_fifos; i++) { + vmsg_create_bus(s, i); + } +} + +static const Property vmsg_properties[] = { + DEFINE_PROP_UINT32("num-fifos", VmsgAmpPciState, cfg.num_fifos, 1), +}; + +static const VMStateDescription vmstate_vmsg_pci = { + .name = TYPE_VMSG_AMP_PCI, + .version_id = 1, + .minimum_version_id = 1, + .fields = (const VMStateField[]) { + VMSTATE_PCI_DEVICE(dev, VmsgAmpPciState), + /* TODO: Add all the sub-devs. */ + VMSTATE_END_OF_LIST() + } +}; + +static void vmsg_amp_pci_class_init(ObjectClass *klass, const void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + PCIDeviceClass *pc = PCI_DEVICE_CLASS(klass); + + device_class_set_props(dc, vmsg_properties); + + pc->realize = vmsg_amp_pci_realizefn; + pc->vendor_id = PCI_VENDOR_ID_XILINX; + pc->device_id = 0x9039; + pc->revision = 1; + pc->class_id = PCI_CLASS_SYSTEM_OTHER; + dc->vmsd = &vmstate_vmsg_pci; + + set_bit(DEVICE_CATEGORY_MISC, dc->categories); +} + +static bool vmsg_bus_amp_pci_map_fifo(VmsgBusAmpPciState *s) +{ + VmsgAmpPciState *pci_s = VMSG_AMP_PCI(s->pcidev); + void *va; + + if (s->shm.mapcount) { + s->shm.mapcount++; + return true; + } + + va = memory_region_get_ram_ptr(&pci_s->mr_ram); + if (!va) { + return false; + } + + if (!s->shm.driver.shm) { + int capacity = spsc_capacity(4 * KiB); + + /* + * Layout: + * 0 - 4KB Reserved + * 4KB - 8KB Driver queue + * 8KB - 12KB Device queue + */ + spsc_init(&s->shm.driver, "driver", capacity, va + 4 * KiB); + spsc_init(&s->shm.device, "device", capacity, va + 8 * KiB); + } + + /* Map queues. */ + s->shm.va = va; + s->shm.mapcount++; + return true; +} + +static void vmsg_bus_amp_pci_unmap_fifo(VmsgBusAmpPciState *s) +{ + assert(s->shm.mapcount); + if (--s->shm.mapcount) { + return; + } + + /* TODO: Actually unmap. */ +} + +static void vmsg_bus_amp_pci_process(VirtIOMSGBusDevice *bd) +{ + VmsgBusAmpPciState *s = VMSG_BUS_AMP_PCI(bd); + spsc_queue *q; + VirtIOMSG msg; + bool r; + + if (!vmsg_bus_amp_pci_map_fifo(s)) { + return; + } + + /* + * We process the opposite queue, i.e, a driver will want to receive + * messages on the backend queue (and send messages on the driver queue). + */ + q = bd->peer->is_driver ? &s->shm.device : &s->shm.driver; + do { + r = spsc_recv(q, &msg, sizeof msg); + if (r) { + virtio_msg_bus_receive(bd, &msg); + } + } while (r); + vmsg_bus_amp_pci_unmap_fifo(s); +} + +static int vmsg_bus_amp_pci_send(VirtIOMSGBusDevice *bd, VirtIOMSG *msg_req) +{ + VmsgAmpPciState *pci_s = VMSG_AMP_PCI(OBJECT(bd)->parent); + VmsgBusAmpPciState *s = VMSG_BUS_AMP_PCI(bd); + + if (!vmsg_bus_amp_pci_map_fifo(s)) { + return VIRTIO_MSG_ERROR_MEMORY; + } + + spsc_send(&s->shm.device, msg_req, sizeof *msg_req); + + /* Notify. */ + msix_notify(PCI_DEVICE(pci_s), s->queue_index); + + vmsg_bus_amp_pci_unmap_fifo(s); + return VIRTIO_MSG_NO_ERROR; +} + +static void vmsg_bus_amp_pci_class_init(ObjectClass *klass, + const void *data) +{ + VirtIOMSGBusDeviceClass *bdc = VIRTIO_MSG_BUS_DEVICE_CLASS(klass); + + bdc->process = vmsg_bus_amp_pci_process; + bdc->send = vmsg_bus_amp_pci_send; +} + +static const TypeInfo vmsg_pci_info[] = { + { + .name = TYPE_VMSG_AMP_PCI, + .parent = TYPE_PCI_DEVICE, + .instance_size = sizeof(VmsgAmpPciState), + .class_init = vmsg_amp_pci_class_init, + .interfaces = (const InterfaceInfo[]) { + { INTERFACE_CONVENTIONAL_PCI_DEVICE }, + { } + }, + }, { + .name = TYPE_VMSG_BUS_AMP_PCI, + .parent = TYPE_VIRTIO_MSG_BUS_DEVICE, + .instance_size = sizeof(VmsgBusAmpPciState), + .class_init = vmsg_bus_amp_pci_class_init, + }, +}; + +static void vmsg_pci_register_types(void) +{ + type_register_static_array(vmsg_pci_info, ARRAY_SIZE(vmsg_pci_info)); +} + +type_init(vmsg_pci_register_types);
"Edgar E. Iglesias" edgar.iglesias@amd.com writes:
Hi all,
This the RFC I'm preparing to send to upstream QEMU for initial RFC review. A couple of limitations: I've not updated the protocol with the new msg_token field yet. We only support a single device per bus (dev_num = 0). The kernel driver only works as a module, when building it into the kernel it panics.
This adds virtio-msg, a new virtio transport. Virtio-msg works by exchanging messages over a bus and doesn't rely on trapping and emulating making it a good fit for a number of applications such as AMP, real-time and safety applications.
Together with the new transport, this series adds a PCI device that implements an AMP setup much like it would look if two SoC's would use virtio-msg across a PCI link.
The virtio-msg spec: https://github.com/Linaro/virtio-msg-spec/
Linux with virtio-msg: https://github.com/edgarigl/linux/tree/edgari/virtio-msg-6.17
To try it, first build Linux with the following as modules: CONFIG_VIRTIO_MSG=m CONFIG_VIRTIO_MSG_AMP=m CONFIG_VIRTIO_MSG_AMP_PCI=m
Boot linux in QEMU with a virtio-msg-amp-pci device, in this example with a virtio-net device attached to it (x86/q35 machine): -device virtio-msg-amp-pci -device virtio-net-device,netdev=n1,bus=/q35-pcihost/pcie.0/virtio-msg-amp-pci/vmsg.0
So this is for x86 right? Can we run the same device on an arm64 setup?
-netdev user,id=nc
Modprobe: modprobe virtio_msg_transport.ko modprobe virtio_msg_amp.ko modprobe virtio_msg_amp_pci.ko
You now should see the virtio device.
Cheers, Edgar
Edgar E. Iglesias (4): virtio: Introduce notify_queue virtio: Add virtio_queue_get_rings virtio: Add the virtio-msg transport virtio-msg-bus: amp-pci: Add generic AMP PCI device
hw/misc/Kconfig | 7 + hw/misc/meson.build | 1 + hw/misc/virtio-msg-amp-pci.c | 324 ++++++++++++ hw/virtio/Kconfig | 4 + hw/virtio/meson.build | 5 + hw/virtio/virtio-msg-bus.c | 89 ++++ hw/virtio/virtio-msg.c | 596 ++++++++++++++++++++++ hw/virtio/virtio.c | 23 + include/hw/virtio/virtio-bus.h | 1 + include/hw/virtio/virtio-msg-bus.h | 95 ++++ include/hw/virtio/virtio-msg-prot.h | 747 ++++++++++++++++++++++++++++ include/hw/virtio/virtio-msg.h | 45 ++ include/hw/virtio/virtio.h | 2 + 13 files changed, 1939 insertions(+) create mode 100644 hw/misc/virtio-msg-amp-pci.c create mode 100644 hw/virtio/virtio-msg-bus.c create mode 100644 hw/virtio/virtio-msg.c create mode 100644 include/hw/virtio/virtio-msg-bus.h create mode 100644 include/hw/virtio/virtio-msg-prot.h create mode 100644 include/hw/virtio/virtio-msg.h