Inter-Cell communication of the Jailhouse Hypervisor
====================================================

The hypervisor isolates cells but sometimes there is a need to exchange data
between cells. For that purpose Jailhouse provides shared memory and signaling
between cells.

One channel is always between exactly two cells, there is no 1:n or n:m
communication.

The interface used between the cell and the hypervisor
------------------------------------------------------

One end of such a communication channel is modeled as a PCI device that a cell
can discover on it's PCI bus. The device model used closely follows the
"ivshmem" device known from Qemu (see qemu docs/specs/ivshmem_device_spec.txt
and https://gitorious.org/nahanni/).
The device implemented by jailhouse supports MSI-X for signaling. While the
spec would allow for multiple interrupts, Jailhouse supports exactly one per
virtual device.

The ivshmem device implemented by the jailhouse hypervisor is different to the
mentioned specification in one regard. The location and the size of the shared
memory region itself are not encoded in a PCI BAR. Device drivers get the
relevant information by accessing custom PCI config space registers. See
hypervisor/pci_ivshmem.c IVSHMEM_CFG_SHMEM_* or the ivshmem demo in the
inmates directory.

Adding Inter-cell communication to cells
----------------------------------------

In order to set up a communication channel between two cells you first have to
add a memory region to both cells. Add a read/write region with matching size
and physical address to both cells. Non-root cells sharing memory with the
root cell need the memory flag "JAILHOUSE_MEM_ROOTSHARED" on the region.
To allow cells to discover shared memory and send each other MSIs you also
need to add a virtual PCI device to both cells. The "type" should be set to
"JAILHOUSE_PCI_TYPE_IVSHMEM" and "shmem_region" should be set to the index
of the memory region. "num_msix_vectors" should be set to 1 and for your root
cell config you should make sure that "iommu" is set to the correct value,
 try using the same value that works for the other pci devices.
The link between two such virtual PCI devices is established by using the same
"bdf". The size and location of the shared memory can be configured freely but
you have to make sure that the values match on both sides.
For an example have a look at the cell configuration files of qemu and the
ivshmem-demo.

Demo code
---------

You can go ahead and connect two non-root cells and run the ivshmem-demo. They
will send each other interrupts.
For the root cell you can find some test code in the following git repository:
https://github.com/henning-schild/ivshmem-guest-code
Check out the jailhouse branch and have a look at README.jailhouse.