About MicroCloud

The MicroCloud snap drives three other snaps (LXD, MicroCeph, and MicroOVN), enabling automated deployment of a highly available LXD cluster for compute with Ceph as the storage driver and OVN as the managed network.

During initialisation, MicroCloud detects the other servers and then prompts you to add disks to Ceph and configure the networking setup.

At the end of this, you’ll have an OVN cluster, a Ceph cluster, and a LXD cluster. LXD itself will have been configured with both networking and storage suitable for use in a cluster.

LXD cluster

MicroCloud sets up a LXD cluster. You can use the microcloud cluster command to show information about the cluster members, or to remove specific members.

Apart from that, you can use LXD commands to manage the cluster. See Clustering in the LXD documentation for more information.

Networking

By default, MicroCloud uses MicroOVN for networking, which is a minimal wrapper around OVN (Open Virtual Network).

For external connectivity, MicroOVN requires an uplink network. This uplink network must support broadcast and multicast traffic (so that IP adverts and packets can flow between the OVN virtual router and the uplink network). Proper Ethernet networks generally fulfil these requirements, but virtual cloud environments often don’t.

Each machine in the MicroCloud cluster should have at least two available network interfaces (which can be connected to the same network or to different networks):

Network interface for intra-cluster traffic

MicroCloud requires one network interface that is pre-configured with an IP address that is within the same subnet as the IPs of the other cluster members. The network that it is connected to must support multicast.

This network interface can be, for example, a dedicated physical network interface, a VLAN, or a virtual function on an SR-IOV-capable network interface. It serves as the dedicated network interface for MicroOVN and is used for multicast discovery (during setup) and all internal traffic between the MicroCloud, OVN, and Ceph members.

If you have a network interface that is configured as a Linux bridge, you can use it for both network interfaces.

See OVN network in the LXD documentation for more information.

If you decide to not use MicroOVN, MicroCloud falls back on the Ubuntu fan for basic networking. MicroCloud will still be usable, but you will see some limitations:

  • When you move an instance from one cluster member to another, its IP address changes.

  • Egress traffic leaves from the local cluster member (while OVN provides shared egress). As a result of this, network forwarding works at a basic level only, and external addresses must be forwarded to a specific cluster member and don’t fail over.

  • There is no support for hardware acceleration, load balancers, or ACL functionality within the local network.

Storage

You have two options for storage in MicroCloud: local storage or distributed storage.

Local storage is faster, but less flexible and not fail-safe. To use local storage, each machine in the cluster requires a local disk. Disk sizes can vary.

For distributed storage, MicroCloud uses MicroCeph, which is a lightweight way of deploying a Ceph cluster. To use distributed storage, you must have at least three disks (attached to at least three different machines).

Troubleshooting

MicroCloud does not manage the services that it deploys. After the deployment process, the individual services are operating independently. If anything goes wrong, each service is responsible for handling recovery.

So, for example, if lxc cluster list shows that a LXD cluster member is offline, follow the usual steps for recovering an offline cluster member (in the simplest case, restart the LXD snap on the machine). The same applies to MicroOVN and MicroCeph.