True Cloud

A concept for a real cloud system, completely open and owned by everyone and nobody in the true spirit of the early internet.


When people talk of "the cloud" what they are really talking about is storage and computing resources rented from one of a small number of vary large companies who allocate them dynamically from within their large, power-hungry and not terribly popular data centres.

There are two groups of customers for this service - corporate customers who pay large fees and get contracts with teeth and private customers who don't. Both have to put a great deal of trust in the operators of their chosen cloud.

Nearly all of the private customers have computers that run well over ninety percent idle, have substantial free disc space and are well connected to the internet (domestic bandwidth keeps going up). These resources are widely distributed both geographically and by network topography.

Why not use the internet in a way that matches the original goal of bomb proof (literally) reliablity. Why not also make it open and free - or at least paid in kind. The idaa goes like this


There are several parts to this.

Resource Management Service

This needs writing and maintaining as open source with open protocols. It shouldn't be too big a project once the core is designed, documented and built - THAT ORDER!

The RMS is a distributed service running in containers (such as Linux docker images or FreeBSD iocage jails) on each node. It is responsible for connecting the node to the cloud, sending resource requests, responding to incoming resource requests, managing local resources, participating in VPN management and keeping track of all of this so as to enforce cloud policies around fair use, resource sharing, VPN size and constitution.

The RMS will first connect to between one and four management VPNs using the disovery service that runs in the cloud. There will be many VPNs available interconnected to provide multiple fallback routes. Requests for resources will be broadast on each connected VPN.

All traffic across the VPNs is end-to-end encrypted except for broadcast resource requests.

The RMS will calculate the user's cloud storage entitlement based on their contribution (probably between one third and a half allowing for system overheads and outages) and set up a Cloud File System (CFS) container providing the entitlement as a parameter.

The RMS will calculate the user's processing entitlement based on their contribution. By default application containers run locally, if entitlement and confguration permit they may overflow to using remote resources to run the container. If the container requires access to some of the cloud data then a suitable export and dedicated VPN must be prepared in the CFS. Remote containers can also be explicitly requested by the user, subject to entitlement.

Common cloud services such as the CDN, the VPN discovery mechanism and the service library will be provided by containers running in resources provided by users. They will scale with the cloud based on a simple policy of requiring a suitable density in each VPN and implemented in the RMS.

RMS's need a mechanism to validate each other.

Management VPN Discovery Service

This has to be accessible from outside the cloud on the public internet. It must have a well known DNS entry and be secured by SSL.

From inside the cloud this service has to be backed by data stored on high reliability storage - mirrored volumes would seem appropriate with a lot of copies (the volume is low, the update rate is low, availability is important).

All that is stored and made available is a list of VPNs soliciting for content. For each VPN this is from inception until it is full.

Each VPN record consists of UUID, list of public IPs, guest port, guest key.

This data will be presented by a server running in a container somewhere in the cloud providing seed data to the cloud's CMS. The server will supply randomised selections of VPN records on demand as an RSS feed.

To connect to a VPN a node must first connect to the related guest VPN, establish its validity with the RMS there and provide the UUIDs for its geographical location and network endpoint. The gaurdian RMS will then evaluate whether this new node is a fit for the VPN - if so the connection details (including a freshly minted key) will be provided to the node's RMS. Only one guest VPN connection is permitted at a time.

Volume Provider

ANother containerised service that runs on each node. The VP manages the storage provided by the user. This storage gets used in a number of ways:

When the VP provides storage for non-local use that storage is provided over a dedicated VPN (nested in the management VPN) connected only to the client and using the iSCSI protocol.

The request protocol for a remote volume goes as follows:

Cloud File System

This containerised service runs in a container that has only local connectivity initially.

On startup the CFS requestes the RMS to obtain eight volumes from eight well distributed VPs, each one fifth the size of the user's entitlement. Once these have been obtained it connects to the VPNs and stitches the volumes together into a ZFS RAIDZ3 pool.

The CFS exposes a UI to the host only providing:

Component Library

This will contain containerised applications preferably in both docker and iocage formats. It may also contain some VM images, such as a VM set up to run the entire system ready to drop into any handy hypervisor. The library will of course be hosted in this cloud, and served using the CDN built into the management service.

The component library will have to make installer images and the VM image for container runners available over the public internet. These packages must be signed (preferably countersigned too) by properly airgapped signing systems. Everything else can be distributed only across cloud management VPNs.

The installer image should only be capable of connecting to a VPN as a guest and pulling a copy of the RMS to bootstrap the setup process. That way everything that is used to run the cloud comes from the cloud over a cloud VPN.

The RMS pulls the remainder of the component library - all of the other core components and user tools over the cloud management VPNs.

The library itself is exported from cloud storage belonging to the development team.


Running in containers on all nodes and using contributed storage, this is a pretty typical CDN (probably built with off the shelf components) with one twist.

It must be possible to seed material for publishing to the public internet and to seed material that is only accessible over cloud management VPNs.


To participate in this cloud a user has to provide a machine with stable connectivity and a high uptime capable of running containers. One very natural implementation would be as a component that could be added to a NAS such as TrueNAS core, a router such as OpenWRT or on an SBC such as a Raspberry-Pi.

© 2023 Steve O'Hara-Smith