INTRODUCING OPEN CLOUD SERVICES
anyone is free to access, use, modify, and share — subject, at most, to measures that preserve privacy and openness.
an online service delivered on demand over the Internet without the user needing to use their own hardware.
Open Cloud Services are to cloud services what open source is to code – cloud services that are freely available, reusable, re-mixable and developed in the open.
Cloud services are more than just code – they consume and generate data and they are hosted on cloud infrastructure. If we extend the principles of openness to data, users gain control over their data. If we apply those principles to hosting, service operators gain independence from cloud providers and, if fully realized, services would be location-independent and decentralized.
Just as open source software builds on each other to the point that today it basically runs the Internet, our servers and most of our phones, Open Cloud Services grow stronger as they connect and depend on each other. And most will be revenue generating services – so with the right protocols and licenses we can create a sustainable and competitive business model for funding open source software.
If we build an open cloud then not only can we create an alternative to the big tech monopolies but we can provide a foundation for a more sustainable, equitable, and meaningful economy. A bold claim for sure, but “the cloud” is everywhere: as the IoT (Internet of things), behind machines learning and AI, as XaaS (“Everything as a Service”). Broadly speaking, “the cloud” is about the virtualization and automation of services and know-how – and as it develops hand-in-hand with technologies such as robotics, 3D printing, and lights out manufacturing, the cloud will encroach more and more on the “real-world” economy.
We know the risks such a future holds: concentration of wealth and power, unemployment, loss of freedom and privacy – but a cloud that rewards openness, collaboration, and individual autonomy can counteract these tendencies and realize the original promise of the Internet.
Three Levels of Openness
We can uncover what requirements a cloud service needs to be fully open by considering what these four freedoms imply for the following basic aspects of cloud services:
‣ Code: Having the code of a cloud service be open source is foundational not only because it is a basic right but also because it provides the transparency necessary to implement the other freedoms. We can think of these as freedoms for developers since they are the ones that exercise them.
‣ Data: All cloud services need data to function and for nearly all the data either is generated as users use the service or is contributed by the users of the service. In this way data freedoms are freedoms for the users.
‣ Hosting: Hosting refers to where and how a cloud service is run. A cloud service that is locked into a proprietary cloud provider and is dependent on proprietary services can not be considered fully free or open. These are freedoms for the operators of a cloud service because it gives them the freedom to choose how and where a service is run.
Reviewing these requirements shows how challenging it can be for an open cloud service to be fully open. So let’s define levels of as a simple way to indicate how open a cloud service is. At the very least an open cloud service needs to be open source, so we can set that as the minimum level. And open hosting isn’t possible without open source and open data, so we can set that the final level. Putting that together we get:
| Level | Code | Data | Hosting |
|---|---|---|---|
| 1 | ✔️ | ||
| 2 | ✔️ | ✔️ | |
| 3 | ✔️ | ✔️ | ✔️ |
Let’s consider some existing websites and cloud services and see where they fit in:
Level 1: Wordpress (but not wordpress.com, which is a proprietary service).
Level 1: Wordpress (but not wordpress.com, which is a proprietary service).
Level 2: Wikipedia releases both its source code and its data under open licenses so it is a Level 2 cloud service. But not Level 3 since it does not support open hosting (nor would that necessarily be a good thing). [OpenStreetMaps]
Level 3: Mastodon micro-blogging; Matrix chat; git is a stand-alone service (but not gitlab.com, which is a proprietary service despite depending on git and the open source gitlab software).
See the Open Cloud Service Definition for a more in-depth discussion of the definition and requirements.
Building a Free and Open Cloud
The Internet as we know it exists today because network providers have agreed to freely exchange the data packets between them and these peering arrangements form the backbone of the Internet. We can envision Open Cloud Services having something like these peering arrangements between them, forming a network from which an open cloud emerges.
The ability of cloud infrastructure and SaaS services to easily interconnect through APIs built on standard protocols is already one of the key reasons cloud technology has come to dominate but with open cloud services these abilities are turbocharged. Open Cloud Services can integrate much more deeply and with greater agility. Its open source code provides transparency; open data means their data can be shared and integrated; open hosting enables them to be remixed and relocated.
The guarantees provided by open cloud services already enable an implicit sharing of code and data but the adoption of standard peering arrangements or protocols between open cloud services that automated establishing connections and data sharing will accelerate the emergence of an open cloud. As these networks of services form, with the connections and operational data themselves open, these aggregations will function as open cloud services themselves, enabling higher-level (and likely more end-user focused) services to build on top of them.
For example, a service that aggregates and processes data might rely on variety of other services for database servers, data processing and machine learning. Another service wants to leverage both the data and the data processing infrastructure for some other purpose. With proprietary services, the business that developed this infrastructure has to make the business decision that they want to make this available then invest in building a commercial offering. Anyone using that platform is now dependent on the whims of that company. A network of Open Cloud services eliminate these frictions and business risks. Any participating service is free to discover, connect to, adapt and build on any other service.
This example also illustrates how the more services that are available and interconnected, the more data that is pooled together, the more the open cloud can achieve the same economics of scale and network effects (the more people use it, the more useful it becomes) that has driven the consolidation of Internet around a few big players.
Challenges and Rewards
For open cloud services to reach their full potential we have to develop new technologies, standards and protocols. Achieving each of these levels has big technical challenges but even bigger rewards.
Level 1: Open Code
Challenges
At minimum, Open Cloud Services will need:
- For transparency, we’ll need standard ways to discover and present a service’s source code and data.
- Collaboration: We’ll need techniques and processes so that open source development practices to all aspects of operating and managing a cloud service.
Rewards
Open Cloud Services expand the reach and accessibility of open source software and make it available to more people and in more areas of their life and work. It will empowers users with more control and provide a platform where the technologies and practices of open collaboration can reach new domains and markets.
The ability to view the source of any web page had a tremendous impact on creating an open web – what if that was extended so you could also see the server-side code that generated a page view or http request? Open Cloud Services make that possible.
Level 2: Open Data
Challenges
Having a consistent set of licenses and policies for Open Cloud Services is important for interoperability, sharing, and trust. The latter is worth highlighting: the trust that comes from its rights and guarantees along with the transparency of open source code is a crucial advantage of the open cloud.
- Open Data: There are many existing open data licenses which need to be evaluated against different scenarios and uses cases. Questions such as “What happens when data is transformed or aggregated by a service?” will need to be answered.
- End-user licenses with privacy guarantees.
-
Licenses for “peering” and cloud funding
- DataCommons To fully leverage the possibilities of open data, we’ll need protocols that provides ubiquitous access while maintaining user privacy and control.
Rewards
One of the big, if not biggest, force driving the consolidation of the Internet around a few big players is the winner-takes-all network effects that comes from having all data of a certain type available in one place, whether its user profiles and their social graphs (e.g. Facebook), search indexes of all the world’s web pages (e.g. Google), short-term apartment rentals (AirBnb) and all sorts of “big data”.
If we can figure out how users and services can freely and safely share their data we can build an Internet-scale data commons bigger than anything one company can hope to control. The rise of OpenStreetMaps as a credible challenge to Google’s decade long dominance of online maps is an illustration of that possibility.
Level 3: Open Hosting
Challenges
To enable services to be freely cloned and relocated requires tools that support reproducible packaging and deployment. To support fully open hosting we’ll need decentralization technology that enables sharing and remixing location independent services.
We’ll also need protocols for establishing and maintaining the peering arrangements described above. If open cloud services are open to each other there will need to be a way to pass on the costs incurred through usage, so these protocols will need address that.
Rewards
Giving anyone the freedom to clone a service and run it on their own infrastructure provides a powerful check against the misuse of a service by its operators. More broadly, decentralized cloud services are more resistant to censorship and grant users control and privacy.
Funding
Challenges
Regardless of its level, an open cloud services will need funding to thrive. Nearly all business models, even non-profit ones, tends to incentive proprietary advantages of one form or another. To balance that we will need a cloud funding mechanism that rewards openness so there are incentives to building an open cloud service and keeping it open as it grows in size and value.
Rewards
As the cloud extends to all reaches of the material world, the impact of creating mechanisms returns on value creation freely flow throughout would be profound:
-
Making open cloud and data commons economically viable will open up tremendous possibilities by dramatically reducing the cost of building and running websites and online services that utilize this open cloud.
-
At the heart of the cloud funding is a mechanism giving every entity in the value chain a say in how revenue is directed. This realignment of economic incentives will foster a healthier Internet by having economic decisions be made by diversity of stakeholders instead of by a few gatekeepers whose financial returns derive from commodifying and maximizing attention.
-
As cloud funding technology matures it can be applied to other intellectual and creative endeavors such as journalism or music to provide a means to make them freely available and fairly compensated.
From Idea to Reality
These technical challenges are big but not insurmountable, the main challenge is the classic chicken or egg problem: how can we drive development and adoption before there’s a critical mass of open cloud services available and cloud funding revenue flowing through this open cloud?
So it is important to build technology that is immediately useful each step of the way. Here is the step-by-step strategy we are envisioning for making this a reality:
What you are reading right now…
“Ensembles” are git repositories that package open cloud services and Unfurl is our tool for managing and deploying them. Ensembles are designed to be the building blocks of an open and decentralized cloud infrastructure: reproducible, relocatable and shareable.
Ensembles provide:
- Immutable history stored in git – enabling an open source development workflow for running live services, including “day two” management.
- Visibility and transparency by providing a consistent interface for discovery.
- Decentralization via a notion of a persistent identity that is defined not by a network location controlled by some entity but rather by a reproducible state.
We’re building a platform for developing and deploying ensembles. This will enable us to prototype the functionality needed for an open cloud, such as collaboration, location-independent reproducible deployments, and cloudfunding.
Pursue an architecture of easily integrable libraries and micro-services so that different components of the platform can be immediately useful for use-cases such as:
-
Tools for individual open source developers and consultants: to securely deploy and manage infrastructure for clients and allow for more flexible financial arrangements, e.g. through usage-based accounting.
-
Tools for open source businesses: Larger and more established that already. Clone and forking example deployments, server-side “view source”, easy to deploy testing and development environments.
-
Projects with shared cloud infrastructure by members can use it to collaborate on development and management and to share usage costs.
-
Even without funding, larger open source projects could use cloud-funding technology to gamify participation, encouraging participation through recognition, contests.
-
Larger projects with funding could use it to compensate and incentive contributors and invest in projects.
Get started
Intrigued? Here’s some things you can do to make the Open Cloud a reality:
-
Give us Feedback Join the community designing this.
-
Build ensembles There are already thousands of Open Cloud Services out there, especially at Level 1. Let’s package them up and make them discoverable.
-
Join Us! We’re an early stage startup in San Francisco dedicated to making this a reality. Work with us on project basis (all open source of course!) or join our founding team.