The Future of Cloud

Author: Piyush Srivastava

Cloud is a great enabler. The concept of a software defined infrastructure has been extremely instrumental in bringing new innovative products to the market. Companies small and large are realizing the benefits of not being hard wired to their hardware and procuring as much as they need, when they need it, on demand. Markets are volatile and unpredictable. Nobody is comfortable making decisions about an uncertain future. The best model that works in those cases is the one that is flexible. Cloud has given the elasticity that enables organizations to take calculated risks, experiment, innovate, and respond to market changes quickly without worrying too much on the ROI on their hardware investments.

Cloud computing and the S-Curve

Cloud computing is at a very interesting point in history. We are seeing some unique trends. Who knew managing data centers would be so interesting? Technologies powering the cloud are seeing rapid innovations driven by several factors: efficient utilization of data center space, security, decentralization and ease of taking an application from development to production. There is a lot happening in this space but there is still much to be done.

Ray Kurzweil in his book The Singularity is Near explains the technology evolution theory as follows:

Each technology develops in three stages:

1. Slow growth (the early phase of exponential growth).
2. Rapid growth (the late, explosive phase of exponential growth).
3. A levelling off phase as a particular technology paradigm matures.

The progression of these three stages looks like the letter S. The S curve illustration above shows how an ongoing exponential trend can be composed of a cascade of S-curves. Each successive S-curve is faster owing to the learnings that feed forward from the previous S-curves.

Cloud computing as a whole can be seen as a composition of several aspects: compute, networking and storage. There is a lot of momentum in network and storage e.g. Network Function Virtualization (NFV) and Software Defined Networking, respectively. For now, I would like to focus on compute.

Virtualization

The concept of virtualization is not new. It dates back all the way to the 1960s when IBM came up with a CP-40 time sharing system. The concept of cloud computing though, is relatively recent and has been continually pushing the limits of virtualization technologies. Hypervisor powered virtual machine virtualization sits on the levelling off (top right) part of the S-curve. There is a lot of maturity in this space and it is driven by a number of large public clouds e.g. Amazon and Rackspace. It is battle tested and mature, but it has some limitations. Booting up a guest as a complete virtual instance of an operating system means the whole kernel gets loaded with every boot. This has some disadvantages: slow boot times, wasted data center space and a large surface area for security attacks. The full Linux kernel is 16 million lines of code and has many vulnerabilities.

Containers

Containers are relatively new. Linux Container (LXC) first came out in 2008. LXC provides OS-level virtualization, meaning the kernel allows for isolated execution environments in the user space. After a slow start, containers are fast becoming mainstream and seeing rapid innovations. They are in the rapid exponential growth phase of the S-curve. Support for OS-level virtualization means that you don’t need to boot up a whole kernel for every virtual instance that you need. For an organization heavily operating in the cloud, this is a big win. It allows them to save a lot on space and still keep their architectures very simple. Single purpose VMs, though desirable, result in a lot of wastage. With containers, those problems go away. Google has been using containers in their data centers for a while, as full blown server virtualization is just too much for the scale that they operate. Docker has been grabbing a lot of attention with its technology, which makes containers really easy to use.

Microkernels

…

Full article: thoughtworks.com/p2magazine/issue11/the-future-of-cloud