Cloudify, from the scalability experts GigaSpaces, is still its early stages. Unlike Google App Engine, Azure, Heroku, etc. this PaaS is more focused on the application life cycle and not on being a “transparent” application server and database. The main focus is automating application and services deployment, monitoring, autoscaling, etc. The closest competitor would be Scalr.
Unlike Scalr, Cloudify’s focus is on Cloud-neutrality. Cloudify is not focusing on using specific Amazon services for scalability but instead to make a neutral Cloud platform. The advantage is that every possible Cloud being it private or public can be used and scenarios like hybrid clouds with Cloud bursting from private to public cloud are possible. The deep understanding of large-scale architectures in a company like GigaSpaces is a guarantee that Cloudify will scale in the future.
Cloudify is still missing some important functionality like security, multi-tenancy, integrations with lower-level automation frameworks (e.g. Chef and Puppet), complex upgrade management [e.g. rolling upgrades, MySQL schema upgrades, A/B testing of new features, etc.], etc. However the roadmap is pointing towards most of these items.
Software architects should understand the possibilities Cloudify, Scalr, etc. bring. By having a reusable automation framework companies are able to spend more development and operations time on bringing new business features and less on reinventing the wheel.
In previous articles I wrote about automating cloud operations as a key to successfully and quickly launching new services.
You can use Chef or Puppet to automate the deployment of servers. However neither solves some of the other problems with cloud operations: autoscaling and disaster recovery.
Scalr is relatively new and still a bit rough around the edges. However it has great promises to simplify deployment, automate scaling and quickly recover from server outage. Scalr uses the out-of-the-box functionality from Amazon to quickly bootstrap an environment via AMIs. Afterwards it implements an alternative autoscaling that does not rely on Amazon’s. Disaster recovery depends on the type of server but it can automatically recover from master database failure and other elements in a typical scalable web architecture.
Scalr is not only a good sample of the 80-20 rule in which they focus on the most common scenarios. However via a plugin mechanism it is very easy to extend. I expect other public cloud providers to contribute plugins in the near future.
At the moment Scalr is still rough around the edges but definitely with the right push of some startups and public cloud providers it should quickly mature. With some custom plugins a telecom operator that is thinking about IaaS and private clouds should definitely look at it and inspire themselves…
In the past automatic deployment of software was something that a handfull of data center specialists dominated. However the cloud is changing this needs. If you need to deploy a battery of web servers, application servers, database clusters, nosql farm, etc. then you need to think of automatic software deployment from day one. Additionally you might have to deploy applications on Google App Engine, Azure, Amazon EC2/S3/etc., Rackspace, GoGrid, etc.
The three core areas to automate are:
- Cloud Provisioning
- Configuration Management and Automation
The main advantage of the Cloud is its capability to autoscale. If you get more requests during the day, you turn up more machines. If you get less during the night then you switch some off. You can even do autoscaling by the hour or less. Since you pay by the hour, your costs will match your revenues.
These installers can be handy when we are talking about a private cloud. However in case a public cloud is used, we are more likely to want to use the public APIs offered by public Cloud providers to provision machines. Unfortunately there are no standard APIs in public Cloud land yet. The best option is to use tools or APIs that can handle multiple clouds: e.g. Openstack, JClouds, Fog, Deltacloud, etc. Using one API to deploy on multiple clouds is key to avoid vendor lock-in.
The truth to be said, there is no clear winner in this space yet because most solutions have limitations and customization will be required. However expect very active development to happen in the coming months.
Configuration Management and Automation
The clear marketleader in this area is Puppet, which becomes even better when combined with mCollective. Puppet is a client/server configuration management solution that allows you to describe what you want to install and configure in a abstraction language. mCollective adds real-time notification. The whole solution is very powerful, although some learning will be required before you are up and running.
Additionally there are tools whose focus is not on the software installation but instead on the deployment of applications once the main software stack is installed. Examples are: ControlTier, Capistrano, Fabric, etc.
With multiple servers, solutions and applications spread over multiple cloud providers, you need to monitor. Monitor to see if they are available, but also monitor to see if you need to switch on extra capacity or if it is safe to switch off some capacity.
Creating an automatec stack of tools to provision, deploy and monitor is an initial investment that will pay itself back very quickly. Other systems could be added to the stack like inventory and asset tracking, software version control, build automation, etc. In general there is no easy solution that gives you everything and this is where open source communities should focus their attention: bringing it all together and simplifying so we do not need experts…