Juju: Deploy, Integrate and Scale Services Instantly on Multiple Clouds
It used to be hard to deploy hardware in a data centre. The Cloud changed all of this. You are literally minutes away from booting a 100 instances. This however has brought other problems. Applications, clusters, software stacks, etc. need to be able to work on many instances that are dynamically scaled up and down. All of them need to be integrated, even if IPs are changing continuously.
The first generation of solutions focused on automating server provisioning. You wrote a receipt of how a web server needed to be provisioned. You then assigned a role to each instance and it would be automatically set-up.
The problem is that server provisioning is only part of managing a complete service stack. Different services need to be able to communicate with one another. Services need to be monitored, backed-up, asset managed, “syslogged”, performance analysed, etc. Each new element in your stack would need single sign-on and to be integrated with your Nagios, Ganglia, Logstash, etc. Each Service scales differently. Some need a shared file system. Others need a high-available proxy or database. Yet others rely on P2P broadcasting which does not natively work in the Cloud.
There is now a second generation solution that promises to deploy, integrate and scale instantly on all major Public Clouds as well as Private Cloud. Instead of focusing on the servers, it focuses on the services that run on top of these servers. Its name is Juju, which means magic. Juju’s magic is called a Charm. A Charm encapsulates a service. A Charm exposes standardized interfaces towards the embedded service allowing other Charms to easily integrate. A Charm also scales the embedded service transparently.
To understand the difference between service orchestration and server provisioning let’s look at some Charms. Where service provisioning tools would be just installing WordPress on a virtual server, Juju also does integration. So when the WordPress Charm is integrated with MySQL, automatically all tables and data are created in the database. A Charm also knows about security, so when you expose WordPress to the world, automatically the right ports are opened. When scaling the MySQL Charm, the Charm knows the concept of Multi-Master Cluster and Read Slaves, so you are able to deploy a very complex MySQL cluster instantly.
The Juju basics
Using Juju is very easy. There are only a hand-full of commands that you actively use: deploy, add-relation, add-unit, expose.
# Bootstrap the Juju environment according to your preferred configuration, e.g. AWS,
# HP Cloud, Open Stack, Local, etc.
# Deploy mysql and wordpress
juju deploy mysql
juju deploy wordpress
# Create a relationship between wordpress and mysql and
# assures the wordpress schema is installed inside Mysql
juju add-relation wordpress mysql
# Open ports hence wordpress is publicly accessible
juju expose wordpress
Now let’s take it to the next level. We want to go to production with a highly scalable and high available Java application that is fully cached and monitored. We also want to mix public and unofficial/private charms to show you how this is done.
# Create 10 instances of tomcat and link it to mysql. This is an example of using an unofficial/private charm
# Also note that we only create a relation once per service, not per node.
juju deploy -n 10 cs:~robert-ayres/precise/tomcat-3 tomcat
juju add-relation tomcat mysql
# Create a high-available database with three masters in a cluster (by adding two more units to the existing one)
juju add-unit -n 2 mysql
# and add 5 read slaves called slavedb
juju deploy -n 5 mysql slavedb
# create a master-slave relation between the 3 masters and 5 slaves
juju add-relation mysql:master slavedb:slave
# give tomcat access to the read instances
juju add-relation tomcat slavedb
# now let’s add memcached to do some caching
juju deploy memcached
juju add-relation memcached tomcat
# Add HA proxy. On Amazon we could use ELB as well.
juju deploy haproxy
juju add-relation haproxy tomcat
juju expose haproxy
# Now include monitoring, so let’s get Nagios deployed and connected to all services
juju deploy nagios
juju add-relation nagios tomcat
juju add-relation nagios memcached
juju add-relation nagios haproxy
juju add-relation nagios mysql
juju add-relation nagios slavedb
juju expose nagios
# And do some deep logging with a “subordinate” Charm that gets installed on each machine
# Subordinate Charms do not run on a separate machine. NRPE allows to get statistics of
# each machine by running locally, unlike the Nagios Charm that can only test if an interface is up.
juju deploy nrpe
juju add-relation nrpe nagios
juju add-relation nrpe tomcat
juju add-relation nrpe memcached
juju add-relation nrpe haproxy
juju add-relation nrpe mysql
juju add-relation nrpe slavedb
# Want to see the nagios monitoring. Let’s log into the first Nagios machine and
# get the admin password
juju ssh nagios/0 sudo cat /var/lib/juju/nagios.passwd
# Want to know which server nagios runs on. Check all the servers and services:
# Now you can log into Nagios with http://<IP>/nagios3, user: nagiosadmin and password
# Note how all services are being deployed and monitored in Nagios.
# What if I don’t like command lines
juju deploy juju-gui
juju expose juju-gui
So what if I am already using the first generation tools?
Juju is complimentary and provides synergies. A Charm can be written in any language. You can call Puppet modules or Chef recipes from within a Charm. Hence if you have a lot of them, you can reuse them. You can even deploy Puppet or Chef as a Charm.
So what else can you do?
There are many charms. Charms for software development, e.g. Tomcat, Rails/Rack, Node.js, Lamp, Python/Django, etc. For data management, e.g. Hadoop, Hbase, Hive, Cassandra, Mysql, Postgres, Mongo, CouchDB, CouchBase, MemBase, ElasticSearch, Zookeeper, Drupal, WordPress, DokuWiki, Git, etc. For Caching/Proxying/Load-Balancing, e.g. Redis, Memcached, Varnish, Solr, Terracotta, NgInx, HAProxy, Squid, NFS, etc. For monitoring and logging, e.g. Nagios, Logstash, Ganglia, RSyslog, Munin, Kibana, NewRelic, OpenTSDB, etc. And many others, e.g. Puppet, Chef, OpenStack deployment, RabbitMQ, CloudFoundry, Jenkins, OpenERP, etc. Look for official Charms in the Charm Store. But there are also unofficial Charms, e.g. for Amazon ELB, RDS, Route53. You can even use Charms from a private local repository.
The next steps?
Juju is constantly being extended. New Charms are coming, improvements in the GUI as well as new core functionalities. A shortly to be released feature is a Charm Bundle that will allow you to bundle multiple Charms into one. So if you have a complex Hadoop environment with Hive and HBase, Ganglia, etc. You package it up into a Bundle. You can then also export and import bundles between different environments.
Be sure to check out the Charm Championship because you can earn $10.000 for creating the best Charm Bundles.
You can get more information on installing Juju and any other aspect on the Juju Homepage: juju.ubuntu.com.
Telruptive is changing focus…
The Top Blogs
Want to reproduce a Telruptive post?
- #KlausStraub CIO @BMW is still showing customer-driven future cars at @tmforumorg @uber @TeslaMotors @google going to eat their lunch 11 months ago
- See my new posts on LinkedIn telruptive.com/2015/11/24/see… 1 year ago
- "Ethereum + IoT = smart contracts on connected devices" by @telruptive @mectors @ethereum @ubuntu on @LinkedIn linkedin.com/pulse/ethereum… 1 year ago
- RT @Agent_Analytics: "The State of #BigData" with Maarten Ectors @telruptive for my #Data Blog! #datascience #InternetOfThings #IoT https:… 1 year ago
- The EU should focus on wealth creation telruptive.com/2015/07/06/the… 1 year ago
- November 2015
- July 2015
- June 2015
- May 2015
- April 2015
- March 2015
- February 2015
- January 2015
- December 2014
- November 2014
- October 2014
- September 2014
- August 2014
- July 2014
- June 2014
- May 2014
- April 2014
- March 2014
- December 2013
- November 2013
- September 2013
- July 2013
- June 2013
- May 2013
- April 2013
- March 2013
- February 2013
- January 2013
- December 2012
- November 2012
- October 2012
- August 2012
- July 2012
- June 2012
- May 2012
- April 2012
- March 2012
- February 2012
- January 2012
- December 2011
- November 2011
- October 2011
- September 2011
- August 2011
- July 2011
- June 2011
- May 2011
- April 2011
- March 2011
- February 2011
- January 2011
- December 2010
- November 2010
- October 2010
- September 2010
All the contents of the Blog, EXCEPT FOR COMMENTS AND QUOTED MATERIAL, constitute the opinion of the Author, and the Author alone; they do not represent the views and opinions of the Author’s employers, supervisors, nor do they represent the view of organizations, businesses or institutions the Author is a part of.
The Author is not responsible for the content of any comments made by the Commenter(s).
While we have made every attempt to ensure that the information contained in this Blog has been obtained from reliable sources, the Author is not responsible for any errors or omissions, or for the results obtained from the use of this information. All information in this Blog is provided "as is", with no guarantee of completeness, accuracy, timeliness or of the results obtained from the use of this information, and without warranty of any kind.