Hands on EC3
In this section of the course, we will use the EC3 CLI following a brief example that you can try by your own to test EC3. We finish the section with hints advices on how to develop your own recipes for EC3.
Let's try EC3 with a very simple example, launching a Kubernetes cluster on top of OpenStack.
1. Create the authentication file
First create a file
with a single line like this:
id = ost; type = OpenStack; host = https://host.domain:5000; username =
<<user>>; password =
<<pass>>; tenant = <tenant>
<<pass>> with the corresponding values for the Openstack account where the cluster will be deployed. Also add the
value in your case.
This file is the
authorization file and can have more than one set of credentials. In
fact, we are going to add also a line for the Infrastructure Manager
service. Add a line at the end of
auth.dat the the file like this:
id = im; type = InfrastructureManager; username = <your_user>; password = <your_pass>
user and password required to access IM service can be created on the
fly, you can create your own ones adding the values you want. You don't
need to have any account previously. This IM service is currently
deployed at UPV's resources and it
is publicly available at https://appsgrycap.i3m.upv.es:31443/im/.
2. Customize the cluster specification
we are going to deploy a cluster in OpenStack with a limit number of
nodes = 10. The parameter to indicate the maximum size of the cluster is
ec3_max_instances and it has to be indicated in the RADL file that describes
the infrastructure to deploy. In our case, we are going to use the ubuntu-openstack recipe,
available in our Github repo. You will need to add some info to this
file regarding the name of the Openstack server and the image ID that
corresponds to an Ubuntu image in that cloud. Please, edit the
'ubuntu-openstack.radl' file to add it in lines 10 and 17.
3. Deploy the cluster
you have the authentication file ready and the ubuntu recipe customized
for your site, you can call EC3 CLI to deploy the cluster. The next
command deploys a Kubernetes cluster
based on an Ubuntu image on top of our Openstack cloud:
$ ec3 launch mycluster kubernetes ubuntu-openstack -a auth.dat -y
Creating infrastructure Infrastructure successfully created with ID: 60 ▄▟▙▄¨ Front-end state: running, IP: 18.104.22.168
This can take several minutes, that include the deployment of the
front-end VM and the configuration on it to behave as the fornt-end of
the Kubernetes cluster we requested.
4. Access the cluster
After the 'launch' command has finished, you can open a ssh session to the front-end:
$ ec3 ssh mycluster
Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-135-generic x86_64) * Documentation: https://help.ubuntu.com/ ... ubuntu@kubeserver:~$
Also you can show basic information about the deployed clusters by executing:
$ ec3 list name state IP nodes provider ------------------------------------------------------------ mycluster configured 22.214.171.124 0 OpenStack
You may use the cluster as usual, depending on the LRMS. In our example, we can simply deploy a pod or a service in K8s. You can find a very easy example that you can run in your cluster following this link: https://kubernetes.io/es/docs/tasks/run-application/run-stateless-application-deployment/.
Notice that CLUES will intercept the jobs submited to the
LRMS to deploy additional working nodes if needed. This might result in a
customizable (180 seconds by default) blocking delay when submitting
jobs when no additional working nodes are
available. This guarantees that jobs will enter execution as
soon as the working nodes are deployed and integrated in the cluster. Working
nodes will be provisioned and relinquished automatically to increase
and decrease the cluster size according to the elasticity policies
provided by CLUES.
That's it! Enjoy your virtual elastic cluster!
This is a very simple example of usage of EC3 CLI. If
you want to experiment more, at the end of the next section you can
follow the instructions to deploy a Kubernetes cluster with SAPS and
Develop your own recipe
Albeit EC3 offers a wide variety of pre-defined templates that you can use, may be you need another software component to be deployed in the cluster, and you want it to be automatically configured and deployed. For that, you can develop your own recipes. The basic structure of a RADL template is the next:
network <network_id> (<features>) system <system_id> (<features>) configure <configure_id> (<Ansible recipes>) deploy <system_id> <num> [<cloud_id>]
configure assign some features or recipes to an identity
<id>. The features are a list of constrains separated by
and, and a
constrain is formed by
<feature name> <operator> <value>. For instance:
system tomcat_node ( cpu.count = 4 and memory.size >= 1024M and net_interface.0.connection = 'net' )
This RADL defines a system with the feature
cpu.count equal to four, the feature
memory.size greater or equal than
1024M and with the feature
net_interface.0.connection bounded to
deploy keyword is a request to deploy a number
of virtual machines. Some identity of a cloud provider can be specified
to deploy on a particular cloud.
If you want to add your own customized templates to EC3, you need to consider some aspects:
imagetemplates, respect the frontend and working nodes nomenclatures. The system section for the frontend must receive the name
front, while at least one type of working node must receive the name
componenttemplates, add a
configuresection with the name of the component. You also need to add an
includestatement to import the configure in the system that you want. See Including a recipe from another for more details.
Also, it is important to provide a
description section in each new template, to be considered by the
ec3 templates command.
Congratulations on reaching the end of this section! Now, we invite you to send us some feedback on the EC3 tool and your personal experience. Please, go to the questionnaire below to send us your comments. You can also skip this questionnaire and write us a mail to firstname.lastname@example.org.