• Hands on EC3

    Hands on EC3

    EC3 keep calm and deploy your cluster

    In this section of the course, we will use the EC3 CLI following a brief example that you can try by your own to test EC3. We finish the section with hints advices on how to develop your own recipes for EC3. 

    Let's try EC3 with a very simple example, launching a Kubernetes cluster on top of OpenStack.

    1. Create the authentication file

    First create a file auth.dat  with a single line like this:

    id = ost; type = OpenStack; host = https://host.domain:5000; username = <<user>>; password = <<pass>>; tenant = <tenant>

    Replace <<user>> and <<pass>> with the corresponding values for the Openstack account where the cluster will be deployed. Also add the <<tenant>> corresponding value in your case.

    This file is the authorization file and can have more than one set of credentials. In fact, we are going to add also a line for the Infrastructure Manager service. Add a line at the end of  auth.dat the the file like this:

    id = im; type = InfrastructureManager; username = <your_user>; password = <your_pass>

    The user and password required to access IM service can be created on the fly, you can create your own ones adding the values you want. You don't need to have any account previously. This IM service is currently deployed at UPV's resources and it is publicly available at https://appsgrycap.i3m.upv.es:31443/im/.

    2. Customize the cluster specification

    Now we are going to deploy a cluster in OpenStack with a limit number of nodes = 10. The parameter to indicate the maximum size of the cluster is called ec3_max_instances and it has to be indicated in the RADL file that describes the infrastructure to deploy. In our case, we are going to use the ubuntu-openstack recipe, available in our Github repo. You will need to add some info to this file regarding the name of the Openstack server and the image ID that corresponds to an Ubuntu image in that cloud. Please, edit the 'ubuntu-openstack.radl' file to add it in lines 10 and 17.

    3. Deploy the cluster

    Once you have the authentication file ready and the ubuntu recipe customized for your site, you can call EC3 CLI to deploy the cluster. The next command deploys a Kubernetes cluster based on an Ubuntu image on top of our Openstack cloud:

    $ ec3 launch mycluster kubernetes ubuntu-openstack -a auth.dat -y

    Creating infrastructure Infrastructure successfully created with ID: 60 ▄▟▙▄¨ Front-end state: running, IP: 132.43.105.28

    This can take several minutes, that include the deployment of the front-end VM and the configuration on it to behave as the fornt-end of the Kubernetes cluster we requested.

    4. Access the cluster

    After the 'launch' command has finished, you can open a ssh session to the front-end:

    $ ec3 ssh mycluster
    Welcome to Ubuntu 18.04.5 LTS (GNU/Linux 4.15.0-135-generic x86_64) * Documentation: https://help.ubuntu.com/ ... ubuntu@kubeserver:~$

    Also you can show basic information about the deployed clusters by executing:

    $ ec3 list name state IP nodes provider ------------------------------------------------------------ mycluster configured 132.43.105.28 0 OpenStack

    You may use the cluster as usual, depending on the LRMS. In our example, we can simply deploy a pod or a service in K8s. You can find a very easy example that you can run in your cluster following this link: https://kubernetes.io/es/docs/tasks/run-application/run-stateless-application-deployment/

    Notice that CLUES will intercept the jobs submited to the LRMS to deploy additional working nodes if needed. This might result in a customizable (180 seconds by default) blocking delay when submitting jobs when no additional working nodes are available. This guarantees that jobs will enter execution as soon as the working nodes are deployed and integrated in the cluster. Working nodes will be provisioned and relinquished automatically to increase and decrease the cluster size according to the elasticity policies provided by CLUES.

    That's it! Enjoy your virtual elastic cluster!

    This is a very simple example of usage of EC3 CLI. If you want to experiment more, at the end of the next section you can follow the instructions to deploy a Kubernetes cluster with SAPS and NFS.

    Develop your own recipe

    Albeit EC3 offers a wide variety of pre-defined templates that you can use, may be you need another software component to be deployed in the cluster, and you want it to be automatically configured and deployed. For that, you can develop your own recipes. The basic structure of a RADL template is the next:

    network <network_id> (<features>) system <system_id> (<features>) configure <configure_id> (<Ansible recipes>) deploy <system_id> <num> [<cloud_id>]

    The keywords network, system and configure assign some features or recipes to an identity <id>. The features are a list of constrains separated by and, and a constrain is formed by <feature name> <operator> <value>. For instance:

    system tomcat_node ( cpu.count = 4 and memory.size >= 1024M and net_interface.0.connection = 'net' )

    This RADL defines a system with the feature cpu.count equal to four, the feature memory.size greater or equal than 1024M and with the feature net_interface.0.connection bounded to 'net'.

    The deploy keyword is a request to deploy a number of virtual machines. Some identity of a cloud provider can be specified to deploy on a particular cloud.

    If you want to add your own customized templates to EC3, you need to consider some aspects:

    • For image templates, respect the frontend and working nodes nomenclatures. The system section for the frontend must receive the name front, while at least one type of working node must receive the name wn.

    • For component templates, add a configure section with the name of the component. You also need to add an include statement to import the configure in the system that you want. See Including a recipe from another for more details.

    Also, it is important to provide a description section in each new template, to be considered by the ec3 templates command.

    Congratulations on reaching the end of this section! Now, we invite you to send us some feedback on the EC3 tool and your personal experience. Please, go to the questionnaire below to send us your comments. You can also skip this questionnaire and write us a mail to ec3@upv.es.