vRO: integration with Yandex.Cloud

vRealize Automation and vRealize Orchestrator integrate seamlessly with a variety of public clouds to create a single enterprise cloud platform. Yandex.Cloud is a new player in this market, and it does not yet have ready-made plugins / packages for integration with many third-party cloud solutions. Therefore, we had to independently develop a package to integrate it with vRO.

The goal of developing the first version of the package is to prepare a minimal set of tools for creating and managing virtual machines in Yandex.Cloud. We did not set ourselves the task of implementing a complete set of processes that meet all the capabilities of the cloud.

How do you usually start working? From the study of the documentation. And here the main problem awaits you: the documentation is scarce and it is very crude. Many elementary issues are resolved only through contacting support, which works quite quickly.

Integration pack vRO – Yandex.Cloud

v1.0.3

API Authentication

To work, you need an IAM token , it is used in request headers when accessing Yandex.Cloud via the API:

 Authorization : Bearer < IAM - TOKEN >  

To receive an IAM token, create a service account and an authorized key for it (do not forget to save the public and private keys). Then there are two ways: creating a token through the CLI or creating a JWT token to exchange it for an IAM token. The choice is yours, but in our processes we use a JWT token and create it through a python script (the documentation contains examples for all major programming languages). This JWT token is transferred to https://iam.api.cloud.yandex.net/iam/v1/tokens, the server will return the IAM token itself and its validity period in response:

 {
  "iamToken": "CggVAgAAAB...........AU4AVABWgA",
  "expiresAt": "2019-10-17T15:00:03.587843Z"
}

The lifetime of an IAM token is 12 hours, so it is better to update it on a schedule, or check its “expiration date” before sending API requests and receive a new token if necessary.

The process of creating a JWT token is called “Get JWT token”, and the process of obtaining a new IAM token is called “Get new IAM token”. The result of the request (token and its validity period) is saved in the “yandexCloud” configuration element (Library -> Yandex).

Main processes

The package contains only processes for the orchestrator. You can draw XaaS blueprints for vRealize Automation yourself, or write and send.

• “New instance” is the process of creating a new virtual machine. The folderId and serviceAccountId values ​​are taken from the configuration item, everything else is the input parameters.
  • The content attribute contains an example of a JSON request for creating a standard VM, and in the WF “Create request” element the values ​​of its fields are adjusted according to the user’s choice.
  • To prevent the user from specifying a specific image when creating a VM, not a list of standard images is loaded into the image field , but a list of image families, and the latest image is selected for it in the “Get image ID” element.
  • When an instance is created, it is assigned two labels: the name of the owner and the name of the business group. Note that the label name must match the regular expression [az] [-_ 0-9a-z], and the value [-_0-9a-z].
• “Day2 instance ops” is the process of managing deployed virtual machines. The first version of the process allows you to use four basic operations: stop, start, restart, delete.
  • At the entrance, the process expects a string with an instance description in the format: “vm_name (vm_id, vm_status)” and the required action. The list of instances in the required format returns the action getInstancesByGroup , if you pass in the name of a business group, then the user will see only the instances of this business group.

Supporting processes and activities

Processes for obtaining information about available resources and networks:

• “Get clouds” – getting a list of clouds, if the response contains only one cloud, then its ID is saved in the yandexCloud configuration item;
• “Get folders” – getting a list of folders in the cloud by its ID. At the time of this writing, this request did not work and even the recommendations of the support service did not help in any way, so the folderId value must be set manually in the yandexCloud configuration element or when calling the “Initialize” process ( see below ). All attempts to call this process return an error: “You are not authorized for this operation.”
• “Get networks” – getting a list of networks;
• “Get subnets” – getting a list of subnets. If you do not specify a value in subnetName , a complete list of subnets is returned.
  • Filtering subnets by name is done by adding a filter to the request in the following format: filter = name = ‘subnet_name’.

Processes used to create and manage instances:

• “Delete instance” – deleting an instance by its ID;
• Get disk types – getting a list of available disk types. Currently, two types of disks are used: network-ssd and network-hdd ;
• “Get images” – getting a list of disk images in the directory. If you pass the value “standard-images” as the folderId, you will receive an array of all standard Yandex.Cloud images in the response. An example of a description of one image:

{
     "productIds": [
         "f2edt1b5di3n9fs5ugq2"
     ],
     "os": {
         "type": "LINUX"
     },
     "id": "fd87i96n5740k9d2g3ac",
     "folderId": "standard-images",
     "createdAt": "2019-02-07T10:58:54Z",
     "name": "centos-7-1549536124",
     "description": "Free Linux distribution based on Red Hat Enterprise Linux. Official website and documentation: https://www.centos.org",
     "family": "centos-7",
     "storageSize": "1052770304",
     "minDiskSize": "8594128896",
     "status": "READY"
}

• “Get instances” – getting a list of instances. It is possible to filter this list by the labels set in instances ( see the description of “New instance” ), the value of the rbru_subtenant label is checked ;

Preparing the package for work

After installing this package, you need to prepare the “Get JWT token” process for work:

• create a service account and an authorized key;
• place the get-yandex-jwt-token.py script and the private part of the authorized key on a server with SSH access (the script requires the python PyJWT and cryptography modules);
• specify in this script the service account ID ( service_account_id ), the used key ID ( key_id ) and the path to the private key;
• register the path to the script on the SSH server in the yandexCloud configuration element in the getJwtTokenScriptPath attribute;
• in the “Get JWT token” process, replace the “Run SSH command (vrassh)” call with your own process for executing SSH commands and configure its parameters;
• check the work of the process, it should return the key.

Next, you need to register all REST hosts and operations on them with vRealize Orchestrator. Yandex chose an option, which is not entirely convenient for us, using dedicated domains for various groups of API requests: one is needed to obtain a token, the other to work with networks, etc. For the processes implemented in the package, four domains are used. To make it a little easier for you to get started, there is a special process “Initialize” in the Configuration folder, which will do for you:

• Saving the ID of the working directory and ID of the service account in the yandexCloud configuration item ;
• Registration of REST hosts (4 pcs) and saving links to them in the configuration element: “Add Yandex REST hosts”;
• Adding all used REST operations: “Add Yandex REST operations”;
• Obtaining a new IAM token: “Get new IAM token”;
• Creating a schedule for updating the IAM token.

Files for the article

vRO-Yandex.Cloud-Integration.zip – integration package.

Translated by Google Translate