Skip to main content

Create an Elastigroup for Azure

This topic shows you how to create an Elastigroup in Azure from scratch.

To get started:

  1. First connect your Azure account to Spot.
  2. In the Spot Console, select Elastigroup > Groups.
  3. Click Create Elastigroup.
Click to view image...
screen

These are the steps to create an Elastigroup Group:

  • Basics
  • Compute
  • Networking
  • Review

Step 1: Basics​

Basic Settings​

These are the Basic settings:

  • Elastigroup Name. Use a naming convention based on the specific workload the Elastigroup will manage, for example dev-eu1-worker.
  • Description (optional): This should be the purpose of this Elastigroup.
  • VM prefix name (optional): The VM prefix name cannot contain spaces, control characters, or these characters: ~ ! @ # $ % ^ & * ( ) = + _ [ ] | ; : . ' ” , < > / ?.
  • Resource group.
  • Region.
  • Availability zones (optional). Specify availability zones that are eligible for your group. It is highly recommended to select multiple availability zones to further diversify the spot markets available to the group.

Image​

Select an image type (marketplace, custom, or shared image), and an image for launching your VMs.

  • Marketplace: Create the group from an image that is available in the Azure Marketplace.

    These are the marketplace image parameters:

    • Publisher: The organization that created the image to use for launching the VMs.
    • Offer: The name of a group of related images created by a publisher. Examples: UbuntuServer, WindowsServer.
    • SKU: An instance of an offer, such as a major release of a distribution. Examples: 18.04-LTS, 2019-Datacenter.

    :::note:

    If the marketplace image does not appear in the Publisher list, you can enter the image manually (click on the link provided).

    :::

  • Custom: Create the group from one of your custom VM Images.

    These are the custom image parameters:

    • Image Resource Group: A list of Resource Groups associated with your subscription.
    • Image Name. This is the label that is assigned to the image.

  • Shared Image: Create the group with an image from your organization’s Shared Image Gallery.

    These are the shared image parameters:

    • Image Resource Group. A list of Resource Groups associated with your subscription.
    • Gallery Name. The list of shared gallery names associated with the selected Resource Group.
    • Image Name. List of shared images associated with the selected Gallery. See below for more information.
    • Version. The available versions of the selected Image. These will be versions that are available in your selected region. If you need the most recent version, choose Latest from the list.

Security & Login​

Select your required Security Type, and enter your authentication details.

  1. Select one of these options from the Security Type list.

    • Standard: This is the basic level of security for your virtual machines.
    • Trusted Launch Virtual Machines: Protects against persistent and advanced attacks on Gen 2 virtual machines with configurable features like secure boot and virtual Trusted Platform Module (vTPM).
    • Confidential Virtual Machines: On top of Trusted launch, Confidential Virtual Machines offers higher confidentially and integrity guaranteed with hardware-based trusted execution environment.

  2. Select at least one available option from the Configured Security Features list.

  3. Specify authentication details for launching your VMs:

    • Username.
    • Your Windows login password.
    tip

    When a Specialized Shared Image is specified, you do not need to specify login information.

Workload Capacity​

  1. Set the workload capacity target, and the minimum and maximum limits:

    • Target. The number of low priority VMs in your Elastigroup.
    • Minimum. In the case of a scale down policy action, this is the minimum number of low priority VMs that must run in the group. The minimum acceptable value is 0.
    • Maximum. In the case of a scale up policy action, this is the maximum number of low priority VMs allowed in the group. The minimum acceptable value is 0.

  2. Set a workload capacity option:

    % spot-instances. The percentage of low priority VMs to include in the group. Use the slider to set the percent. The remaining percentage will be on-demand instances.

    • Specific on-demand Count. The number of on-demand VMs to include in the group.

Step 2: Compute​

VM Sizes​

You can use attribute-based Spot VM size selection, or you can manually create a list of VM sizes to launch:

  • Attribute-based: you can specify a set of VM size attributes (such as vCPU, memory, and storage) and select VM sizes to exclude.

    View image

  • Manual: find and select the VM size to update.

    View image
important

You need to select at least one Spot VM size and one On-demand (OD) VM size.

You can expand Unsupported VM sizes to see why a VM size isn’t available.

View image

Each VM size provides the following information:

  • VM Size: list of relevant VM sizes. This list can change according to the selected image (the OS type) and the region selected in the Basics tab.
  • Type
  • vCPUs
  • Memory (GiB)
  • Storage (GiB)
  • Spot Cost/Month: the cost per month according to Azure pricing.
  • Preferred Spot: select preferred spot VM sizes. Selecting a VM size as preferred indicates that the group should launch the preferred VM sizes prior to the remaining VM sizes that are defined as spot sizes.

You can change the columns by clicking the column selector.

On-Demand​

Select an on-demand VM size. At least one VM needs to be defined as on demand. This is applicable when you have on demand as the preferred lifecycle or significant as it provides a fallback in case spot VMs are unavailable.

tip

To maximize cost savings, provide the group with all possible Spot VMs compatible with the expected workload. The more VM sizes you select, the greater the chances that the group will find an available spot VM to run on.

Market Statistics​

You can view the market statistics of your selection.

  • Selected Markets: This shows the amount of selected markets.
  • Average Savings: The average savings you can achieve by utilizing the selected spot markets.

Availability Settings​

These are the availability settings:

  • Draining Timeout. This is the time (in seconds) to deregister and drain the VM before its termination. During the detachment process, the VM is unregistered from the Load Balancer/Application Gateway, DNS, and other relevant components. Typically, draining timeout is used for load-balanced integrations. The default timeout is 120 seconds.

  • Fallback to On-Demand. Select this option to automatically fall back to an On-Demand VM when no spot VMs are available.

  • Continuous Optimization. Select when Elastigroup can move workloads from On-Demand to spot VMs.

    • Once available. Elastigroup moves the workloads when your spot VM types become available.

    • Custom. For example, if you want your on-demand instances to revert to spot instances daily, Monday through Friday, between 9 AM and 8 PM, you need to select:

      • From: Monday, start time: 09:00, to: Monday, end time: 20:00.

      • From: Tuesday, start time: 09:00, to: Tuesday, end time: 20:00.

      • From: Wednesday, start time: 09:00, to: Wednesday, end time: 20:00.

      • From: Thursday, start time: 09:00, to: Thursday, end time: 20:00.

      • From: Friday, start time: 09:00, to: Friday, end time: 20:00.

        note

        If you select From: Monday, start time: 09:00, to: Friday, end time: 20:00, your on-demand instances can revert to spot instances past 8 PM on Mondays, Tuesdays, Wednesdays, and Thursdays. It will set the entire time between Monday at 9 AM all the way through Friday at 8 PM, not daily between 9 AM and 8 PM.

Strategy​

These are the strategy settings:

  • Cluster Orientation: These are the orientation strategy options:

    • Availability. Select VMs to ensure the best market availability within your group.
    • Cost. Prioritize VM types by their costs. This option will also consider availability in the selection process.

  • Utilize CRG: Capacity reservation group (CRG) can be used in Elastigroup. You must create a CRG in Azure before Elastigroup can utilize the CRG. CRG is only be available when it correlates with the enabled VMs in the group configuration.

    1. Select Utilize CRG and how you want the group to use your CRG:

      • Utilize over spot instances: Provides CRG utilization as a priority before the group picks up the next OD market.
      • Utilize over On-Demand: Provides CRG utilization as a priority before the group picks up the next On-Demand market.

    2. Select the CRG you want Elastigroup to use:

      • Automatic: Elastigroup searches for available CRG slots in your subscription and utilizes the available slots in the configuration of each group.
      • Manual: Provide details for the CRG you want to be utilized as part of the group.

Advanced Options​

  • Scheduling: You can define actions that occur at specific times on the VM. For example, you can shut down the VM and turn it back on with the same persisted resources (Storage & Network) at specific times. This is done by defining the times in the Cron expressions.

    To schedule: Click + Add Task, select an action type, and enter the times you want to define with Cron expressions.

  • Tags. Add tag keys and values you want associated with the group VMs.

  • Managed Identity. Select the Managed Identity for your VMSS instances.

  • User Data: (or Custom Data): You can configure a script that will run during every VM startup. It’s useful for launching VMs with all the preset configurations and software installations. Elastigroup can load custom user data (such as custom scripts) when provisioning VMs. Custom data is not available for specialized shared images.

    Make sure your script doesn’t require additional extensions. For example, you may need to add an extension for custom data to work.

    Extension for custom data

    In the group > compute > launchSpecification > extensions, add the extension. For example:


       "extensions": [
           {
             "name": "extensionName",
             "type": "customScript",
             "publisher": "Microsoft.Azure.Extensions",
             "apiVersion": "2.0",
             "minorVersionAutoUpgrade": true,
             "publicSettings": {},
             "protectedSettings": {},
             "enableAutomaticUpgrade": false,
             "protectedSettingsFromKeyVault": {
               "sourceVault": "/subscriptions/1234-1234-1234/resourceGroups/rg_test/providers/Microsoft.KeyVault/vaults/testKeyVault",
               "secretUrl": "https://testKeyVault.vault.azure.net/secrets/SecretTest/123456"
             }
           }
         ],

  • Shutdown Script: You can configure a shutdown script, but this requires an agent to be installed on the instance.

Step 3: Networking​

Virtual Network​

Enter/select the virtual network.

  1. Virtual Network. Select the specific Virtual Network (VN) for your group.

  2. Set the public IP SKUs for all network interfaces, with one of these options:

    • None.
    • Basic.
    • Standard.

  3. For each network interface, select these parameters:

    1. Subnet: Select the Subnet inside your VN.
    2. Network Security Group: The network security group to associate with the VM.
    3. Application Security Group: Provides security micro-segmentation for virtual networks in Azure. This lets you define network security policies based on workloads, for example, applications, instead of explicit IP addresses. Assign the group to one or more application security groups.
    4. Assign Public IP. Mark this checkbox if you want VMs in this group to launch with a Public IP. You should then select one or more static public IPs from the list. The list includes only IPs from the availability zones selected for the group.
    note

    • The items that appear in the list of application security groups depend on the Virtual Network that you selected and may be different in each network.

    • To add another network interface, click + Add Network Interface.

    • To limit dynamic IP creation during VM deployment:

      • Select IPs that are indicated as No zone/Zone redundant. These will ensure the most AZ flexibility.
      • The optimal number of public IPs for the pool is twice your maximum capacity. For example, if your maximum capacity is 6 VMs, then choose at least 12 public IP addresses.
      • If you select zonal IPs (e.g., in zones 1, 2, 3), then distribute them equally across the zones.

Load Balancer​

Optionally add one or more load balancers. Elastigroup will automatically register new VMs to the configured load balancer.

note
  • A group can have only one internal and one public (standard) load balancer.
  • If you assigned a Basic Public IP to your network interfaces, public load balancers will not be available.
  • To add another load balancer, click + Add Load Balancer.

For each load balancer, select these parameters:

  1. Set the public IP SKUs for all network interfaces, with one of these load balancer types:

    • Public Load Balancer: Limited to one per group.
    • Internal Load Balancer: Limited to one per group.
    • Application Gateway.

  2. Select the Load Balancer.

  3. Select from the available Backend Pools.

Health Checks​

Optionally add Health Checks.

  1. Select one or more health check types:

    • VM state.
    • Application Gateway (only available if at least one application gateway is defined in the group).

  2. Enter a health check grace period (sec). This is the time to allow a VM to boot and applications to fully start before the first health check. The default value is 120 secs. You cannot enter a value below 120 seconds.

  3. If you previously selected the VM State health check, the Auto healing option is enabled. Auto healing checks VM health and replaces unhealthy VMs.

    • Enter the Unhealthy Duration (sec). This is the time to keep an unhealthy VM before it is terminated and replaced. The default value is 120 secs. You cannot enter a value below 60 seconds.

Step 4: Review and Create the Group​

  1. Review your configuration

    • Summary

      You can make changes to the configuration in each section by clicking the Pencil icon.

    • JSON

      You can make changes in the configuration by editing the JSON directly by turning on Edit mode. You can collapse and expand each section of the JSON. Furthermore, you can export the configuration to a JSON file, and you can also copy it to your clipboard.

      Changes made in the JSON file won’t be reflected in the Summary page. If you make changes in the JSON file and then navigate back to one of the wizard tabs, the changes won’t be saved.

      Learn more about using the Spot API to [configure your Elastigroup group](need create link).

    • Terraform

    You can make changes in the Terraform by turning on Edit mode.

  2. To create the group, click Create.