How To Make AWS Nimble Workstation Persistent

Amazon Studio in the Cloud and Nimble Studio is a service for creative studios to produce video, visual effects, animations, and interactive content. The cloud-based studio provides on-demand access to virtual workstations, elastic file storage, render farm capacity, and tools to manage security, permissions, and collaborations.

The Problem

Following a recent update of Studio in the Cloud and Nimble Studio, workstations now have a lifecycle that is possible to control through an AWS SDK. However, workstations with a defined lifecycle could be a problem if you want to ensure some data is persistent. The benefits of the lifecycle implementation are that a machine in a Stopped state is much quicker to launch later on and retains its previous state.

If a Studio in the Cloud and Nimble Studio machine is up longer than 11.5 hours in the state Ready or Stopped the machine will then terminate the streaming session.

It is possible to extend the streaming machine time limit. The maximum for the Stoppedstate, for instance, is up to four days, as you can see in the image below. If a persistent Stopped state longer than four consecutive days is required, a programmatic solution is necessary.

Solution

To solve this problem we must set up a process that will automatically start and stop a workstation before its termination, checking every day. We will detail the steps below so you can implement all the workflow to get the job done.

How to Deploy

1. First, create a lambda in charge of starting nimble instances called start-nimble-workstation. This is pretty straightforward; we list all the workstations in the Studio in the Cloud and Nimble Studio and start them if they return the state Stopped. Concurrently we check if the termination date will happen prior to the next 24 hours. One thing to note, you may have to change the REGION_NAME variable to match which region you set up your Studio in the Cloud and Nimble Studio.

import json
import boto3
import time
from botocore.config import Config
import datetime

def lambda_handler(event, context):
    REGION_NAME = "eu-west-2"
    
    # Create a Session
    session = boto3.Session()
    
    # Create a Studio in the Cloud and Nimble Studio client to interact with Studio in the Cloud and Nimble Studio
    nimble_client = session.client('nimble', region_name= REGION_NAME)
        
    # List available studios for this account
    response = nimble_client.list_studios()
        
    # Get the studioId
    studio_id = response['studios'][0]['studioId']
    
    # Check existing streaming session
    response = nimble_client.list_streaming_sessions(studioId=studio_id)
    sessions = response['sessions']
    
    for session in sessions:
        terminateDate = session['terminateAt'].replace(tzinfo=None)
        nextJobDate = (datetime.datetime.now() + datetime.timedelta(days=1)).replace(tzinfo=None)
        
        if session['state'] == 'STOPPED' and nextJobDate > terminateDate:
            session_id = session['sessionId']
            # Start streaming session
            response = nimble_client.start_streaming_session(
                sessionId=session_id,
                studioId=studio_id,
            )
    
    return {
        'Version': boto3.__version__,
        'studioId': studio_id,
        'statusCode': 200,
        'body': event
    }

2. Create a second lambda, called stop-nimble-workstation, to stop instances in essentially the same manner as the function in charge of starting nimble instances.

import json
import boto3
from botocore.config import Config

def lambda_handler(event, context):
    REGION_NAME = "eu-west-2"
    
    # Create a Session
    session = boto3.Session()
    
    # Create a Studio in the Cloud and Nimble Studio client to interact with Studio in the Cloud and Nimble Studio
    nimble_client = session.client('nimble', region_name= REGION_NAME)
        
    # List available studios for this account
    response = nimble_client.list_studios()
        
    # Get the studioId
    studio_id = response['studios'][0]['studioId']
    
    # Check existing streaming session
    response = nimble_client.list_streaming_sessions(studioId=studio_id)
    sessions = response['sessions']
    
    for session in sessions:
        if session['state'] == 'READY':
            session_id = session['sessionId']
            response = nimble_client.stop_streaming_session(studioId=studio_id,sessionId=session_id)

    
    return {
        'Version': boto3.__version__,
        'studioId': studio_id,
        'statusCode': 200,
        'body': event
    }

Make sure, that the boto3version in both of your lambdas is above 1.19.11, so you have access to the latest API methods of Studio in the Cloud and Nimble Studio:

• list_streaming_session
• start_streaming_session
• stop_streaming_session

You can check this article to update a python module in your lambda.

3. Create a state machine that will execute the entire workflow. For that, navigate to the Step functions dashboard in the AWS console. Then click on Create state machine. Check, Write your workflow in code, and paste the code below.

{
    "Comment": "Start and stop to make nimble workstations persistent",
    "StartAt": "Start Nimble Workstation",
    "States": {
      "Start Nimble Workstation": {
        "Type": "Task",
        "Resource": "arn:aws:states:::lambda:invoke",
        "OutputPath": "$.Payload",
        "Parameters": {
          "FunctionName": "arn:aws:lambda:eu-west-2:677859638064:function:start-nimble-workstation:$LATEST"
        },
        "Retry": [
          {
            "ErrorEquals": [
              "Lambda.ServiceException",
              "Lambda.AWSLambdaException",
              "Lambda.SdkClientException"
            ],
            "IntervalSeconds": 2,
            "MaxAttempts": 6,
            "BackoffRate": 2
          }
        ],
        "Next": "Wait"
      },
      "Wait": {
        "Type": "Wait",
        "Seconds": 1200,
        "Next": "Stop Nimble Workstation"
      },
      "Stop Nimble Workstation": {
        "Type": "Task",
        "Resource": "arn:aws:states:::lambda:invoke",
        "OutputPath": "$.Payload",
        "Parameters": {
          "FunctionName": "arn:aws:lambda:eu-west-2:677859638064:function:stop-nimble-workstation:$LATEST"
        },
        "Retry": [
          {
            "ErrorEquals": [
              "Lambda.ServiceException",
              "Lambda.AWSLambdaException",
              "Lambda.SdkClientException"
            ],
            "IntervalSeconds": 2,
            "MaxAttempts": 6,
            "BackoffRate": 2
          }
        ],
        "End": true
      }
    }
  }

Click on Next, give a name to your state machine, then click on Create state machine.

4. Create a Cron Job.In order to do so navigate to the EventBridge dashboard in the AWS console. On the left panel, click on Rules.
Click on Create rule, and give the rule a name.

On the Define pattern section, check Schedule, then check Cron expression, write 0 4 * * ? * so it will execute every day at 4 AM.

On the Select targets section, look for the Step Functions state machine in the first scrolling menu and, on the second one, pick the state machine name you created before. Click on Create. Check if the rule is enabled.

Conclusion

You have now implemented a full start and stop the process to keep your workstations persistent!

About TrackIt

TrackIt is an Amazon Web Services Advanced Consulting Partner specializing in cloud management, consulting, and software development solutions based in Venice, CA.

TrackIt specializes in Modern Software Development, DevOps, Infrastructure-As-Code, Serverless, CI/CD, and Containerization with specialized expertise in Media & Entertainment workflows, High-Performance Computing environments, and data storage.

TrackIt’s forté is cutting-edge software design with deep expertise in containerization, serverless architectures, and innovative pipeline development. The TrackIt team can help you architect, design, build, and deploy customized solutions tailored to your exact requirements.

In addition to providing cloud management, consulting, and modern software development services, TrackIt also provides an open-source AWS cost management tool that allows users to optimize their costs and resources on AWS.