Building A Reusable Testing Pipeline for AWS Amplify Plugins
Authored by Nathan Agez, certified software engineer at TrackIt. The content and opinions in this post are those of the third-party author. AWS is not responsible for the content or accuracy of this post.
About AWS Amplify
AWS Amplify is a set of tools and cloud services that help front-end developers build scalable full-stack web and mobile applications powered by AWS (Amazon Web Services). Amplify tools can be used together or on their own, to develop applications with popular web frameworks including JavaScript, React, Angular, Vue, Next.js, and mobile platforms including Android, iOS, React Native, Ionic and Flutter
One component of AWS Amplify is the Amplify-CLI. It provides intuitive configuration and maintenance of backend resources for Authentication, Storage, API and Analytics right from your desktop. It also enables 3rd party development of plug-ins that can extend the native resource management capabilities.
Amplify Video: The Need to Refactor the Codebase
With the rapid growth of Streaming Media applications, AWS Solutions Architects recognized that developers could benefit from a new Amplify-CLI managed resource type that focused on live and video-on-demand streaming. In April 2019, they made video a first-class citizen in application development by releasing an open source plugin for the Amplify-CLI named Amplify Video. AWS Amplify Video helps developers enrich their applications with live and video-on-demand streaming architectures without the time and expertise it would normally take a video engineer to implement these systems.
After the release, project maintainers struggled to balance new feature enhancements with community support as organizations like Orangetheory Fitness started using Amplify Video to launch entirely new services like OrangetheoryAtHome™.
To improve feature introduction velocity and supportability, AWS project maintainers collaborated with TrackIt to refactor the existing code base and build a test pipeline for integration tests, unit tests, and end-to-end (E2E) tests. Refactoring the code would ensure easier maintenance and the testing pipeline would allow new contributors to quickly test new features that were added to the plugin.
Building a Reusable Testing Pipeline for Amplify Plugins
TrackIt recognized that it made sense to build a generic pipeline that would not just be applicable to the Amplify Video plugin, but to all Amplify plugins, enabling members of the Amplify community to promptly test their new plugins using the pipeline.
Testing Framework
The testing framework comprises 3 categories of tests: unit, integration, and end-to-end (e2e) (still in the planning stage). The unit & integration testing framework was created using Jest.
Amplify CLI Headless Mode
Any user who calls the Amplify Video plugin on the Amplify CLI is provided with prompts to indicate the next steps to take.
In order to automate the testing process of the Amplify Video plugin in the CI/CD pipeline, we refactored the plugin’s code to add a headless mode that automatically answers the Amplify CLI’s prompts.
The Amplify CLI Headless Mode enables users to enter parameters and arguments to save time and automate the testing process.
Learn more about the Amplify CLI Headless Mode here: https://github.com/awslabs/amplify-video/pull/192
Challenge: Identifying all the Different Infrastructure Permutations
We realized that the more dependencies a specific infrastructure choice has, the more strenuous it becomes to manually write a script that takes all the possible permutations (4000+) of answers to CLI prompts into consideration. We had to find a more efficient way to leverage the Amplify CLI’s Headless Mode.
To address this challenge, we created a script that automatically generates shell scripts using the graph and generates all the permutations.
The algorithm (in the code snippet above) goes through a specific file called ‘test-helper’ that contains the values, types, and relations between CLI prompts to create a tree of all the possible paths a user can take while answering the CLI’s questions. The tree is then used to recursively generate all the paths from the first node until there is no next node. These paths are then used to generate scripts containing the payload to run a specific permutation.
Creating an Integration Testing Workflow
For the CI/CD pipeline, we used GitHub Actions, which in this application enables users to deploy the project inside an environment and run their tests.
For the integration testing, we added a new workflow that runs the different shell scripts and deploys the resources. The advantage of the framework is that it allows users to set up workflows before a test and shut them down after the test is complete. Once resources are deployed without errors, the team can test different scenarios and once that is complete, the deployed resources are immediately deleted to eliminate unnecessary spending.
Creating A Dev-Test Mode to Accelerate Testing
The testing process involved deploying resources in the cloud to run tests and then deleting the resources once the tests were complete. The TrackIt team realized that this process of constantly deploying and deleting resources can be tiresome and time-consuming.
To address this issue, we created a dev-test mode in the test framework that allows users to leverage an already existing project locally and to run tests. The team added a condition NODE_ENV (as seen in the screenshots below) that indicates the environment the user is in – dev or test.
This means that the main test pipeline is only used by GitHub Actions to initially deploy the resources. Users in the dev phase can hence run their tests against a local project, making it easier to debug, test only the new tests they’ve added, etc.
An example of a test running locally:
With the setup of this workflow, whenever a contributor wants to add new tests for a specific feature/branch/patch he must take note of the test directory’s architecture. It is composed of integration tests and unit tests.
Integration tests require resources to be deployed. For instance, testing an Amazon CloudFront distribution endpoint would require deploying and creating an Amazon CloudFront endpoint on the console along with a private key to sign URLs. Unit tests, however, are run against the code-base and do not require deployed resources.
All unit tests go into the “tests” directory whereas integration tests go into the “tests/integration” directory (see screenshot below).
Users get to run their tests on a local testing mode called “dev-test” mode. It performs tests on a local Amplify project instead of deploying resources each time the user runs a new integration test. Depending on the test the user writes, they will probably have to fetch data inside a specific Amplify directory. In this case they have to check the following environment variables:
process.env.NODE_ENV !== 'test' && process.env.AMP_PATH
This value means that all the tests are running on a local Amplify project on the user’s machine.
Conclusion
The main goal for the testing pipeline was to create an easy-to-maintain solution that would enable Amplify users to quickly build and test their own plugins without having to deploy resources manually and execute laborious tests. Users willing to create and test their own plugins can do so in three simple steps by referencing this article:
- Quickly create plugins with the Amplify CLI Headless mode
- Set up a Jest testing framework
- Write bash scripts for the Headless mode which will deploy resources for the tests automatically
Documentation & Resources
https://docs.amplify.aws/cli/usage/headless
https://github.com/awslabs/amplify-video/wiki/Project-Navigation#tests
https://github.com/awslabs/amplify-video/wiki/Tests
https://github.com/awslabs/amplify-video/issues/17
About TrackIt
TrackIt is an Amazon Web Services Advanced Consulting Partner specializing in cloud management, consulting, and software development solutions based in Venice, CA.
TrackIt specializes in Modern Software Development, DevOps, Infrastructure-As-Code, Serverless, CI/CD, and Containerization with specialized expertise in Media & Entertainment workflows, High-Performance Computing environments, and data storage.
TrackIt’s forté is cutting-edge software design with deep expertise in containerization, serverless architectures, and innovative pipeline development. The TrackIt team can help you architect, design, build and deploy a customized solution tailored to your exact requirements.
In addition to providing cloud management, consulting, and modern software development services, TrackIt also provides an open-source AWS cost management tool that allows users to optimize their costs and resources on AWS.
Schedule a consultation with TrackIt
Author Bio
Nathan Agez is a software engineer at TrackIt with master’s degrees in computer science from EPITECH France. He has deep knowledge of the AWS Amplify ecosystem and also has extensive experience in building Serverless infrastructure for a variety of projects. He is passionate about open source, automation, travel, and loves playing his guitar when he’s not busy coding.