In a previous article, titled ‘Building E-Commerce Order Management Systems – Technical Article #1 – Architecture’, the TrackIt team outlined the different strategic and architectural choices developers can make to build robust, cost-effective, and flexible order management systems.
This second article in the E-Commerce Technical series focuses on the implementation process and details the different steps involved in setting up an OMS.
The first step in the implementation process is to create a Serverless project. Readers can follow the instructions in the Serverless Framework Quick Start Guide to install the Serverless Framework open-source CLI and deploy a sample service that reports deployment information and other optional metrics to the Serverless Framework Dashboard.
The second step in the implementation process is to set up the logic for the order management system. This begins with the addition of the first step Function and Lambda functions.
We need to create the serverless.yml
to define the basic Serverless configuration with Step Functions and Lambda functions. In the example below, the state machine has only one state ProcessOrder
:
service: 'oms-article' plugins: - serverless-step-functions configValidationMode: error provider: name: aws runtime: nodejs14.x memorySize: 1024 # https://aws.amazon.com/fr/blogs/compute/operating-lambda-performance-optimization-part-2/ region: 'us-west-2' functions: processOrderHandler: handler: src/processOrder.handler stepFunctions: stateMachines: processOrder: events: - http: path: processOrder method: GET definition: Comment: "Process Order - Process an order" StartAt: ProcessOrder States: ProcessOrder: Type: Task Resource: Fn::GetAtt: [processOrderHandler, Arn] End: true
Serverless Configuration
Next, we create the handler src/processOrder/index.js
const processOrderHandler = async () => { console.log('PROCESS ORDER HANDLER'); }; module.exports = { handler: processOrderHandler, };
Handler Code Snippet
After adding the first Step Function, the next step is to configure the OMS database and define a model that defines how data is stored within the database.
We use the Sequelize ORM (Object-Relational Mapping) to connect to our database – the configuration is added in src/processOrder/index.js
. We also define a simple
model and create an entry every time the Step Function is called. The database table is created manually for the sake of simplicity.Order
const processOrderHandler = async () => { console.log('PROCESS ORDER HANDLER'); }; module.exports = {const Sequelize = require('sequelize'); const dbConfig = { host: 'yourDBHost', port: 'yourDBPort', database: 'yourDBName', username: 'yourUsername', password: 'yourPassword', dialect: 'postgres', // one of 'mysql' | 'mariadb' | 'postgres' | 'mssql' schema: 'yourSchema', pool: { max: 5, idle: 30 }, dialectOptions: { connect_timeout: 3000, }, }; const sequelize = new Sequelize(dbConfig.database, dbConfig.username, dbConfig.password, dbConfig); class Order extends Sequelize.Model {} Order.init({ id: { type: Sequelize.INTEGER, autoIncrement: true, primaryKey: true, }, status: { type: Sequelize.STRING(20), allowNull: true, }, }, { sequelize, schema: sequelize.options.schema, tableName: 'order', createdAt: 'created_at', updatedAt: 'updated_at', }); const processOrderHandler = async () => { console.log('PROCESS ORDER HANDLER'); // We create a DB entry to save the order status const order = await Order.create({ status: 'success', }); return order; }; handler: processOrderHandler, }; module.exports = { handler: processOrderHandler, Order, };
Sequelize Configuration and Model Definition
Once the OMS database is configured and the orders are saved in the database for each execution, we set up an SQS queue that receives an order creation notification when a new order is created.
We attempt to send a message to a FIFO (First-In-First-Out) SQS queue – predominantly used to forward messages between services – and we verify that it was sent properly.
const Sequelize = require('sequelize'); const AWS = require('aws-sdk'); const dbConfig = { host: 'yourDBHost', port: 'yourDBPort', database: 'yourDBName', username: 'yourUsername', password: 'yourPassword', dialect: 'postgres', // one of 'mysql' | 'mariadb' | 'postgres' | 'mssql' schema: 'yourSchema', pool: { max: 5, idle: 30 }, dialectOptions: { connect_timeout: 3000, }, }; const sequelize = new Sequelize(dbConfig.database, dbConfig.username, dbConfig.password, dbConfig); class Order extends Sequelize.Model {} Order.init({ id: { type: Sequelize.INTEGER, autoIncrement: true, primaryKey: true, }, status: { type: Sequelize.STRING(20), allowNull: true, }, }, { sequelize, schema: sequelize.options.schema, tableName: 'order', createdAt: 'created_at', updatedAt: 'updated_at', }); const sqs = new AWS.SQS({ apiVersion: '2020-10-06' }); const processOrderHandler = async () => { console.log('PROCESS ORDER HANDLER'); // We create a DB entry to save the order status const order = await Order.create({ status: 'success', }); // We attempt to send a message to the queue const res = await sqs.sendMessage({ MessageBody: JSON.stringify({ message: 'the order has been processed', id: order.id, }), QueueUrl: 'https://yourQueueUrl', // MessageGroupId and MessageDeduplicationId params only apply to FIFO queues, can be removed otherwise MessageGroupId: 'yourGroupId', MessageDeduplicationId: Math.floor(Math.random() * Math.floor(1000000)).toString(), }).promise(); if (res.stack) { console.log('Error while sending message to queue', { res }); } return order; }; module.exports = { handler: processOrderHandler, Order, };
SQS Queue Configuration + Message Sending and Verification
Adding error handling to automatically re-execute Lambdas in case of errors, such as temporary database connection issues, is important when setting up a reliable and robust order management system.
We edit the serverless.yml
file to add a retry mechanism. The interval, max attempts and backoff rate need to be adapted to the reader’s project.
service: 'oms-article' plugins: - serverless-step-functions configValidationMode: error provider: name: aws runtime: nodejs14.x memorySize: 1024 # in MB. vCPUs are proportional to the memory size. 1024MB is often the most cost effective allocation region: 'us-west-2' functions: processOrderHandler: handler: src/processOrder.handler stepFunctions: stateMachines: processOrder: events: - http: path: processOrder method: GET definition: Comment: "Process Order - Process an order" StartAt: ProcessOrder States: ProcessOrder: Type: Task Resource: Fn::GetAtt: [processOrderHandler, Arn] End: true Retry: - ErrorEquals: - States.ALL IntervalSeconds: 5 MaxAttempts: 5 BackoffRate: 2
Adding the Retry Mechanism to the Serverless Configuration File
Adding GraphQL is essential when users wish to implement external services that request OMS data for their functioning.
The serverless.yml
file is edited to add the AppSync configuration that enables users to configure the GraphQL API and make it available to third-party services.
service: 'oms-article' plugins: - serverless-step-functions - serverless-appsync-plugin configValidationMode: error provider: name: aws runtime: nodejs14.x memorySize: 1024 # in MB. vCPUs are proportional to the memory size. 1024MB is often the most cost effective allocation region: 'us-west-2' functions: processOrderHandler: handler: src/processOrder.handler orderResolver: handler: appSync/resolvers/OrderResolver.handler stepFunctions: stateMachines: processOrder: events: - http: path: processOrder method: GET definition: Comment: "Process Order - Process an order" StartAt: ProcessOrder States: ProcessOrder: Type: Task Resource: Fn::GetAtt: [processOrderHandler, Arn] End: true Retry: - ErrorEquals: - States.ALL IntervalSeconds: 5 MaxAttempts: 5 BackoffRate: 2 custom: appSync: name: OMS-dev authenticationType: AWS_IAM schema: "./appSync/schemas/schema.graphql" dataSources: - type: AWS_LAMBDA name: OrderResolver description: 'order resolver' config: functionName: orderResolver defaultMappingTemplates: # default templates. Useful for Lambda templates that are often repetitive. Will be used if the template is not specified in a resolver request: "./requests/default.vtl" response: "./responses/default.vtl" mappingTemplatesLocation: "./appSync/mappingTemplates" mappingTemplates: - dataSource: OrderResolver type: Query field: getOrder
Adding the AppSync Configuration to the Serverless Configuration File
The next step is to create a GraphQl schema (appSync/schemas/schema.graphql
) that matches the order table:
schema { query: Query } type Query { getOrder(id: ID!): Order } type Order { id: ID! status: String }
Creating a GraphQL Schema for the Order table
We then need to create default mapping templates to define the request/response format for the handler.
appSync/mappingTemplates/requests/default.vtl
#** The value of 'payload' after the template has been evaluated will be passed as the event to AWS Lambda. *# { "version" : "2017-02-28", "operation": "Invoke", "payload": { "params": $utils.toJson($ctx.args), "selectionSetList": $utils.toJson($ctx.info.selectionSetList) } }
Request Mapping Template
appSync/mappingTemplates/responses/default.vtl
$util.toJson($context.result)
Response Mapping Template
Lastly, we define the GraphQL resolver appSync/resolvers/OrderResolver.js
that processes the requests and prepares the response.
const { Order, } = require('../../src/processOrder'); const fieldsMapping = { id: { request: (request) => ({ ...request, attributes: [ ...(request.attributes || []), 'id', ], }), response: (order, response) => ({ ...response, id: order.id, }), }, status: { request: (request) => ({ ...request, attributes: [ ...(request.attributes || []), 'status', ], }), response: (order, response) => ({ ...response, status: order.status, }), }, }; const buildRequest = ({ params, selectionSetList }) => { const baseRequest = { where: { id: params.id, }, }; return selectionSetList.reduce((request, field) => fieldsMapping[field].request(request), baseRequest); }; const buildResponse = ({ data, selectionSetList }) => { const baseResponse = {}; return selectionSetList.reduce( (response, field) => fieldsMapping[field].response(data, response), baseResponse, ); }; const handler = async (event) => { console.log('ORDER RESOLVER HANDLER'); const request = buildRequest({ params: event.params, selectionSetList: event.selectionSetList }); const data = await Order.findOne(request); const response = buildResponse({ data, selectionSetList: event.selectionSetList }); return response; }; module.exports = { handler, };
Adding the GraphQL Resolver
We can now execute GraphQL queries on the endpoint or on the AWS console. Here is an example:
query MyQuery { getOrder(id: "4") { id status } }
Example of a GraphQL Query
The package.json
file used for this project is as follows:
{ "name": "oms-article", "version": "1.0.0", "description": "oms-article", "main": "index.js", "dependencies": { "aws-sdk": "^2.793.0", "sequelize": "^5.22.3", "pg": "^8.2.1" }, "devDependencies": { "eslint": "^7.1.0", "eslint-config-airbnb-base": "^14.1.0", "eslint-plugin-import": "^2.20.2", "sequelize-cli": "^5.5.1", "serverless": "^2.41.2", "serverless-appsync-plugin": "^1.11.3", "serverless-step-functions": "^2.30.0" }, "author": "", "license": "" }
The package.json File Used in this Article
At this point, readers will have implemented a Serverless project that’s connected to an OMS database and also has the basic logic required to run an order management system.
Solution: Developers can split the project into different folders to ensure readability.
Solution: Developers can create SQL indexes to optimize SELECT data requests.
Solution: Developers can create SQL indexes to optimize SELECT data requests.
To develop features, developers should have their own environment to run tests. Developers often realize that it’s not easy to run E2E tests since they require a lot of setup.
Solution: Developers can create SQL indexes to optimize SELECT data requests.
const { generateDatabase } = require('./databaseGenerator'); const { generateServerless } = require('./serverlessGenerator'); const { generateEnv } = require('./envGenerator'); const { questionUser } = require('./questionUser'); const { generateQueues } = require('./queuesGenerator'); const config = { SERVICE_NAME: 'oms’, REGION: 'us-west-2', NODE_ENV: 'test_qa', API_GATEWAY_ENDPOINT_TYPE: 'regional', }; // UNIT TESTS const unitEnv = { TEST_HOST: '127.0.0.1', TEST_PORT: '5432', TEST_DB: 'postgres', TEST_USER: 'postgres', TEST_PASSWORD: 'postgres', }; const main = async () => { const { answersQa, envName } = await questionUser(); const queuesEnv = await generateQueues(answersQa.STAGE_QA); const context = { name: envName, config, answersQa, queuesEnv, unitEnv, }; await generateDatabase(answersQa); await generateEnv(context); const endpoint = await generateServerless(context); await generateEnv({ ...context, endpoint, }); }; try { main() .then((r) => r) .catch((err) => console.log(err)); } catch (err) { console.log(err); }
Script to Generate Dev Environment for Testing
This article has detailed the steps readers can take to implement robust order management systems and has also addressed some of the most common issues developers face when setting up an OMS. The next article in the E-Commerce Technical series will focus on OMS testing and different considerations companies need to make in order to test and ensure the proper functioning of their order management systems.
TrackIt is an Amazon Web Services Advanced Consulting Partner specializing in cloud management, consulting, and software development solutions based in Venice, CA.
TrackIt specializes in Modern Software Development, DevOps, Infrastructure-As-Code, Serverless, CI/CD, and Containerization with specialized expertise in Media & Entertainment workflows, High-Performance Computing environments, and data storage.
TrackIt’s forté is cutting-edge software design with deep expertise in containerization, serverless architectures, and innovative pipeline development. The TrackIt team can help you architect, design, build, and deploy customized solutions tailored to your exact requirements.
In addition to providing cloud management, consulting, and modern software development services, TrackIt also provides an open-source AWS cost management tool that allows users to optimize their costs and resources on AWS.