AWS Lambda Automated Deployment

Objective

The document details the steps to build, deploy and maintain different versions of lambda functions across various product stages such as dev, test, stage and prod across multiple AWS accounts.

 

AWS Lambda Versions & Aliases

AWS Lambda is a compute service that lets you run code without provisioning or managing servers. AWS Lambda executes your code only when needed and scales automatically.

By using versioning, you can manage your in-production function code in AWS Lambda better. When you use versioning in AWS Lambda, you can publish one or more versions of your Lambda function. As a result, you can work with different variations of your Lambda function in your development workflow, such as development, beta, and production.

By default, developers work on the Lambda that is pointing to the $latest version and once it is published manually through console or through AWS Command Line Interface or through AWS SDKs, a new version will be created automatically with auto incremented version number.

While publishing the lambda, we can additionally set an alias to the lambda to point to the stage to which we want to publish the lambda changes.

A Simple workflow of publishing a lambda to Dev instance is shown below –

 

 

 

 

The iOS/android/web App can invoke the AWS Lambda function through the ARN with appropriate permissions set via the IAM role as shown below.

 

 

 

 

Lambda Alias Versioning

 

In addition to the versioning maintained by AWS for each lambda, there is a need to version the Alias as well since there could potentially be older version of the iOS App which would still need to invoke older lambda functions. Provided schematic below explains alias versioning.

 

 

 

 

 

Apex Serverless Deployment Tool

Apex is the serverless automation tool that provides a way to build, publish and maintain AWS lambda functions. The lambda functions need to be defined as per the Apex standardized folder structure as described below.

 

Lambda Folder Structure (Apex Standards)

project.json – configuration file that contains global variables that will be applied across all lambda functions included under the project.

 

Functions (Parent Folder) – Parent folder which describes the project under which all lambda functions are defined. 

                  hello-world (Child folder – containing the lambda function)

main.py               – AWS Lambda code.

function.json       – configuration file that will include function level overrides as well variables pertaining to the specific function.

requirements.txt  – package dependency list which will be downloaded, built and zipped along with lambda by Apex function.

event.json            – test input for the lambda function.

 

 

Automated Deployment with Jenkins and Apex

Jenkins is an open source automation server which helps to automate the non-human part of software development process, with continuous integration and facilitating technical aspects of continuous delivery.

Bringing it all together, we can configure each code submission to GitHub repository to trigger a build in Jenkins to invoke APEX Serverless commands to publish the lambda function with the appropriate alias.

The schematic below describes the process –

 

 

Conclusion

By combining multiple tools such as Apex and Jenkins we can setup an automated process to deploy code committed to GitHub repository to various environments, spread across multiple AWS accounts while also supporting backward compatibility of mobile apps which can have users spread across multiple versions of the app.

This automated deployment process frees up developers from worrying about complicated deployments of a serverless project and focus on what’s most important – building a world class app.

Importance of relationship

Importance of relationship

A database cannot exists without a relationship. It connects tables which are logical related to each other. Even for NoSQL database we require to create some sort of relationship between collections.

For e.g. I have an application which manages society. A society will have flats and each flat will have registered members. I want to list all the members present in a flat. To achieve this, I will add an association between House entity and Person entity. This association can be of 2 types, reference and embedded.

In NoSQL, managing both type of association and enforcing this constraint is a pain in itself. We need a framework which should manage relationship-level integrity.

OneToMany relational attribute

Nodedata provides simple annotation based solution for this. To understand this, let go through below models first.

Next let’s add relationship between class and student in class entity

This is it. Now it is fully managed by framework.

Implementation and CRUD over relational attribute

As explained in the the blog (//url for nodedata first blog), Nodedata will automatically create the Rest API over these models. We will now insert data and add association between them.

Framework also provide a short-hand to create the relationship as follows.

Both of the above set of request will insert new flat which have one member named ‘Steve’. Now, to get house and its members, we can execute following API

Following will add another member ‘Clark’ into the  flat number 001

Following will remove ‘Clark’ from ‘First’ class

Following will update the ‘Steve’ name to ‘Steve Smith’.

Above query will also update all the places wherever this instance is used.

Relationship strategy

NoSQL is a schema-less storage which provides flexibility in designing the database. Which means as per the actions present in the application we should design the relationship saving strategy. We can achieve this by configuring the annotation. We will go through case wise and see how it can be achieved.

Case 1: Action to fetch in only number of students present in the class.

Following annotation will save references of ‘Steve’ student in ‘First’ class

// code

Case 2: Action to get complete class information along with student information

Following annotation will embedded ‘Steve’ student in ‘First’ class

// code

Case 3: Action to get partial information of students present in the class

With growing number of students, document size will also grow, so this will require us to minimize the properties which we want to keep along with class object. Following annotation will embedded ‘Steve’ student’s name only in ‘First’ class object.

// code

Case 4: Action to get complete class information along with student information where each student is having large set of properties.

Since the student is having large set of properties, it will be difficult to embedded them in class object. Following annotation will save references of ‘Steve’ student in ‘First’ class and on fetching ‘First’ class it will also return the complete student object.

// code

Case 5: Action to delete all the students on deleting the class so that no orphan is present

Following annotation will embedded ‘Steve’ student in ‘First’ class. On deleting the ‘First’ class, all the students will also get deleted.

// code

There is 1 more very important aspect is database design change. Suppose initially you went with the action mentioned in Case 1 and after some time the requirement is changed to Case 3. Well this can be achieved by changing the annotation.

But what about existing data migration ? Well, let the framework do it for you. Just execute the put on all the class objects with the same data and it will update the class entity according to the new relation mentioned.

// code for executing the migration

Prerequisite

  1. mongodb is installed and server is running on default port.
  2. Node 6.9.0 is installed.
  3. IDE like vscode is available to you.
  4. Rest client like postman,curl for testing.

Installation

  1. git clone https://github.com/ratneshsinghparihar/nodedata-demo-sample.git
  2. cd nodedata-demo-sample/Demo-Sample
  3. npm install

Code changes

  • Add a model blogmodel.ts inside models folder.
  • Add a repository blogmodelrepository.ts inside repositories folder.
    • Model with @document tell the system to create a document inside blogs collection.
    • repository with blog name will create all necessary rest end points.

Testing

  1. npm start
  2. Post a JSON {“name”: “testBlog”} to http://localhost:9999/data/blogs
  3. Hit the api to get data(http://localhost:9999/data/blogs)
  4. Hit the api with put method http://localhost:9999/data/blogs/{{blogId}} with body {“name”: “testBlog1”}
  5. Hit the api with delete method http://localhost:9999/data/blogs/{{blogId}}

Conclusion

Now you can see how creating rest apis is super easy with node-data. If you want to know more check out the our github
 
https://github.com/ratneshsinghparihar/Node-Data

Or visit our main page

https://nodedataio.azurewebsites.net/

Nodejs Authentication using node data

Every application needs a way to register new users and then authenticate them.
Node data provides an easier way to implement it.

Prerequisite

  1. mongodb is installed and server is running on default port.
  2. Node 6.9.0 is installed.
  3. IDE like vscode is available to you.
  4. Rest client like postman,curl for testing.

Installation

  1. git clone https://github.com/riteshagarwal100/nodedataauth.git
  2. cd nodedataauth/Demo-Sample
  3. npm install

Code To Verify

  • Check the below code in current-user-detail-service.ts present in root directory
  • This code is managed by the developer, who decides how to parse the incoming user register request and then save it in users custom mongodb collection.
  • Node data uses bcrypt-nodejs, so encrypt the password field using it.


Testing

  1. npm start
  2. Post a JSON {
    “name”:”username”,
    “password”:”pwd”
    } to http://localhost:9999/register
  3. This will create a new user in the data base,with above credentials
  4. Post a JSON {
    “username”:”username”,
    “password”:”pwd”
    } to http://localhost:9999/login
  5. This will authenticate the user, and user will be logged in.
  6. This will create a session for this user
  7. Post a JSON {
    “name”:”teach”,
    “courses”:[“a”,”b”]
    }to http://localhost:9999/data/teacher
  8. This will create a teacher Object in data base
  9. Hit GET for http://localhost:9999/data/teacher
  10. This will return all the teachers present in the data base

Conclusion

It’s super easy to create authentication using node data.
 
https://github.com/ratneshsinghparihar/Node-Data

Or visit our main page

https://nodedataio.azurewebsites.net/

Ways to create  REST API’s

For your first application lets assume that you are building an application for blogs and you want to create  REST apis which can be consume by frontend/mobile application .

    • API should create blogs into database.
    • API should update blogs into database.
    • API should delete blogs from database.
    • API should find and fetch a blog using any existing blog id from database.

Prerequisite

  1. mongodb is installed and server is running on default port.
  2. Node 6.9.0 is installed.
  3. IDE like vscode is available to you.
  4. Rest client like postman,curl for testing.

Installation

  1. git clone https://github.com/ratneshsinghparihar/nodedata-demo-sample.git
  2. cd nodedata-demo-sample/Demo-Sample
  3. npm install

Code changes

  • Add a model blogmodel.ts inside models folder.
  • Add a repository blogmodelrepository.ts inside repositories folder.
  • Model with @document tell the system to create a document inside blogs collection.
  • repository with blog name will create all necessary rest end points.

Testing

  1. npm start
  2. Post a JSON {“name”: “testBlog”} to http://localhost:9999/data/blogs
  3. Hit the api to get data(http://localhost:9999/data/blogs)
  4. Hit the api with put method http://localhost:9999/data/blogs/{{blogId}} with body {“name”: “testBlog1”}
  5. Hit the api with delete method http://localhost:9999/data/blogs/{{blogId}}

Conclusion

Now you can see how creating rest apis is super easy with node-data. If you want to know more check out the our github
 
https://github.com/ratneshsinghparihar/Node-Data

Or visit our main page

https://nodedataio.azurewebsites.net/

NoSQL…Hang on a sec

Since there is too much information available online, one has to carefully decide whether to go for NoSQL or not. And if you decide to use NoSQL, then which database or datastore should be the right choice for you.

Now to answer the first question of NoSQL or not, you should analyze your requirement in detail. Amongst the numerous points that have to be considered, I will mention a few important ones here:

  1. Size of your Data: Evaluate how your data is! Is storing your data in relational database like MySQL or Oracle an option? How many tables do you need to create? How many columns per table will you have on an average and most importantly how many rows each table will have?
  2. Flexibility: In a relational database, you need to create the schema first. Do you need some flexibility over there? Like in one of my projects I worked on a logging module and the structure of the logs differed. Therefore I wanted to have flexible schema for it.
  1. Data Retrieval or Faster Write Access: While some applications require fast data retrieval, some require fast write access and a few require both. Think about Google Search where fast data retrieval is very important whereas for an application like Twitter,  lot of tweets require  lots of write operation.
  1. Concurrent Read & Write Access: It’s not just the speed of the application that matters but also the concurrent read and write access that should be taken into consideration. Think about the number of Facebook users writing simultaneously on various sections of the website.
  1. Are you creating application for Analytics?
  1. Social Network Integration: Does your application have social-network features? Facebook is a big inspiration to choose NoSQL if your application has similar features. Facebook Engineering Notes and Facebook Engineering Papers are good sources of information about the latest technologies used at Facebook.
I hope these pointers help you take an effective technology decision for your product. I will give more pointers on this topic during this NoSQL series. Look here for the next post on databases.

News You Like Receives Rave Reviews!

Ranjan Kumar received accolades for developing the cool News app- News You Like – for Google Chrome.

News You Like lets you look through news snippets of your interest. You can simply click on Settings and a page will appear in the foreground where you can enter topics you are interested in. Once you click Save, you will find your preferences in the list of news topics. So you can click on the respective tabs to read the related news.

With this app, one does not have to filter through to read news of one’s own interest. The best part of this app is that you can change your preferences from time to time. You can remove certain topics in case you want to see less number of topics.

Ranjan has been covered in the October issue of Digit magazine for this innovative contribution. To know more, download this useful app from Chrome Web Store