Introduction
Article 1:How to manage AWS Lambda Functions with Serverless Framework
You can use AWS Lambda for various purposes. It is a common way to connect their services with event triggers.
AWS Lambda doesn’t require provisioning and managing of servers. You can also edit functions in the online editor inside the AWS Management Console, so it’s easy to get started. On the other hand, you may face difficulties if you don’t manage them carefully.
In this article, I will explain how to manage AWS Lamdba Functions with Serverless Framework.
AWS Lambda functions and its configuration can be complicated but Serverless Framework helps to manage them simply.
Other methods
（Skip here for the main contents.）
As Lambda is one of the AWS services, you can control it via the AWS CLI. Also, you can write shell scripts that use it.
If you want to go with the AWS Way, you will be using AWS CodePipeline, AWS CodeBuild, AWS CloudFormation, as this page explains.
Still, there wouldn’t be people saying “This seems easy. Let’s try.”
So I searched and found tools to make the process easier. They are called Lamvery and Apex(the maker of Lamvery rivaled Apex but gave up at the end).
I tried both. Still, those two had some problems such as less flexibility or lack of some features.
What is Serverless Framework
Serverless Framework (I will say “Serverless” after here) allows you to manage AWS Lambda, Azure and GCP easily via CLI.
I often use AWS, but I see many customers use Azure, so Serverless is helpful in that it allows me to manage various infrastructures in the same way.
Unless I don’t specify, the topics from here are limited about AWS.
What you can do
You can manage the settings below with the configuration in YAML.
- Configurations of Lambda functions (you need to write code for function body).
- Triggers of AWS Lambda Functions
- Resources for AWS Lambda Functions
- S3
- DynamoDB
- SNS Topic
- Other
- IAM role
Serverless uses CloudFormation under the hood.
Since creating and updating the Stack is transparent, you don’t have to be aware of them.
Install Serverless
Installation
You need npm installed. Installation guide of npm is below.
Hit npm command to install .
npm install -g serverless
Create IAM group
Permission is required to deploy AWS Lambda Functions or to create and manage resources with Serverless.
I created an exclusive IAM group and put the necessary permissions in it. Regarding what’s necessary, there are various discussions in this issue, so you can refer to this if you want to implement fine graind permissions.
Below is what I did for one of our projects.
I attached the following AWS managed policies to the group:
- AWSLambdaFullAccess
- AmazonS3FullAccess
- AWSCloudFormationReadOnlyAccess
And created custom policies that allow the following actions, then attached them to the group:
- cloudformation:ValidateTemplate
- cloudformation:Describe*
- cloudformation:List*
- cloudformation:Get*
- cloudformation:PreviewStackUpdate
- cloudformation:CreateStack
- cloudformation:DeleteStack
- cloudformation:UpdateStack
After this steps, add IAM users in the IAM group.
Install and set up the AWS CLI
Other than this, you have to install and configure AWS CLI in your local developing machine. For the detail, refer this page.
Definition of Service
What is “Service”
The Serverless documentation says it is like a “Project”. At first, you can create one Service without thinking about the details. Then after you get used to it, you can divide it into different services as necessary.
Structure of a Service
The minimum settings are …
- myService/ directory
- serverless.yml
- Lambda Functions program
You have to specify necessary settings on serverless.yml.
Create Service
Hit the command below to create Service.
sls create --template --path myService
This time, I used “aws-python3” as a template. For other templates, refer this page.
Various settings
Here, I will explain about various configurations in serverless.yml.
I wrote already what you can manage by Serverless(see below). I will explain about each one.
- Configuration of Lambda Functions(Program as usual in the contents of Functions )
- Triggers of AWS Lambda Functions
- Every resource that Lambda Functions use
- IAM Role
All available properties for serverless.yml are covered in [this page].(https://serverless.com/framework/docs/providers/aws/guide/serverless.yml/)
Please refer this page for better understanding.
General settings
Specify general information in the service and provider section.
service:
name: myService
provider:
name: aws
runtime: python3.6
region: ap-northeast-1
role: arn:aws:iam::nnnnnnnnnnnn:role/Foo_role
stage: ${opt:stage, 'dev'}
deploymentBucket:
# Packages created by Serverless will be placed in this bucket.
# If you don’t specify the name, the bucket can be automatically generated with a random name. (Not sure）
name: my-project-lambda-deployment
serverSideEncryption: AES256 # when using server-side encryption
“stage” is one of the categories such as development, staging, and production. I will describe it later.
Function: Settings of AWS Lambda Functions
Settings of Lambda Functions have to be written in function section. So here, I define only function name and other settings. I write actual program code in the other file.
functions:
my_awesome_function: # key
handler: handler.my_awesome_function
name: my_awesome_function # name
events:
- xxx
I will explain a few points here.
About handler section, I mentioned that “write actual program code in the other file”, but the function specified on hander section will be called.
For example, I specified “my_awesome_function” in the handler section. In this case, You need to create the handler.py file and define a function name my_awesome_function in the file. The filename doesn’t have to be “hander.extention”.
You don’t have to write anything in the name section. In such case, key value will be used. Still, I recommend you to specify the name.
On the reference page of serverless.yaml, as an example, “name” includes Stage name, 「${self:provider.stage}-lambdaName」. As we try in this way – Stage name include Function name, we faced problems. Instead, we decided to use the same name as a key name which I will explain later.
I will explain the events in the next chapter.
Event: Triggers of Lambda Functions
Triggers of Lambda functions must be defined in the events sub-section in the function section. Serverless allows you to configure Lambda Functions to be triggered by events from various AWS services.
For the details, refer to serverless.yml reference page.
In our project, uploading files to S3 is the trigger. Here is one thing to be considered. Because S3 bucket is under the management of Serverless, even if you specify “S3” in the event section, you cannot use existing S3 bucket.
This seems to be by design (for the detail, refer to this issue) but to avoid this inconvenience, there are plugins to enable existing S3 bucket. I will write about this later.
Resource: Resources for Lambda Functions
You have to specify resources for AWS Lambda Functions in resources section such as S3 bucket and Dynamo DB. Here, you may need to follow the syntax of CloudFormation Template. Please refer to AWS reference.
AWS Resource Types Reference – AWS CloudFormation
Edit Lambda Functions
Just edit program then mostly finished.
# handler.py
# Just execute “import “
import boto3
# pip install required
import requests
# Lambda Functions
def my_awesome_function(event, context):
greet("hello world!")
# You can also define the different Lambda Functions in the same file.
def my_great_function(event, context):
greet("hello world!!!")
# It is better to create Functions for common use.
def greet(str)
print(str)
You can use libraries with “import” statement. Some libraries may not installed. In such cases, you need to execute “pip install”.
Packaging and Deployment
Install the necessary python package
As I mentioned before, you need to install non-default Python packages by executing “pip install”. Note that they need to be located in the service directory or one of its subdirectories. The easiest way is to place it in the same directory as handler.py. Below is the command:
This is how to write.
cd /path/to/myService
pip install (package name) -t.
Postscript: I posted a better way as a separate article.
Serverless で Python のパッケージを使った Lambda 関数をデプロイ | もばらぶエンジニアブログ
Deploy
This is how to deploy.
sls deploy
Hit this command to create packages of Python program and configuration, and they will be uploaded to S3 bucket. Then it is deployed through CloudFormation.
Other topics
Since the post is getting longer, I will post other topics below in a different article.
- Variables on serverless.yml
- About stage
- serverless-external-s3-event plugin
- Other、tips etc
A postscript. I post the second article.
Serverless Framework による AWS Lambda 関数の管理2 | もばらぶエンジニアブログ
Conclusion
Serverless Framework allows you to manage AWS Lambda Function and other resources with the simple configuration and program code. I recommend this way because it is so much easier compared to the official(?)AWS method.
About the image on the top
The image on the top of the page is licensed under the MIT license.
