An AWS Lambda Function to resize/reduce images automatically. When an image isput on AWS S3 bucket, this package will resize/reduce it and put to S3.
From nodejs10.x
, AWS Lambda doesn't bundle ImageMagick
and image related libraries.
https://forums.aws.amazon.com/thread.jspa?messageID=906619&tstart=0
Therefore, if you'd deploy with nodejs10.x
runtime (but we prefer and default as it), it needs to install AWS Lambda Layer with this function.This project can support it automatically, see LAYERS in detail.
Clone this repository and install dependencies:
git clone git@github.com:ysugimoto/aws-lambda-image.git
cd aws-lambda-image
npm install .
When upload to AWS Lambda, the project will bundle only needed files - no devdependencies will be included.
Configuration file you will find under the name config.json
in project root.It's copy of our example file config.json.sample
. More or less it looks like:
{
"bucket": "your-destination-bucket",
"backup": {
"directory": "./original"
},
"reduce": {
"directory": "./reduced",
"prefix": "reduced-",
"quality": 90,
"acl": "public-read",
"cacheControl": "public, max-age=31536000"
},
"resizes": [
{
"size": 300,
"directory": "./resized/small",
"prefix": "resized-",
"cacheControl": null
},
{
"size": 450,
"directory": "./resized/medium",
"suffix": "_medium"
},
{
"size": "600x600^",
"gravity": "Center",
"crop": "600x600",
"directory": "./resized/cropped-to-square"
},
{
"size": 600,
"directory": "./resized/600-jpeg",
"format": "jpg",
"background": "white"
},
{
"size": 900,
"directory": "./resized/large",
"quality": 90
}
]
}
name | field | type | description |
---|---|---|---|
bucket | - | String | Destination bucket name at S3 to put processed image. If not supplied, it will use same bucket of event source. |
jpegOptimizer | - | String | Determine optimiser that should be used mozjpeg (default) or jpegoptim ( only JPG ). |
acl | - | String | Permission of S3 object. See AWS ACL documentation. |
cacheControl | - | String | Cache-Control of S3 object. If not specified, defaults to original image's Cache-Control. |
keepExtension | - | Boolean | Global setting fo keeping original extension. If true , program keeps orignal file extension. otherwise use strict extension eg JPG,jpeg -> jpg |
backup | - | Object | Backup original file setting. |
bucket | String | Destination bucket to override. If not supplied, it will use bucket setting. |
|
directory | String | Image directory path. Supports relative and absolute paths. Mode details in DIRECTORY.md | |
template | Object | Map representing pattern substitution pair. Mode details in DIRECTORY.md | |
prefix | String | Prepend filename prefix if supplied. | |
suffix | String | Append filename suffix if supplied. | |
acl | String | Permission of S3 object. See AWS ACL documentation. | |
cacheControl | String | Cache-Control of S3 object. If not specified, defaults to original image's Cache-Control. | |
keepExtension | Boolean | If true , program keeps orignal file extension. otherwise, use strict extension eg JPG,jpeg -> jpg |
|
move | Boolean | If true , an original uploaded file will delete from Bucket after completion. |
|
reduce | - | Object | Reduce setting following fields. |
quality | Number | Determine reduced image quality ( only JPG ). |
|
jpegOptimizer | String | Determine optimiser that should be used mozjpeg (default) or jpegoptim ( only JPG ). |
|
bucket | String | Destination bucket to override. If not supplied, it will use bucket setting. |
|
directory | String | Image directory path. Supports relative and absolute paths. Mode details in DIRECTORY.md | |
template | Object | Map representing pattern substitution pair. Mode details in DIRECTORY.md | |
prefix | String | Prepend filename prefix if supplied. | |
suffix | String | Append filename suffix if supplied. | |
acl | String | Permission of S3 object. See AWS ACL documentation. | |
cacheControl | String | Cache-Control of S3 object. If not specified, defaults to original image's Cache-Control. | |
keepExtension | Boolean | If true , program keeps orignal file extension. otherwise, use strict extension eg JPG,jpeg -> jpg |
|
resize | - | Array | Resize setting list of following fields. |
size | String | Image dimensions. See ImageMagick geometry documentation. | |
format | String | Image format override. If not supplied, it will leave the image in original format. | |
crop | String | Dimensions to crop the image. See ImageMagick crop documentation. | |
gravity | String | Changes how size and crop . See ImageMagick gravity documentation. |
|
quality | Number | Determine reduced image quality ( forces format JPG ). |
|
jpegOptimizer | String | Determine optimiser that should be used mozjpeg (default) or jpegoptim ( only JPG ). |
|
orientation | Boolean | Auto orientation if value is true . |
|
bucket | String | Destination bucket to override. If not supplied, it will use bucket setting. |
|
directory | String | Image directory path. Supports relative and absolute paths. Mode details in DIRECTORY.md | |
template | Object | Map representing pattern substitution pair. Mode details in DIRECTORY.md | |
prefix | String | Prepend filename prefix if supplied. | |
suffix | String | Append filename suffix if supplied. | |
acl | String | Permission of S3 object. See AWS ACL documentation. | |
cacheControl | String | Cache-Control of S3 object. If not specified, defaults to original image's Cache-Control. | |
keepExtension | Boolean | If true , program keeps orignal file extension. otherwise, use strict extension eg JPG,jpeg -> jpg |
|
optimizers | - | Object | Definitions for override the each Optimizers command arguments. |
pngquant | Array | Pngquant command arguments. Default is ["--speed=1", "256"] . |
|
jpegoptim | Array | Jpegoptim command arguments. Default is ["-s", "--all-progressive"] . |
|
mozjpeg | Array | Mozjpeg command arguments. Default is ["-optimize", "-progressive"] . |
|
gifsicle | Array | Gifsicle command arguments. Default is ["--optimize"] . |
Note that the optmizers
option will force override its command arguments, so if you define these configurations, we don't care any more about how optimizer works.
If you want to check how your configuration will work, you can use:
npm run test-config
To use the automated deployment scripts you will need to haveaws-cli installed and configured.
Deployment scripts are pre-configured to use some default values for the Lambdaconfiguration. I you want to change any of those just use:
npm config set aws-lambda-image:profile default
npm config set aws-lambda-image:region eu-west-1
npm config set aws-lambda-image:memory 1280
npm config set aws-lambda-image:timeout 5
npm config set aws-lambda-image:name lambda-function-name
npm config set aws-lambda-image:role lambda-execution-role
Note that aws-lambda-image:name
and aws-lambda-image:role
are optional.If you want to change lambda function name or execution role, type above commands before deploy.
And make sure AWS Lambda Layer has installed in your account/region.See LAYERS for instructions.
Command below will deploy the Lambda function on AWS, together with setting uproles and policies.
npm run deploy
Notice: Because there are some limitations in Claudia.js
support forpolicies, which could lead to issues with Access Denied
when processingimages from one bucket and saving them to another, we have decided to introducesupport for custom policies.
Policies which should be installed together with our Lambda function are storedin policies/
directory. We keep there policy that grants access to allbuckets, which is preventing possible errors with Access Denied
describedabove. If you have any security-related concerns, feel free to change the:
"Resource": [
"*"
]
in the policies/s3-bucket-full-access.json
to something more restrictive,like:
"Resource": [
"arn:aws:s3:::destination-bucket-name/*"
]
Just keep in mind, that you need to make those changes before you do thedeployment.
To complete installation process you will need to take one more action. It willallow you to install S3 Bucket event handler, which will send information aboutall uploaded images directly to your Lambda function.
npm run add-s3-handler --s3_bucket="your-bucket-name" --s3_prefix="directory/" --s3_suffix=".jpg"
You are able to install multiple handlers per Bucket. So, to add handler for PNGfiles you just need to re-run above command with different suffix, ie:
npm run add-s3-handler --s3_bucket="your-bucket-name" --s3_prefix="directory/" --s3_suffix=".png"
As an addition, you can also setup and SNS message handler in case you wouldlike to process S3 events over an SNS topic.
npm run add-sns-handler --sns_topic="arn:of:SNS:topic"
To update Lambda with you latest code just use command below. Script will buildnew package and automatically publish it on AWS.
npm run update
For more scripts look into package.json.
You can handle resize/reduce/backup process on success/error result onindex.js
. ImageProcessor::run
will return Promise
object, run youroriginal code:
processor.run(config)
.then(function(proceedImages)) {
// Success case:
// proceedImages is list of ImageData instance on you configuration
/* your code here */
// notify lambda
context.succeed("OK, numbers of " + proceedImages.length + " images has proceeded.");
})
.catch(function(messages) {
// Failed case:
// messages is list of string on error messages
/* your code here */
// notify lambda
context.fail("Woops, image process failed: " + messages);
});
ImageMagick
(installed on AWS Lambda)MIT License.
Yoshiaki Sugimoto
Thanks for testing fixture images:
import asyncio import time from concurrent.futures import ThreadPoolExecutor import threading from urllib import request from urllib.parse import urlparse import aiobotocore from aiohttp.client_reqre
aws lambda使用 by Daitan 通过大潭 我们通过使用AWS Lambda服务机器学习模型学到了什么 (What We Learned by Serving Machine Learning Models Using AWS Lambda) Moving machine learning (ML) models from training to serving in producti
因此,您在开发帐户上工作,并且Terraform陷入了一个循环,难道不让您轻易销毁剩余资源吗? 进入nuke CLI的世界! 在撰写本文时,我使用的是v0.1.16版 用Go语言编写的《 Gruntwork》不会破坏掉aws-nuke那样多的对象; 自2017年5月以来一直存在 如果您使用~/.aws/credentials ,请在~/.aws/credentials选择(明智地!)帐户别名(在我
本文来源: https://amazon.qwiklabs.com/focuses/10541?catalog_rank=%7B%22rank%22%3A1%2C%22num_filters%22%3A1%2C%22has_search%22%3Atrue%7D&parent=catalog&search_id=6895488 主要是其试验经过。 Task1:创建两个S3 命名分别为image
写在前面 使用aws lambda已经一年多了,下面使用java构建一个简单的lambda服务,大家可以自己扩展想要的功能,废话不多说,开始吧。 AWS 上 Java Lambda 应用记要 public class LambdaFunctionHandler implements RequestHandler<Object,Object > { int warmNum = 0;
文档:AWS Lambda foundations - AWS Lambda 1.简介 Lambda 函数是 Lambda 的基本。 可以使用 Lambda 控制台、Lambda API、AWS CloudFormation 或 AWS SAM 配置函数。 Lambda 在事件发生时调用该函数。 Lambda 并行运行您的函数的多个实例,并受并发和扩展限制的约束。 2.概念 Lambda 运行函数
Python 中的 AWS Lambda 函数日志记录 您的 Lambda 函数带有一个 CloudWatch Logs 日志组,其中包含您的函数的每个实例的日志流。运行时会将每个调用的详细信息发送到该日志流,然后中继日志和来自您的函数代码的其他输出。 要从函数代码输出日志,您可以使用 print 方法或使用写入到 stdout 或 stderr 的任何日志记录库。以下示例记录环境变量和事件对象的
为你用来运行的函数和存储的内容设置限额。限额可以保证每个区域的lambda可以有足够的资源去处理任务,也可以特殊申请额外的限额来实现更多的操作。 一般的: 并发执行数: 1000 代码以及库的存储容量: 75GB 每个虚拟私有云(VPC)的弹性网络接口数: 250 配置、部署和执行的限额: 函数的内存配置: 128MB-10240MB 函数超时时间: 900s 函数环境变量: 4KB 函数的资源配
是否有可能将AWS Lambda与Apache Kafka集成在一起?我想在lambda函数中放一个消费者。当使用者收到消息时,lambda函数执行。
我很难使用基于Java的Lambda函数设置来接收来自SNS的消息。我的函数如下所示: 它编译得很好,我将jar文件上传到Lambda(通过网络控制台)没有任何问题。 但是,当我使用代表SNSEvent模型的JSON发布到它(通过SNS发布到订阅的Lambda函数)时,Lambda函数抛出以下异常: 在类com上加载方法处理程序时出错。我的公司。LambdaHandler:java类。NoClas
在dynamoDB更新中调用lambda函数时,我遇到了问题。我已经检查了AWS Lambda:ClassNotFoundException和AWS Lambda NoClassDefFoundError,但没有成功。 我写这个lambda函数调用时,有任何更新的发电机。我按照这个教程。https://docs.aws.amazon.com/lambda/latest/dg/with-dynamo
Apollo AWS Lambda with GraphQL subscriptions ⚠️ This documentation is currently for 1.0.0-alpha.X package which supports only subscriptions-transport-ws and drops the legacy protocol and client suppor
playwright-aws-lambda Support for Playwright running on AWS Lambda and Google Cloud Functions. NOTE: Currently only Chromium is supported. Install npm install playwright-core playwright-aws-lambda --s
AWS Lambda C++ Runtime C++ implementation of the lambda runtime API Design Goals Negligible cold-start overhead (single digit millisecond). Freedom of choice in compilers, build platforms and C standa