AWS Lambda函数从kinesis流中无限读取记录

Question

我有一个kinesis流，有一个碎片和一个用Python编写的lambda函数。我添加了kinesis流作为批量大小为5的事件源。我在kinesis中添加了数百条记录，lambda函数得到了正确的调用和执行。但是对于最后3条记录，lambda函数被无限地调用，即使函数返回是成功的。

from __future__ import print_function

import base64
import json
import urllib2
import json

print('Loading function')

def is_valid_url(url):
    try:
        urllib2.urlopen(url)
        print("Valid URL ...")
        return True         # URL Exist
    except ValueError, ex:
        print("URL is not formatted...")
        return False        # URL not well formatted
    except urllib2.URLError, ex:
        print("Invalid URL ...")
        return False        # URL don't seem to be alive


def lambda_handler(event, context):     
    for record in event['Records']:
        # Kinesis data is base64 encoded so decode here
        payload = base64.b64decode(record['kinesis']['data'])
        params = json.loads(payload)
        print("Decoded payload: " + payload  + " : " +     str(is_valid_url(params['url'])) + " : " + str(len(event['Records'])))
    return 'Successfully processed {} records.'.format(len(event['Records']))

  START RequestId: d6033244-1c43-40ea-8886-f38b8c48daa3 Version:   $LATEST 
  Loading function 
  Valid URL ... 
  Decoded payload: { "url": "https://google.com" }
  Valid URL ... 
  Decoded payload: { "url": "https://google.com" }
  Valid URL ... 
  Decoded payload: { "url": "https://google.com" }
  Valid URL ...  
  Decoded payload: { "url": "https://google.com" }
  END RequestId: d6033244-1c43-40ea-8886-f38b8c48daa3 
  REPORT RequestId: d6033244-1c43-40ea-8886-f38b8c48daa3    Duration: 3003.00 ms    Billed Duration: 3000 ms Memory Size: 128 MB    Max Memory Used: 10 MB   
  2016-03-04T17:32:01.030Z d6033244-1c43-40ea-8886-f38b8c48daa3 Task timed out after 3.00 seconds

Answer 1

因为您的函数正在超时，Lambda将运行视为错误。Kinesis的错误处理策略是重试记录，直到它脱离裁剪范围（通常是24小时），因此您的函数将重试24小时，或者直到它没有超时。

根据你发布的内容，我不知道为什么你的函数会超时。一个快速的解决办法是简单地增加Lambda控制台上的超时值（在配置选项卡的高级下）