redis之lua脚本: 原子性 & 调试 & 嵌入高级语言

实验环境:
redis: 6.0.9

redis执行lua脚本时, 出错不会回滚(rollback)

我们知道, 使用lua脚本可以在执行一串redis命令时, 实现一定原子性(lua脚本中多条指令执行过程中不会被插入新的指令), 但是并不能在命令执行出错时回滚之前的结果, 如下示例:

demo.lua

redis.call('get', 'xx')

redis.call('set', 'a1', 'b1')

redis.call('set', 'a2')

显然最后的set a2是有语法错误的, 在执行前先确认下数据库是空的:

127.0.0.1:6379> keys *
(empty array)
127.0.0.1:6379> 

然后执行下demo.lua(测试用的redis-server密码是123456):

$ redis-cli  -a '123456' --eval demo.lua
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
(error) ERR Error running script (call to f_c49b51d8e861ccfd226dedb5ed80d1bd3c257235): @user_script:7: @user_script: 7: Wrong number of args calling Redis command From Lua script

上述demo.lua最后一条命令执行报错了, 我们看下数据库, 发现set a1 b1还是写进去了, 并没有回退

127.0.0.1:6379> keys *
1) "a1"
127.0.0.1:6379> get a1
"b1"
127.0.0.1:6379> 

小结:
redis在执行lua脚本在执行出错(往往是语法错误: 如参数个数不对, 类型不对)时, 已经执行的结果不会回滚.

调试redis lua脚本

为了避免一些低级的错误, 我们在使用编程语言(如go, java)中利用lua脚本实现一些较复杂的功能时, 需要对lua脚本先进行调试. redis-cli工具自带了调试功能. 用一个例子来说明:

简单介绍下需求:
红包活动, 业务上需要控制红包个数(每个红包多少钱由业务上控制, 脚本不用管).
保证不能发超, 且每人最多只能领一次红包

测试用的脚本 test.lua:

-- 剩余红包个数
local rpRemainedKey = KEYS[1];

-- 已领红包 hash
local userAwardedKey = KEYS[2];

-- 领取红包的用户id
local userID = ARGV[1];

-- 领取红包实际多少钱
local money = ARGV[2];

-- 判断用户是否已经领过红包了
local userAwarded = redis.call("HGET", userAwardedKey, userID);
if userAwarded then
  return -1;
end

-- 判断红包剩余数
local remained = redis.call("GET", rpRemainedKey);
local iRemained = tonumber(remained);
if iRemained <= 0 then
  return -2;
end

-- 红包数 减一
iRemained = redis.call("DECR", rpRemainedKey);

redis.call("HSET", userAwardedKey, userID, money);

-- 返回还剩余多少个红包
return iRemained

简单调试

请注意redis-cli eval时的格式: KEYS 和 ARGV是通过","来分隔的

$ redis-cli  -a '123456' --eval test.lua rp-remained user-awarded , zhangsan 88

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
(error) ERR Error running script (call to f_359ab1f8ebcc70ab7f608a16e245baecf9b6bcdf): @user_script:22: user_script:22: attempt to compare nil with number

报错说是: 22行的if iRemained <= 0 then 将nil和number比较, 说明GET rp-remained是nil, 也就是没有在redis-server中设置红包个数, 通过SET rp-remained 5设置5个红包再执行上述命令:

$ redis-cli  -a '123456' --eval test.lua rp-remained user-awarded , zhangsan 88
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
(integer) 4

说明发红包成功, 目前红包剩余4个.

再执行一次, 发现用户已领红包, 直接返回-1

$ redis-cli  -a '123456' --eval test.lua rp-remained user-awarded , zhangsan 88
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
(integer) -1

使用lua debugger来调试

关于redis使用lua debugger的更多信息, 看这里: Redis Lua scripts debugger

注意下面的命令加了–ldb选项

我们这里来演示下:

$ redis-cli  -a '123456' --ldb --eval test.lua rp-remained user-awarded , zhangsan 88
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
Lua debugging session started, please use:
quit    -- End the session.
restart -- Restart the script in debug mode again.
help    -- Show Lua script debugging commands.

* Stopped at 2, stop reason = step over
-> 2   local rpRemainedKey = KEYS[1];
lua debugger> s
* Stopped at 5, stop reason = step over
-> 5   local userAwardedKey = KEYS[2];
lua debugger> s
* Stopped at 8, stop reason = step over
-> 8   local userID = ARGV[1];
lua debugger> s
* Stopped at 11, stop reason = step over
-> 11  local money = ARGV[2];
lua debugger> s
* Stopped at 14, stop reason = step over
-> 14  local userAwarded = redis.call("HGET", userAwardedKey, userID);
lua debugger> p money
<value> "88"
lua debugger> s
<redis> HGET user-awarded zhangsan
<reply> "88"
* Stopped at 15, stop reason = step over
-> 15  if userAwarded then
lua debugger> p userAwarded
<value> "88"
lua debugger> s
* Stopped at 16, stop reason = step over
-> 16    return -1;
lua debugger> s

(integer) -1

(Lua debugging session ended -- dataset changes rolled back)

127.0.0.1:6379> 

熟悉gdb或者dlv的同学应该对上述调试界面感觉非常亲切, 这里直接体验下效果更好.
更多调试指令, 通过在 lua debugger>下输入help来查看.

关于lua语言的false值

先说下结论: lua中只有false和nil是假值, 其他都是真值, 测试如下:

test.lua:

function judge(v)
  if v then
    print(v, "is true value")
  else
    print(v, "is false value")
  end
end

judge(0)
judge(1)
judge(-1)
judge(nil)
judge(false)
judge(true)
judge({})
judge("")
judge("abc")

运行看看:

$ lua test.lua
0       is true value
1       is true value
-1      is true value
nil     is false value
false   is false value
true    is true value
table: 0x87c420 is true value
        is true value
abc     is true value

嵌入高级语言

上述脚本我们在自己开发机上调试好了, 最终还是要用其他语言来调的, 这里以go语言为例子:
跑go代码之前先将数据复原:

127.0.0.1:6379> keys *
1) "rp-remained"
127.0.0.1:6379> get "rp-remained"
"5"

main.go:

package main

import (
	"context"
	"fmt"
	"log"
	"sync"

	"github.com/go-redis/redis/v8"
)

var (
	ctx = context.Background()
	rc  *redis.Client
)

const rpLuaScript = `
-- 剩余红包个数
local rpRemainedKey = KEYS[1];

-- 已领红包 hash
local userAwardedKey = KEYS[2];

-- 领取红包的用户id
local userID = ARGV[1];

-- 领取红包实际多少钱
local money = ARGV[2];

-- 判断用户是否已经领过红包了
local userAwarded = redis.call("HGET", userAwardedKey, userID);
if userAwarded then
  return -1;
end

-- 判断红包剩余数
local remained = redis.call("GET", rpRemainedKey);
local iRemained = tonumber(remained);
if iRemained <= 0 then
  return -2;
end

-- 红包数 减一
iRemained = redis.call("DECR", rpRemainedKey);

redis.call("HSET", userAwardedKey, userID, money);


-- 返回还剩余多少个红包
return iRemained
`

func main() {
	InitRedisCli()

	var wg sync.WaitGroup
	for i := 0; i < 10000; i++ {
		userID := fmt.Sprintf("zhangsan:%d", i%10) // zhangsan0 - zhangsan9 每个人分别请求100次, 共1000次
		money := i + 88
		wg.Add(1)
		go func() {
			defer wg.Done()
			cmd := rc.Eval(ctx, rpLuaScript, []string{"rp-remained", "user-awarded"}, userID, money)
			remained, err := cmd.Int() // 因为lua脚本中return的是int, 所以这里转为int
			if err != nil {
				log.Printf("userID:%v, money:%v, err:%\n", userID, money, err)
				return
			}
			if remained >= 0 {
				log.Printf("success: userID:%v, money:%v, after get rp, remained:%v\n", userID, money, remained)
			} else {
				log.Printf("failed: userID:%v, money:%v, after get rp, remained:%v\n", userID, money, remained)
			}
		}()
	}
	wg.Wait()
}

func InitRedisCli() {
	rc = redis.NewClient(&redis.Options{
		Addr:     "localhost:6379",
		Password: "123456", // no password set
	})

	if err := rc.Ping(ctx).Err(); err != nil {
		panic(err)
	}
}

实验步骤:

1. go run main.go 2>&1 | tee main.log
2. 查看redis中红包领取情况: 红包刚好被领完, 领红包的共5个人, 符合预期.

   127.0.0.1:6379> get "rp-remained"
   "0"
   127.0.0.1:6379> hgetall "user-awarded"
    1) "zhangsan:4"
    2) "92"
    3) "zhangsan:5"
    4) "93"
    5) "zhangsan:7"
    6) "115"
    7) "zhangsan:9"
    8) "137"
    9) "zhangsan:2"
   10) "90"
   127.0.0.1:6379> 
   

3. 查看go代码日志中获奖的人: 和redis中的数据刚好对得上, 符合预期

   $ grep success main.log
   2022/01/26 16:46:33 success: userID:zhangsan:4, money:92, after get rp, remained:4
   2022/01/26 16:46:33 success: userID:zhangsan:5, money:93, after get rp, remained:3
   2022/01/26 16:46:33 success: userID:zhangsan:7, money:115, after get rp, remained:2
   2022/01/26 16:46:33 success: userID:zhangsan:9, money:137, after get rp, remained:1
   2022/01/26 16:46:33 success: userID:zhangsan:2, money:90, after get rp, remained:0
   

总结

1. redis lua脚本执行过程中, 出错不会回滚
2. redis-cli --eval时, 再加上–ldb即可调试lua脚本
3. 使用高级语言(如go)调用lua脚本来操作redis时, 注意返回值的类型, 具体参看示例代码

参考:

1. https://redis.io/topics/ldb
2. https://redis.io/topics/transactions