SpringBoot2+Redis动态切换db数据源(db)最佳实践
平光明
动态DB切换代码已开源，有用star一下
https://github.com/it235/knife4j-redis-lettuce
需求
在使用Redis的时候，默认是16个库，非常小的项目默认0库就够了，但是对于体量稍微大一些的项目，需要将其他各个库充分利用，比如：
- db0存公用的热点缓存
- db1存商品服务的缓存
- db2存订单相关的缓存
- db3存库存相关的缓存
- …
这个时候我们就需要实现多库切换进行操作，接下来我们看看如何在SpringBoot中实现多DB的切换动作。
实现
lettuce默认实现(只能取1个db)
- 添加Maven依赖（版本默认采用当前parent的）
<dependencies> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-pool2</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-redis-reactive</artifactId> </dependency> </dependencies> - yml配置
spring: redis: host: 192.168.0.245 password: 123456 port: 6379 database: 1 # 使用库 1 timeout: 60s lettuce: # lettuce基于netty，线程安全，支持并发 pool: max-active: 8 max-wait: -1ms max-idle: 2 min-idle: 0 - Configuration配置
import com.alibaba.fastjson.parser.ParserConfig; import org.springframework.cache.annotation.EnableCaching; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.data.redis.connection.RedisConnectionFactory; import org.springframework.data.redis.core.RedisTemplate; import org.springframework.data.redis.serializer.StringRedisSerializer; @EnableCaching @Configuration public class FastJsonRedisConfig { @Bean public RedisTemplate<String,Object> redisTemplate(RedisConnectionFactory factory){ RedisTemplate<String,Object> template = new RedisTemplate <>(); template.setConnectionFactory(factory); Jackson2JsonRedisSerializer<Object> jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer<>(); ObjectMapper om = new ObjectMapper(); om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY); om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL); jackson2JsonRedisSerializer.setObjectMapper(om); StringRedisSerializer stringRedisSerializer = new StringRedisSerializer(); // key采用String的序列化方式 template.setKeySerializer(stringRedisSerializer); // hash的key也采用String的序列化方式 template.setHashKeySerializer(stringRedisSerializer); // value序列化方式采用jackson template.setValueSerializer(jackson2JsonRedisSerializer); // hash的value序列化方式采用jackson template.setHashValueSerializer(jackson2JsonRedisSerializer); template.afterPropertiesSet(); return template; } }
以上我们就可以注入restTemplate进行redis的相关操作了，这样做只能满足最基本的取db0的操作，如果要使用到其他库，接下来我们看第二种方案
动态切换DB
源码：https://github.com/it235/knife4j-redis-lettuce
思路
如果要动态切换DB，我们需要考虑以下几个问题：
- DB个数该如何指定？比如项目只采用0-5库
- DB切换时直接用一个
redisTemplate是否可以？ - DB1切换到DB2后，其他线程如果要继续使用DB1该如何选取？
- 工具类该如何封装？
构思：接下来我构思来解决上面几个问题
- DB个数指定，我们能否通过yml配置文件指定个数，比如：[0,1,2,3,4,5,6]这样
- DB切换时
redisTemplate一个显然可以，代码如下，但不是特别合适，因为database的指定是在connectionFactory.setDatabase(num);这里完成，所以我们需要操作连接工厂，通过最上面实现的代码可以看到，factory是由redisTemplate去指定的template.setConnectionFactory(factory);，如果我们采用一个redisTemplate的方式，那么我们要不停的进行resetConnection的方式。//不推荐使用该方式 @Autowired private StringRedisTemplate redisTemplate; public void setDataBase(int num) { LettuceConnectionFactory connectionFactory = (LettuceConnectionFactory) redisTemplate.getConnectionFactory(); if (connectionFactory != null && num != connectionFactory.getDatabase()) { //切换DB connectionFactory.setDatabase(num); //是否允许多个线程操作共用同一个缓存连接，默认 true，false 时每个操作都将开辟新的连接 connectionFactory.setShareNativeConnection(false); this.redisTemplate.setConnectionFactory(connectionFactory); connectionFactory.resetConnection(); } }
这里我们最合适的方式，就是为每个库建立一个单独的redisTemplate，然后通过一个Map<dbnum , redisTemplate>维护，需要时告诉我具体的dbNum，我来根据你想要的从池子中取出进行操作即可，这样也不会涉及到db的频繁切换
多redisTemplate难点
- 如何通过factory创建多个？
- 如何维护Bean的生命周期？yml如何规划
- redis工具类该如何定义？
动手实现
- 第一步，定义理想中的yml
# 此处Key由spring改为自己定义的 knife4j: redis: # 是否采用json序列化方式，若不采用jackson序列化 jsonSerialType: 'Fastjson' host: localhost password: knife port: 6379 databases: [0,1,2,3,4,5,6] # 要使用的库，会根据此处填写的库生成redisTemplate timeout: 60s lettuce: # lettuce基于netty，线程安全，支持并发 pool: max-active: 50 max-wait: -1ms max-idle: 8 min-idle: 0 - 第二步，读取yml，并创建
redisTemplate注入到Spring容器中package com.github.it235.register; /** * @description: 根据yml创建redisTemplate并注入到Spring容器 * @author: www.it235.com * @date: Created in 2020/9/27 15:27 */ public class Knife4jRedisRegister implements EnvironmentAware, ImportBeanDefinitionRegistrar { private static final Logger logger = LoggerFactory.getLogger(Knife4jRedisRegister.class); private static Map<String, Object> registerBean = new ConcurrentHashMap<>(); private Environment environment; private Binder binder; @Override public void setEnvironment(Environment environment) { this.environment = environment; this.binder = Binder.get(this.environment); } @Override public void registerBeanDefinitions(AnnotationMetadata importingClassMetadata, BeanDefinitionRegistry beanDefinitionRegistry) { RedisEntity redisEntity; try { redisEntity = binder.bind("knife4j.redis", RedisEntity.class).get(); } catch (NoSuchElementException e) { logger.error("Failed to configure knife4j redis: 'knife4j.redis' attribute is not specified and no embedded redis could be configured."); return; } boolean onPrimary = true; //根据多个库实例化出多个连接池和Template List<Integer> databases = redisEntity.getDatabases(); for (Integer database : databases) { //单机模式 RedisStandaloneConfiguration configuration = new RedisStandaloneConfiguration(); configuration.setHostName(String.valueOf(redisEntity.getHost())); configuration.setPort(Integer.parseInt(String.valueOf(redisEntity.getPort()))); configuration.setDatabase(database); String password = redisEntity.getPassword(); if (password != null && !"".equals(password)) { RedisPassword redisPassword = RedisPassword.of(password); configuration.setPassword(redisPassword); } //池配置 GenericObjectPoolConfig genericObjectPoolConfig = new GenericObjectPoolConfig(); RedisProperties.Pool pool = redisEntity.getLettuce().getPool(); genericObjectPoolConfig.setMaxIdle(pool.getMaxIdle()); genericObjectPoolConfig.setMaxTotal(pool.getMaxActive()); genericObjectPoolConfig.setMinIdle(pool.getMinIdle()); if (pool.getMaxWait() != null) { genericObjectPoolConfig.setMaxWaitMillis(pool.getMaxWait().toMillis()); } Supplier<LettuceConnectionFactory> lettuceConnectionFactorySupplier = () -> { LettuceConnectionFactory factory = (LettuceConnectionFactory) registerBean.get("LettuceConnectionFactory" + database); if (factory != null) { return factory; } LettucePoolingClientConfiguration.LettucePoolingClientConfigurationBuilder builder = LettucePoolingClientConfiguration.builder(); Duration shutdownTimeout = redisEntity.getLettuce().getShutdownTimeout(); if(shutdownTimeout == null){ shutdownTimeout = binder.bind("knife4j.redis.shutdown-timeout", Duration.class).get(); } if (shutdownTimeout != null) { builder.shutdownTimeout(shutdownTimeout); } LettuceClientConfiguration clientConfiguration = builder.poolConfig(genericObjectPoolConfig).build(); factory = new LettuceConnectionFactory(configuration, clientConfiguration); registerBean.put("LettuceConnectionFactory" + database, factory); return factory; }; LettuceConnectionFactory lettuceConnectionFactory = lettuceConnectionFactorySupplier.get(); BeanDefinitionBuilder builder = BeanDefinitionBuilder.genericBeanDefinition(LettuceConnectionFactory.class, lettuceConnectionFactorySupplier); AbstractBeanDefinition factoryBean = builder.getRawBeanDefinition(); factoryBean.setPrimary(onPrimary); beanDefinitionRegistry.registerBeanDefinition("lettuceConnectionFactory" + database, factoryBean); // StringRedisTemplate GenericBeanDefinition stringRedisTemplate = new GenericBeanDefinition(); stringRedisTemplate.setBeanClass(StringRedisTemplate.class); ConstructorArgumentValues constructorArgumentValues = new ConstructorArgumentValues(); constructorArgumentValues.addIndexedArgumentValue(0, lettuceConnectionFactory); stringRedisTemplate.setConstructorArgumentValues(constructorArgumentValues); stringRedisTemplate.setAutowireMode(AutowireCapableBeanFactory.AUTOWIRE_BY_NAME); beanDefinitionRegistry.registerBeanDefinition("stringRedisTemplate" + database, stringRedisTemplate); // 定义RedisTemplate对象 GenericBeanDefinition redisTemplate = new GenericBeanDefinition(); redisTemplate.setBeanClass(RedisTemplate.class); redisTemplate.getPropertyValues().add("connectionFactory", lettuceConnectionFactory); redisTemplate.setAutowireMode(AutowireCapableBeanFactory.AUTOWIRE_BY_NAME); JsonSerialType jsonSerialType = redisEntity.getJsonSerialType(); RedisSerializer stringRedisSerializer = null; Jackson2JsonRedisSerializer<Object> jackson2JsonRedisSerializer = null; if(jsonSerialType != null && jsonSerialType == JsonSerialType.Fastjson){ //若配置，则采用该方式 jackson2JsonRedisSerializer = new CustomFastJsonRedisSerializer<>(Object.class); ParserConfig.getGlobalInstance().setAutoTypeSupport(false); stringRedisSerializer = new CustomStringRedisSerializer(); } else { // 内置默认序列化(此处若不设置则采用默认的JDK设置，也可以在使用使自定义序列化方式) jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer<>(Object.class); ObjectMapper om = new ObjectMapper(); om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY); om.activateDefaultTyping(om.getPolymorphicTypeValidator(), ObjectMapper.DefaultTyping.NON_FINAL); jackson2JsonRedisSerializer.setObjectMapper(om); stringRedisSerializer = new StringRedisSerializer(); } // key采用String的序列化方式，value采用json序列化方式 redisTemplate.getPropertyValues().add("keySerializer",stringRedisSerializer); redisTemplate.getPropertyValues().add("hashKeySerializer",stringRedisSerializer); redisTemplate.getPropertyValues().add("valueSerializer",jackson2JsonRedisSerializer); redisTemplate.getPropertyValues().add("hashValueSerializer",jackson2JsonRedisSerializer); //注册Bean beanDefinitionRegistry.registerBeanDefinition("redisTemplate" + database, redisTemplate); logger.info("Registration redis ({}) !", database); if (onPrimary) { onPrimary = false; } } } } - 第三步，创建
redisManager进行管理redisTemplate
/**
* @description: 给工具类提供manager，由先的configuration进行初始化和赋值
* @author: www.it235.com
* @date: Created in 2020/9/27 15:27
*/
public class Knife4jRedisManager {
private Map<String, RedisTemplate> redisTemplateMap;
private Map<String, StringRedisTemplate> stringRedisTemplateMap;
public Knife4jRedisManager(Map<String, RedisTemplate> redisTemplateMap ,
Map<String, StringRedisTemplate> stringRedisTemplateMap) {
this.redisTemplateMap = redisTemplateMap;
this.stringRedisTemplateMap = stringRedisTemplateMap;
}
public RedisTemplate redisTemplate(int dbIndex) {
RedisTemplate redisTemplate = redisTemplateMap.get("redisTemplate" + dbIndex);
return redisTemplate;
}
public StringRedisTemplate stringRedisTemplate(int dbIndex) {
StringRedisTemplate stringRedisTemplate = stringRedisTemplateMap.get("stringRedisTemplate" + dbIndex);
stringRedisTemplate.setEnableTransactionSupport(true);
return stringRedisTemplate;
}
public Map<String, RedisTemplate> getRedisTemplateMap() {
return redisTemplateMap;
}
public Map<String, StringRedisTemplate> getStringRedisTemplateMap() {
return stringRedisTemplateMap;
}
}
- 第四部，创建
Configuration，管理redisTemplate，加载redisManager
/**
* @description: 核心配置，取出spring中的redisTemplate暂存到map中，并初始化redisManager
* @author: www.it235.com
* @date: Created in 2020/9/27 15:27
*/
@AutoConfigureBefore({RedisAutoConfiguration.class})
@Import(Knife4jRedisRegister.class)
@EnableCaching
@Configuration
public class Knife4jRedisConfiguration implements EnvironmentAware , ApplicationContextAware {
private static final Logger logger = LoggerFactory.getLogger(Knife4jRedisConfiguration.class);
private static String key1 = "redisTemplate";
private static String key2 = "stringRedisTemplate";
Map<String, RedisTemplate> redisTemplateMap = new HashMap<>();
Map<String, StringRedisTemplate> stringRedisTemplateMap = new HashMap<>();
private Binder binder;
private Environment environment;
@Override
public void setEnvironment(Environment environment) {
this.environment = environment;
this.binder = Binder.get(this.environment);
}
@PostConstruct
public Map<String,RedisTemplate> initRedisTemplate(){
RedisEntity redisEntity;
try {
redisEntity = binder.bind("knife4j.redis", RedisEntity.class).get();
} catch (NoSuchElementException e) {
throw new RuntimeException("Failed to configure knife4j redis: 'knife4j.redis' attribute is not specified and no embedded redis could be configured.");
}
//根据多个库实例化出多个连接池和Template
List<Integer> databases = redisEntity.getDatabases();
if(databases == null || databases.size() == 0){
logger.warn("no config property knife4j.redis.databases , default use db0！！！");
databases.add(0);
}
//根据指定的数据库个数来加载对应的RedisTemplate
for (Integer database : databases) {
String key = key1 + database;
RedisTemplate redisTemplate = applicationContext.getBean(key , RedisTemplate.class);
if(redisTemplate != null){
redisTemplateMap.put(key , redisTemplate);
}
key = key2 + database;
if(stringRedisTemplateMap != null){
StringRedisTemplate stringRedisTemplate = applicationContext.getBean(key , StringRedisTemplate.class);
stringRedisTemplateMap.put(key , stringRedisTemplate);
}
}
if(redisTemplateMap.size() == 0 && stringRedisTemplateMap.size() == 0){
throw new RuntimeException("load redisTemplate failure , please check knife4j.redis property config！！！");
}
return redisTemplateMap;
}
private ApplicationContext applicationContext;
@Override
public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
this.applicationContext = applicationContext;
}
@Bean
public Knife4jRedisManager knife4jRedisManager(){
return new Knife4jRedisManager(redisTemplateMap , stringRedisTemplateMap);
@Bean
public RedisBaseUtil redisBaseUtil(){
return new RedisBaseUtil();
}
}
spring.factories配置
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
com.github.it235.config.Knife4jRedisConfiguration\
- 第五部，编写工具类
/**
* @description: redis工具类基类，集成通用的方法，由`configuration`进行初始化
* @author: jianjun.ren
* @date: Created in 2020/9/27 11:20
*/
public class RedisBaseUtil {
@Autowired
protected Knife4jRedisManager knife4jRedisManager;
protected int defaultDB = 0;
public void delete(String key) {
delete(0,key);
}
public void delete(int dbIndex ,String key) {
knife4jRedisManager.redisTemplate(dbIndex).delete(key);
}
public boolean set(String key, Object value) {
return set(0,key,value);
}
public boolean set(int dbIndex ,String key, Object value) {
try {
knife4jRedisManager.redisTemplate(dbIndex).opsForValue().set(key, value);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
public void delete(Collection<String> keys){
delete(defaultDB , keys);
}
public void delete(int dbIndex ,Collection<String> keys){
knife4jRedisManager.redisTemplate(dbIndex).delete(keys);
}
public Set<String> getKeys(String redisKey) {
return getKeys(0,redisKey);
}
public Set<String> getKeys(int dbIndex ,String redisKey) {
Set<Object> keys = knife4jRedisManager.redisTemplate(dbIndex).opsForHash().keys(redisKey);
Set<String> retKeys = new HashSet<>();
for (Object key : keys) {
retKeys.add(String.valueOf(key));
}
return retKeys;
}
/**
* 每个redis
* 指定缓存失效时间
* @param key 键
* @param time 时间(秒)
* @return
*/
public boolean expire(String key, long time) {
return expire(defaultDB , key,time);
}
public boolean expire(int dbIndex ,String key, long time) {
try {
if (time > 0) {
knife4jRedisManager.redisTemplate(dbIndex).expire(key, time, TimeUnit.SECONDS);
}
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
/**
* 根据key 获取过期时间
* @param key 键 不能为null
* @return 时间(秒) 返回0代表为永久有效
*/
public long getExpire(int dbIndex , String key) {
return knife4jRedisManager.redisTemplate(dbIndex).getExpire(key, TimeUnit.SECONDS);
}
public long getExpire(String key) {
return getExpire(defaultDB , key);
}
/**
* 判断key是否存在
* @param key 键
* @return true 存在 false不存在
*/
public boolean hasKey(String key) {
return hasKey(defaultDB , key);
}
public boolean hasKey(int dbIndex ,String key) {
try {
return knife4jRedisManager.redisTemplate(dbIndex).hasKey(key);
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
}
- 编写测试类
@RestController
@RequestMapping("/test")
public class RedisTestController {
@Autowired
private RedisBaseUtil redisBaseUtil;
/**
* 单值操作测试
* @param key
* @return
*/
@GetMapping("/val/{key}")
public String test(@PathVariable("key") String key){
redisBaseUtil.set(key , "默认库设置");
redisBaseUtil.set(1 , key , "指定1库设置值");
//查看key是否存在
boolean flag = redisValUtil.hasKey(1, key);
System.out.println("指定1库获取值：" + flag);
return "ok";
}
}
到此测试并查看数据库，就会发现，基础功能已经完成了。
如果有帮助，别忘了star三连：https://github.com/it235/knife4j-redis-lettuce
类似资料：