log4j日志脱敏实现方案
卫和洽
因为监管要求，需要对全行的日志进行脱敏操作，相关的敏感信息需要进行掩码处理。目前我们所有系统都是使用log4j来进行日志打印的。因此本次的实现方案最开始也是从log4j着手出发，看有没有相关的方案。这篇文章也从2个大方向来进行实现。
1. 基于log4j实现
日志打印流程：
获取Logger
LoggerFactory#getLogger() --> getILoggerFactory()返回Log4jLoggerFactory --> 返回Log4jLogger
输出日志流程：
AbstractLogger --> messageFactory.newMessage() 返回工程对应的message --> 包装成LogEvent进行打印
log4j三大组件
log4j三个组件：
Logger：日志记录器，如何处理日志
Appender：日志输出目的地，日志输出到哪
Layout：日志格式化，以什么形式输出
具体每个组件这里就不展开描述了，通过上面其实也可以看出来，Logger和Layout都可以作为我们日志掩码处理的地方。
PatternLayout 正则表达式
log4j 配置文件里的PatternLayout 属性里有提供正则的方式进行替换，具体实现如下：
<PatternLayout pattern="%m%n">
<replace regex="正则表达式" replacement="替换，比如****" />
</PatternLayout>
这种实现方式只能全局替换，不能自定义掩码规则，例如姓名：张三 ，我需要对名进行掩码，打印成：张*，这种就没办法进行处理了。
另外正则表达式会导致性能问题，所以这个方案对我们并不适用。
自定义PatternLayout
这个方式其实实用性很小，基本不可能会这样对日志进行脱敏，但是也算自己踩过的坑，也写出来给大家借鉴一下。这里先说结论：
这种方式只能获取到一行行的日志，还需要自行进行解析判断哪些是敏感信息来进行脱敏.如果真的考虑用这种方式，则还需要搭配正则表达式来进行解析，正则在我们组是不让用的，因为曾经出现过比较严重的性能问题。当然这只是我个人实践得出的结论，也许这种方式也是有更好的办法欢迎大佬提出，目前个人认为是只能达到这种程度。
log4j提供了几种Layout，通常我们用的比较多的是PatternLayout，这种格式比较灵活。因此可以通过自定义PatternLayout，保留原有的功能上，对PatternLayout增加脱敏的相关处理。又因为Log4j把很多类都声明成final，让这些类都不可以继承，所以要想保留原有的功能可以先将原有的类复制成一个新类，再新类里加自己的逻辑。
如果找不到PatternLayout，可以先找到AbstractStringLayout,再通过它的实现类找到。这里我们复制一下原有代码，修改一下类名。这里需要注意的是有的地方需要人工判断这里是否是需要替换成我们自定义的类CustomPatternLayout.
@Plugin(name="CustomPatternLayout",category="Core",elementType="layout",printObject=true)
public final class CustomPatternLayout extends AbstractStringLayout{
// 修改一下构造方法名称，这里就不写了，
// 如果是直接从class里复制的代码，有的方法需要手动添加上@Override
...
}
可以进行脱敏的地方是在序列化成字符串的方法里：
@Override
public StringBuilder toSerializable(final LogEvent event, final StringBuilder buffer) {
final int len = formatters.length;
for (int i = 0; i < len; i++) {
formatters[i].format(event, buffer);
}
// 对数据进行脱敏处理,例如：
String strCustomize = (buffer.toString()).replace("张三", "***");
buffer.setLength(0);
buffer.append(strCustomize);
return buffer;
}
通过这种方式不仅修改类比较麻烦（因为要保留原有的PatternLayout功能），又因为PatternLayout声明成final不让你继承，所以要改动的地方还是比较多的，还很容易改错可能导致一些原有功能都无法保证的问题。虽然这个放在第一位，但是还是不推荐用这种方式。
自定义messageFactory
根据日志输出流程，AbstractLogger会选择默认的MessageFactory，这个是可以配置的，可以通过增加配置文件进行配置。具体操作是在 src/main/resources目录下增加log4j2.component.properties配置文件，具体配置如下：
log4j2.messageFactory=xxxx/CustomParameterizedMessageFactory
xxx为你自定义的MessageFactory的包路径。这样在项目启动的时候就可以从这个配置文件中读取到你配置的自定义消息工厂CustomParameterizedMessageFactory。具体的源码是在这个地方进行读取的：
- 从logger.info()方法进去
- 往下走到AbstractLogger类的logMessage()方法，这里会从this.messageFactory.newMessage(message,p)里获取一个message
- messageFactory是AbstractLogger的一个成员变量，所以可以跟踪到具体这个变量是在哪里设置的
- 通过AbstractLogger构造方法，可以看到messageFactory 是通过createDefaultMessageFactory()方法创建的
- 这个方法返回的是DEFAULT_MESSAGE_FACTORY_CLASS的实例
private static MessageFactory2 createDefaultMessageFactory() {
try {
final MessageFactory result = DEFAULT_MESSAGE_FACTORY_CLASS.newInstance();
return narrow(result);
} catch (final InstantiationException | IllegalAccessException e) {
throw new IllegalStateException(e);
}
}
- DEFAULT_MESSAGE_FACTORY_CLASS 是通过AbstractLogger是static 静态方法进行设置的
public static final Class<? extends MessageFactory> DEFAULT_MESSAGE_FACTORY_CLASS =
createClassForProperty("log4j2.messageFactory", ReusableMessageFactory.class,
ParameterizedMessageFactory.class);
- 这个方法已经可以看出这里去获取了我们配置的log4j2.messageFactory，再进去就是从PropertiesUtils获取了配置文件路径：log4j.component.properties
从源码里我们也可以得知，如果未配置自定义的messageFactory，log4j默认是使用ParameterizedMessageFactory。因此我们自定义的消息工厂也可以在原有默认的工厂上进行扩展，所以我们可以复制原本的ParameterizedMessageFactory，在这基础上进行修改。
复制ParameterizedMessageFactory，修改类名为自定义消息工厂，这里我使用的就是CustomParameterizedMessageFactory。因为每个MessageFactory都是会返回自己的message，所以我们还需要创建自定义Message。
复制ParameterizedMessageFactory对应的ParameterizedMessage和ParameterFormatter（因为都声明了final，所以只能复制不让继承），分别命名为CustomParameterizedMessage和ParameterFormatter
MessageFactory代码：
package cn.saytime.entity.log4j2;
import org.apache.logging.log4j.message.AbstractMessageFactory;
import org.apache.logging.log4j.message.Message;
import org.apache.logging.log4j.message.MessageFactory;
import org.apache.logging.log4j.message.ParameterizedMessageFactory;
/**
* @description: 自定义log4j2参数化消息工厂
* @date: 2019-01-07 19:42
**/
public class CustomParameterizedMessageFactory extends AbstractMessageFactory {
/**
* Instance of ParameterizedMessageFactory.
*/
public static final ParameterizedMessageFactory INSTANCE = new ParameterizedMessageFactory();
private static final long serialVersionUID = -8970940216592525651L;
/**
* Constructs a message factory.
*/
public CustomParameterizedMessageFactory() {
super();
}
/**
* Creates {@link CustomParameterizedMessage} instances.
*
* @param message The message pattern.
* @param params The message parameters.
* @return The Message.
*
* @see MessageFactory#newMessage(String, Object...)
*/
@Override
public Message newMessage(final String message, final Object... params) {
return new CustomParameterizedMessage(message, params);
}
/**
* @since 2.6.1
*/
@Override
public Message newMessage(final String message, final Object p0) {
return new CustomParameterizedMessage(message, p0);
}
/**
* @since 2.6.1
*/
@Override
public Message newMessage(final String message, final Object p0, final Object p1) {
return new CustomParameterizedMessage(message, p0, p1);
}
/**
* @since 2.6.1
*/
@Override
public Message newMessage(final String message, final Object p0, final Object p1, final Object p2) {
return new CustomParameterizedMessage(message, p0, p1, p2);
}
/**
* @since 2.6.1
*/
@Override
public Message newMessage(final String message, final Object p0, final Object p1, final Object p2, final Object p3) {
return new CustomParameterizedMessage(message, p0, p1, p2, p3);
}
/**
* @since 2.6.1
*/
@Override
public Message newMessage(final String message, final Object p0, final Object p1, final Object p2, final Object p3, final Object p4) {
return new CustomParameterizedMessage(message, p0, p1, p2, p3, p4);
}
/**
* @since 2.6.1
*/
@Override
public Message newMessage(final String message, final Object p0, final Object p1, final Object p2, final Object p3, final Object p4, final Object p5) {
return new CustomParameterizedMessage(message, p0, p1, p2, p3, p4, p5);
}
/**
* @since 2.6.1
*/
@Override
public Message newMessage(final String message, final Object p0, final Object p1, final Object p2, final Object p3, final Object p4, final Object p5,
final Object p6) {
return new CustomParameterizedMessage(message, p0, p1, p2, p3, p4, p5, p6);
}
/**
* @since 2.6.1
*/
@Override
public Message newMessage(final String message, final Object p0, final Object p1, final Object p2, final Object p3, final Object p4, final Object p5,
final Object p6, final Object p7) {
return new CustomParameterizedMessage(message, p0, p1, p2, p3, p4, p5, p6, p7);
}
/**
* @since 2.6.1
*/
@Override
public Message newMessage(final String message, final Object p0, final Object p1, final Object p2, final Object p3, final Object p4, final Object p5,
final Object p6, final Object p7, final Object p8) {
return new CustomParameterizedMessage(message, p0, p1, p2, p3, p4, p5, p6, p7, p8);
}
/**
* @since 2.6.1
*/
@Override
public Message newMessage(final String message, final Object p0, final Object p1, final Object p2, final Object p3, final Object p4, final Object p5,
final Object p6, final Object p7, final Object p8, final Object p9) {
return new CustomParameterizedMessage(message, p0, p1, p2, p3, p4, p5, p6, p7, p8, p9);
}
}
CustomParameterizedMessage代码：
package cn.saytime.entity.log4j2;
import org.apache.logging.log4j.message.Message;
import org.apache.logging.log4j.util.Constants;
import org.apache.logging.log4j.util.StringBuilderFormattable;
import org.apache.logging.log4j.util.StringBuilders;
import java.text.SimpleDateFormat;
import java.util.*;
/**
* @description: 自定义log4j2 参数化消息
* @author: hao_zhang2
* @date: 2019-01-07 19:43
**/
public class CustomParameterizedMessage implements Message, StringBuilderFormattable {
// Should this be configurable?
private static final int DEFAULT_STRING_BUILDER_SIZE = 255;
/**
* Prefix for recursion.
*/
public static final String RECURSION_PREFIX = ParameterFormatter.RECURSION_PREFIX;
/**
* Suffix for recursion.
*/
public static final String RECURSION_SUFFIX = ParameterFormatter.RECURSION_SUFFIX;
/**
* Prefix for errors.
*/
public static final String ERROR_PREFIX = ParameterFormatter.ERROR_PREFIX;
/**
* Separator for errors.
*/
public static final String ERROR_SEPARATOR = ParameterFormatter.ERROR_SEPARATOR;
/**
* Separator for error messages.
*/
public static final String ERROR_MSG_SEPARATOR = ParameterFormatter.ERROR_MSG_SEPARATOR;
/**
* Suffix for errors.
*/
public static final String ERROR_SUFFIX = ParameterFormatter.ERROR_SUFFIX;
private static final long serialVersionUID = -665975803997290697L;
private static final int HASHVAL = 31;
// storing JDK classes in ThreadLocals does not cause memory leaks in web apps, so this is okay
private static ThreadLocal<StringBuilder> threadLocalStringBuilder = new ThreadLocal<>();
private String messagePattern;
private transient Object[] argArray;
private String formattedMessage;
private transient Throwable throwable;
private int[] indices;
private int usedCount;
@Deprecated
public CustomParameterizedMessage(final String messagePattern, final String[] arguments, final Throwable throwable) {
this.argArray = arguments;
this.throwable = throwable;
init(messagePattern);
}
public CustomParameterizedMessage(final String messagePattern, final Object[] arguments, final Throwable throwable) {
this.argArray = arguments;
this.throwable = throwable;
init(messagePattern);
}
public CustomParameterizedMessage(final String messagePattern, final Object... arguments) {
this.argArray = arguments;
init(messagePattern);
}
public CustomParameterizedMessage(final String messagePattern, final Object arg) {
this(messagePattern, new Object[]{arg});
}
public CustomParameterizedMessage(final String messagePattern, final Object arg0, final Object arg1) {
this(messagePattern, new Object[]{arg0, arg1});
}
private void init(final String messagePattern) {
this.messagePattern = messagePattern;
final int len = Math.max(1, messagePattern == null ? 0 : messagePattern.length() >> 1); // divide by 2
this.indices = new int[len]; // LOG4J2-1542 ensure non-zero array length
final int placeholders = ParameterFormatter.countArgumentPlaceholders2(messagePattern, indices);
initThrowable(argArray, placeholders);
this.usedCount = Math.min(placeholders, argArray == null ? 0 : argArray.length);
}
private void initThrowable(final Object[] params, final int usedParams) {
if (params != null) {
final int argCount = params.length;
if (usedParams < argCount && this.throwable == null && params[argCount - 1] instanceof Throwable) {
this.throwable = (Throwable) params[argCount - 1];
}
}
}
@Override
public String getFormat() {
return messagePattern;
}
@Override
public Object[] getParameters() {
return argArray;
}
@Override
public Throwable getThrowable() {
return throwable;
}
@Override
public String getFormattedMessage() {
if (formattedMessage == null) {
final StringBuilder buffer = getThreadLocalStringBuilder();
formatTo(buffer);
formattedMessage = buffer.toString();
StringBuilders.trimToMaxSize(buffer, Constants.MAX_REUSABLE_MESSAGE_SIZE);
}
return formattedMessage;
}
private static StringBuilder getThreadLocalStringBuilder() {
StringBuilder buffer = threadLocalStringBuilder.get();
if (buffer == null) {
buffer = new StringBuilder(DEFAULT_STRING_BUILDER_SIZE);
threadLocalStringBuilder.set(buffer);
}
buffer.setLength(0);
return buffer;
}
@Override
public void formatTo(final StringBuilder buffer) {
if (formattedMessage != null) {
buffer.append(formattedMessage);
} else {
if (indices[0] < 0) {
ParameterFormatter.formatMessage(buffer, messagePattern, argArray, usedCount);
} else {
ParameterFormatter.formatMessage2(buffer, messagePattern, argArray, usedCount, indices);
}
// 此处进行特殊处理
}
}
public static String format(final String messagePattern, final Object[] arguments) {
return ParameterFormatter.format(messagePattern, arguments);
}
@Override
public boolean equals(final Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
final CustomParameterizedMessage that = (CustomParameterizedMessage) o;
if (messagePattern != null ? !messagePattern.equals(that.messagePattern) : that.messagePattern != null) {
return false;
}
if (!Arrays.equals(this.argArray, that.argArray)) {
return false;
}
//if (throwable != null ? !throwable.equals(that.throwable) : that.throwable != null) return false;
return true;
}
@Override
public int hashCode() {
int result = messagePattern != null ? messagePattern.hashCode() : 0;
result = HASHVAL * result + (argArray != null ? Arrays.hashCode(argArray) : 0);
return result;
}
public static int countArgumentPlaceholders(final String messagePattern) {
return ParameterFormatter.countArgumentPlaceholders(messagePattern);
}
public static String deepToString(final Object o) {
return ParameterFormatter.deepToString(o);
}
public static String identityToString(final Object obj) {
return ParameterFormatter.identityToString(obj);
}
@Override
public String toString() {
return "CustomCustomParameterizedMessage[messagePattern=" + messagePattern + ", stringArgs=" +
Arrays.toString(argArray) + ", throwable=" + throwable + ']';
}
}
class ParameterFormatter {
/**
* Prefix for recursion.
*/
static final String RECURSION_PREFIX = "[...";
/**
* Suffix for recursion.
*/
static final String RECURSION_SUFFIX = "...]";
/**
* Prefix for errors.
*/
static final String ERROR_PREFIX = "[!!!";
/**
* Separator for errors.
*/
static final String ERROR_SEPARATOR = "=>";
/**
* Separator for error messages.
*/
static final String ERROR_MSG_SEPARATOR = ":";
/**
* Suffix for errors.
*/
static final String ERROR_SUFFIX = "!!!]";
private static final char DELIM_START = '{';
private static final char DELIM_STOP = '}';
private static final char ESCAPE_CHAR = '\\';
private static ThreadLocal<SimpleDateFormat> threadLocalSimpleDateFormat = new ThreadLocal<>();
private ParameterFormatter() {
}
static int countArgumentPlaceholders(final String messagePattern) {
if (messagePattern == null) {
return 0;
}
final int length = messagePattern.length();
int result = 0;
boolean isEscaped = false;
for (int i = 0; i < length - 1; i++) {
final char curChar = messagePattern.charAt(i);
if (curChar == ESCAPE_CHAR) {
isEscaped = !isEscaped;
} else if (curChar == DELIM_START) {
if (!isEscaped && messagePattern.charAt(i + 1) == DELIM_STOP) {
result++;
i++;
}
isEscaped = false;
} else {
isEscaped = false;
}
}
return result;
}
static int countArgumentPlaceholders2(final String messagePattern, final int[] indices) {
if (messagePattern == null) {
return 0;
}
final int length = messagePattern.length();
int result = 0;
boolean isEscaped = false;
for (int i = 0; i < length - 1; i++) {
final char curChar = messagePattern.charAt(i);
if (curChar == ESCAPE_CHAR) {
isEscaped = !isEscaped;
indices[0] = -1; // escaping means fast path is not available...
result++;
} else if (curChar == DELIM_START) {
if (!isEscaped && messagePattern.charAt(i + 1) == DELIM_STOP) {
indices[result] = i;
result++;
i++;
}
isEscaped = false;
} else {
isEscaped = false;
}
}
return result;
}
static int countArgumentPlaceholders3(final char[] messagePattern, final int length, final int[] indices) {
int result = 0;
boolean isEscaped = false;
for (int i = 0; i < length - 1; i++) {
final char curChar = messagePattern[i];
if (curChar == ESCAPE_CHAR) {
isEscaped = !isEscaped;
} else if (curChar == DELIM_START) {
if (!isEscaped && messagePattern[i + 1] == DELIM_STOP) {
indices[result] = i;
result++;
i++;
}
isEscaped = false;
} else {
isEscaped = false;
}
}
return result;
}
static String format(final String messagePattern, final Object[] arguments) {
final StringBuilder result = new StringBuilder();
final int argCount = arguments == null ? 0 : arguments.length;
formatMessage(result, messagePattern, arguments, argCount);
return result.toString();
}
static void formatMessage2(final StringBuilder buffer, final String messagePattern,
final Object[] arguments, final int argCount, final int[] indices) {
if (messagePattern == null || arguments == null || argCount == 0) {
buffer.append(messagePattern);
return;
}
int previous = 0;
for (int i = 0; i < argCount; i++) {
buffer.append(messagePattern, previous, indices[i]);
previous = indices[i] + 2;
recursiveDeepToString(arguments[i], buffer, null);
}
buffer.append(messagePattern, previous, messagePattern.length());
}
static void formatMessage3(final StringBuilder buffer, final char[] messagePattern, final int patternLength,
final Object[] arguments, final int argCount, final int[] indices) {
if (messagePattern == null) {
return;
}
if (arguments == null || argCount == 0) {
buffer.append(messagePattern);
return;
}
int previous = 0;
for (int i = 0; i < argCount; i++) {
buffer.append(messagePattern, previous, indices[i]);
previous = indices[i] + 2;
recursiveDeepToString(arguments[i], buffer, null);
}
buffer.append(messagePattern, previous, patternLength);
}
static void formatMessage(final StringBuilder buffer, final String messagePattern,
final Object[] arguments, final int argCount) {
if (messagePattern == null || arguments == null || argCount == 0) {
buffer.append(messagePattern);
return;
}
int escapeCounter = 0;
int currentArgument = 0;
int i = 0;
final int len = messagePattern.length();
for (; i < len - 1; i++) { // last char is excluded from the loop
final char curChar = messagePattern.charAt(i);
if (curChar == ESCAPE_CHAR) {
escapeCounter++;
} else {
if (isDelimPair(curChar, messagePattern, i)) { // looks ahead one char
i++;
// write escaped escape chars
writeEscapedEscapeChars(escapeCounter, buffer);
if (isOdd(escapeCounter)) {
// i.e. escaped: write escaped escape chars
writeDelimPair(buffer);
} else {
// unescaped
writeArgOrDelimPair(arguments, argCount, currentArgument, buffer);
currentArgument++;
}
} else {
handleLiteralChar(buffer, escapeCounter, curChar);
}
escapeCounter = 0;
}
}
handleRemainingCharIfAny(messagePattern, len, buffer, escapeCounter, i);
}
private static boolean isDelimPair(final char curChar, final String messagePattern, final int curCharIndex) {
return curChar == DELIM_START && messagePattern.charAt(curCharIndex + 1) == DELIM_STOP;
}
private static void handleRemainingCharIfAny(final String messagePattern, final int len,
final StringBuilder buffer, final int escapeCounter, final int i) {
if (i == len - 1) {
final char curChar = messagePattern.charAt(i);
handleLastChar(buffer, escapeCounter, curChar);
}
}
private static void handleLastChar(final StringBuilder buffer, final int escapeCounter, final char curChar) {
if (curChar == ESCAPE_CHAR) {
writeUnescapedEscapeChars(escapeCounter + 1, buffer);
} else {
handleLiteralChar(buffer, escapeCounter, curChar);
}
}
private static void handleLiteralChar(final StringBuilder buffer, final int escapeCounter, final char curChar) {
// any other char beside ESCAPE or DELIM_START/STOP-combo
// write unescaped escape chars
writeUnescapedEscapeChars(escapeCounter, buffer);
buffer.append(curChar);
}
private static void writeDelimPair(final StringBuilder buffer) {
buffer.append(DELIM_START);
buffer.append(DELIM_STOP);
}
private static boolean isOdd(final int number) {
return (number & 1) == 1;
}
private static void writeEscapedEscapeChars(final int escapeCounter, final StringBuilder buffer) {
final int escapedEscapes = escapeCounter >> 1; // divide by two
writeUnescapedEscapeChars(escapedEscapes, buffer);
}
private static void writeUnescapedEscapeChars(int escapeCounter, final StringBuilder buffer) {
while (escapeCounter > 0) {
buffer.append(ESCAPE_CHAR);
escapeCounter--;
}
}
private static void writeArgOrDelimPair(final Object[] arguments, final int argCount, final int currentArgument,
final StringBuilder buffer) {
if (currentArgument < argCount) {
recursiveDeepToString(arguments[currentArgument], buffer, null);
} else {
writeDelimPair(buffer);
}
}
static String deepToString(final Object o) {
if (o == null) {
return null;
}
if (o instanceof String) {
return (String) o;
}
final StringBuilder str = new StringBuilder();
final Set<String> dejaVu = new HashSet<>(); // that's actually a neat name ;)
recursiveDeepToString(o, str, dejaVu);
return str.toString();
}
private static void recursiveDeepToString(final Object o, final StringBuilder str, final Set<String> dejaVu) {
if (appendSpecialTypes(o, str)) {
return;
}
if (isMaybeRecursive(o)) {
appendPotentiallyRecursiveValue(o, str, dejaVu);
} else {
tryObjectToString(o, str);
}
}
private static boolean appendSpecialTypes(final Object o, final StringBuilder str) {
if (o == null || o instanceof String) {
str.append((String) o);
return true;
} else if (o instanceof CharSequence) {
str.append((CharSequence) o);
return true;
} else if (o instanceof StringBuilderFormattable) {
((StringBuilderFormattable) o).formatTo(str);
return true;
} else if (o instanceof Integer) { // LOG4J2-1415 unbox auto-boxed primitives to avoid calling toString()
str.append(((Integer) o).intValue());
return true;
} else if (o instanceof Long) {
str.append(((Long) o).longValue());
return true;
} else if (o instanceof Double) {
str.append(((Double) o).doubleValue());
return true;
} else if (o instanceof Boolean) {
str.append(((Boolean) o).booleanValue());
return true;
} else if (o instanceof Character) {
str.append(((Character) o).charValue());
return true;
} else if (o instanceof Short) {
str.append(((Short) o).shortValue());
return true;
} else if (o instanceof Float) {
str.append(((Float) o).floatValue());
return true;
}
return appendDate(o, str);
}
private static boolean appendDate(final Object o, final StringBuilder str) {
if (!(o instanceof Date)) {
return false;
}
final Date date = (Date) o;
final SimpleDateFormat format = getSimpleDateFormat();
str.append(format.format(date));
return true;
}
private static SimpleDateFormat getSimpleDateFormat() {
SimpleDateFormat result = threadLocalSimpleDateFormat.get();
if (result == null) {
result = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ");
threadLocalSimpleDateFormat.set(result);
}
return result;
}
private static boolean isMaybeRecursive(final Object o) {
return o.getClass().isArray() || o instanceof Map || o instanceof Collection;
}
private static void appendPotentiallyRecursiveValue(final Object o, final StringBuilder str,
final Set<String> dejaVu) {
final Class<?> oClass = o.getClass();
if (oClass.isArray()) {
appendArray(o, str, dejaVu, oClass);
} else if (o instanceof Map) {
appendMap(o, str, dejaVu);
} else if (o instanceof Collection) {
appendCollection(o, str, dejaVu);
}
}
private static void appendArray(final Object o, final StringBuilder str, Set<String> dejaVu,
final Class<?> oClass) {
if (oClass == byte[].class) {
str.append(Arrays.toString((byte[]) o));
} else if (oClass == short[].class) {
str.append(Arrays.toString((short[]) o));
} else if (oClass == int[].class) {
str.append(Arrays.toString((int[]) o));
} else if (oClass == long[].class) {
str.append(Arrays.toString((long[]) o));
} else if (oClass == float[].class) {
str.append(Arrays.toString((float[]) o));
} else if (oClass == double[].class) {
str.append(Arrays.toString((double[]) o));
} else if (oClass == boolean[].class) {
str.append(Arrays.toString((boolean[]) o));
} else if (oClass == char[].class) {
str.append(Arrays.toString((char[]) o));
} else {
if (dejaVu == null) {
dejaVu = new HashSet<>();
}
// special handling of container Object[]
final String id = identityToString(o);
if (dejaVu.contains(id)) {
str.append(RECURSION_PREFIX).append(id).append(RECURSION_SUFFIX);
} else {
dejaVu.add(id);
final Object[] oArray = (Object[]) o;
str.append('[');
boolean first = true;
for (final Object current : oArray) {
if (first) {
first = false;
} else {
str.append(", ");
}
recursiveDeepToString(current, str, new HashSet<>(dejaVu));
}
str.append(']');
}
//str.append(Arrays.deepToString((Object[]) o));
}
}
private static void appendMap(final Object o, final StringBuilder str, Set<String> dejaVu) {
// special handling of container Map
if (dejaVu == null) {
dejaVu = new HashSet<>();
}
final String id = identityToString(o);
if (dejaVu.contains(id)) {
str.append(RECURSION_PREFIX).append(id).append(RECURSION_SUFFIX);
} else {
dejaVu.add(id);
final Map<?, ?> oMap = (Map<?, ?>) o;
str.append('{');
boolean isFirst = true;
for (final Object o1 : oMap.entrySet()) {
final Map.Entry<?, ?> current = (Map.Entry<?, ?>) o1;
if (isFirst) {
isFirst = false;
} else {
str.append(", ");
}
final Object key = current.getKey();
final Object value = current.getValue();
recursiveDeepToString(key, str, new HashSet<>(dejaVu));
str.append('=');
recursiveDeepToString(value, str, new HashSet<>(dejaVu));
}
str.append('}');
}
}
private static void appendCollection(final Object o, final StringBuilder str, Set<String> dejaVu) {
// special handling of container Collection
if (dejaVu == null) {
dejaVu = new HashSet<>();
}
final String id = identityToString(o);
if (dejaVu.contains(id)) {
str.append(RECURSION_PREFIX).append(id).append(RECURSION_SUFFIX);
} else {
dejaVu.add(id);
final Collection<?> oCol = (Collection<?>) o;
str.append('[');
boolean isFirst = true;
for (final Object anOCol : oCol) {
if (isFirst) {
isFirst = false;
} else {
str.append(", ");
}
recursiveDeepToString(anOCol, str, new HashSet<>(dejaVu));
}
str.append(']');
}
}
private static void tryObjectToString(final Object o, final StringBuilder str) {
// it's just some other Object, we can only use toString().
try {
str.append(o.toString());
} catch (final Throwable t) {
handleErrorInObjectToString(o, str, t);
}
}
private static void handleErrorInObjectToString(final Object o, final StringBuilder str, final Throwable t) {
str.append(ERROR_PREFIX);
str.append(identityToString(o));
str.append(ERROR_SEPARATOR);
final String msg = t.getMessage();
final String className = t.getClass().getName();
str.append(className);
if (!className.equals(msg)) {
str.append(ERROR_MSG_SEPARATOR);
str.append(msg);
}
str.append(ERROR_SUFFIX);
}
static String identityToString(final Object obj) {
if (obj == null) {
return null;
}
return obj.getClass().getName() + '@' + Integer.toHexString(System.identityHashCode(obj));
}
}
2. 包装LoggerFactory(装饰器模式)+fastjson filter过滤器
使用装饰器模式，兹定于LogFactory和Logger，在不改变原本功能的情况下，对其进行扩展
什么是装饰器模式
装饰器模式（Decorator Pattern）允许向一个现有的对象添加新的功能，同时又不改变其结构。这种类型的设计模式属于结构型模式，它是作为现有的类的一个包装。
这种模式创建了一个装饰类，用来包装原有的类，并在保持类方法签名完整性的前提下，提供了额外的功能。
具体做法：
- 在LoggerFactory 里新增一个方法==getLogger(Class<?> clazz) ==,用于替换我们旧的LoggerFactory，这里我们可以重新命名，也可以保留名称。我这里保留了原本的名称，这样我在项目里只需要全局替换LoggerFactory的包路径即可（这也是这个方法的缺点之一）
- 因为
getLogger方法返回值是Logger，这里我们返回自定义的CustomLogger，但是为了不破坏原有的Logger，在这个方法里还是先通过org.slf4j.LoggerFactory.getLogger(Class<?> clazz)方法获取原本的Logger，在通过new CustomLogger(logger)通过构造器传给CustomLogger - CustomLogger实现
org.slf4j.Logger接口，这里就需要重写Logger里的全部方法.我们新增一个私有变量logger(org.slf4j.Logger)，然后将所有重写的方法都使用这个logger的方法。因为这里的logger也是个接口，所以具体的实现类是通过LoggerFactory返回回来的，因此就算以后使用的不是Loger4jLogger,这个功能也不会失效导致需要重写 - 在info方法上进行扩展处理，例如对有2个参数的info方法进行处理：
@Override
public void info(String format,Object arg1,Object arg2){
logger.info(format,maskMsg(arg1),maskMsg(arg2));
}
对2个参数arg1和arg2都进行掩码处理，调用maskMsg方法。这里掩码是做在调用原本的logger方法以前，算是一种功能的增强
掩码处理实现
掩码处理的是Object对象，因参数可能多个，这里将maskMsg入参设置为可变参数Object… args ,对其进行循环处理，maskMsg方法代码如下：
private static Object[] maskMsg(Object... args){
if(args == null){
return args;
}else{
for(int i=0;i<args.length;++i){
try{
args[i] = MaskUtils.maskObj(args[i]);
}catch(Exception e){
}
}
return args;
}
}
具体的处理在工具类maskObj方法里
public static Object maskObj(Object obj){
// 开关控制是否进行脱敏
if(!isMaskable()){
return obj;
}
try{
if(!IbjectUtils.isEmpty(obj) && !(obj instanceof number) && !(obj instanceof String) ){
return JSON.toJSONString(obj,MASK_FILTER,SerializerFeature.EMPTY);
}else{
return obj;
}
}catch(Exception e){
// 异常处理
}
}
这里并没有对String类型的数据进行处理，主要是因为我们系统日质量较大，经过组内评估部处理。如果需要处理的话，还可以对String转JSON继续处理
实际上就是通过fastjson里的JSON.toJSONString(Object object,SerialzeFilter filter,SerialZerFeatrue… featrures)方法，filter就是我们自定义的MASK_FILTER是自定义的过滤器，ValueFilter可以重写process方法，具体代码如下：
// lamda表达式写法
public static final VauleFilter MASK_FILTER = (object, name , value ) -> {
if(!ObjectUtils.isEmpty(value) && !ObjectUtils.isEmpty(name) ){
// 根据name获取对应的敏感类型
CfssCoreMaskEnum maskEnum = CfssMaskUtils.getSensitiveType(name);
return maskEnum == null ? value : getMaskStr()maskEnum,String,valueOf(value);
}
return value;
}
自定义过滤器初始化的时候重写ValueFilter里的process方法，process三个参数中，第二个参数是序列化时对象里的字段名称，第三个参数value对应的是此字段的内容，只要能拿到这2个，就可以根据字段名称是否匹配上掩码关键字进行掩码处理。
当可以拿到字段名称和字段值，就很容易就行处理了：
- 维护一个静态map作为缓存，key为敏感信息关键字，value为敏感字段的枚举值，在系统启动的时候加载好。因为我们系统有的是驼峰，有的是下划线，有的是纯大写，所以我们这边维护的key都是大写的，如果带下划线_的，则在map中put去除下划线的值和原本的值，这样如果驼峰或则下划线的只需要维护一个就可以了，增加命中map key值的命中率。
- 匹配上了对应的敏感类型，则调对应的工具类处理即可
- 如果敏感字段有更新，则增加监听处理。因为敏感在i段都是配置在apollo里，需要在监听类加上对应的注解，然后再监听的方法上使用@ApolloconfigChangeListener 注解即可
总结：
类似资料：