在Java8流中有多个Regex从行中读取文本

Question

我想有一个以上的regex如下所示，我如何将其添加到flatmap迭代器中，以便在单个流读取期间将该行的所有匹配值列表？

static String reTimeStamp="((?:2|1)\\d{3}(?:-|\\/)(?:(?:0[1-9])|(?:1[0-2]))(?:-|\\/)(?:(?:0[1-9])|(?:[1-2][0-9])|(?:3[0-1]))(?:T|\\s)(?:(?:[0-1][0-9])|(?:2[0-3])):(?:[0-5][0-9]):(?:[0-5][0-9]))";
static String reHostName="host=(\\\")((?:[a-z][a-z\\.\\d\\-]+)\\.(?:[a-z][a-z\\-]+))(?![\\w\\.])(\\\")";
static String reServiceTime="service=(\\d+)ms";

private static final PatternStreamer quoteRegex1 = new PatternStreamer(reTimeStamp);
private static final PatternStreamer quoteRegex2 = new PatternStreamer(reHostName);
private static final PatternStreamer quoteRegex3 = new PatternStreamer(reServiceTime);


public static void main(String[] args) throws Exception {
    String inFileName = "Sample.log";
    String outFileName = "Sample_output.log";
    try (Stream<String> stream = Files.lines(Paths.get(inFileName))) {
        //stream.forEach(System.out::println);
        List<String> timeStamp = stream.flatMap(quoteRegex1::results)
                                    .map(r -> r.group(1))
                                    .collect(Collectors.toList());

        timeStamp.forEach(System.out::println);
        //Files.write(Paths.get(outFileName), dataSet);
    }
}

这个问题是从匹配模式和使用Java8 stream将流写入文件中的扩展

Answer 1

您可以简单地连接流：

String inFileName = "Sample.log";
String outFileName = "Sample_output.log";
try (Stream<String> stream = Files.lines(Paths.get(inFileName))) {
    List<String> timeStamp = stream
        .flatMap(s -> Stream.concat(quoteRegex1.results(s),
                        Stream.concat(quoteRegex2.results(s), quoteRegex3.results(s))))
        .map(r -> r.group(1))
        .collect(Collectors.toList());

    timeStamp.forEach(System.out::println);
    //Files.write(Paths.get(outFileName), dataSet);
}

但是请注意，这将在每一行中执行三个单独的搜索，这不仅意味着性能较低，而且一行中匹配项的顺序不会反映它们的实际出现情况。这似乎不是你的模式的问题，但个别搜索甚至暗示可能的重叠匹配。

该链接答案的PatternStreamer还贪婪地将一个字符串的匹配项收集到ArrayList中，然后创建流。基于spliterator的解决方案更好，如本答案中所示。

public final class MultiPatternSpliterator
extends Spliterators.AbstractSpliterator<MatchResult> {
    public static Stream<MatchResult> matches(String input, String... patterns) {
        return matches(input, Arrays.stream(patterns)
                .map(Pattern::compile).toArray(Pattern[]::new));
    }
    public static Stream<MatchResult> matches(String input, Pattern... patterns) {
        return StreamSupport.stream(new MultiPatternSpliterator(patterns,input), false);
    }
    private Pattern[] pattern;
    private String input;
    private int pos;
    private PriorityQueue<Matcher> pendingMatches;

    MultiPatternSpliterator(Pattern[] p, String inputString) {
        super(inputString.length(), ORDERED|NONNULL);
        pattern = p;
        input = inputString;
    }

    @Override
    public boolean tryAdvance(Consumer<? super MatchResult> action) {
        if(pendingMatches == null) {
            pendingMatches = new PriorityQueue<>(
                pattern.length, Comparator.comparingInt(MatchResult::start));
            for(Pattern p: pattern) {
                Matcher m = p.matcher(input);
                if(m.find()) pendingMatches.add(m);
            }
        }
        MatchResult mr = null;
        do {
            Matcher m = pendingMatches.poll();
            if(m == null) return false;
            if(m.start() >= pos) {
                mr = m.toMatchResult();
                pos = mr.end();
            }
            if(m.region(pos, m.regionEnd()).find()) pendingMatches.add(m);
        } while(mr == null);
        action.accept(mr);
        return true;
    }
}

Pattern[] p = Stream.of(reTimeStamp, reHostName, reServiceTime)
        .map(Pattern::compile)
        .toArray(Pattern[]::new);
try (Stream<String> stream = Files.lines(Paths.get(inFileName))) {
    List<String> timeStamp = stream
        .flatMap(s -> MultiPatternSpliterator.matches(s, p))
        .map(r -> r.group(1))
        .collect(Collectors.toList());

    timeStamp.forEach(System.out::println);
    //Files.write(Paths.get(outFileName), dataSet);
}

虽然重载的方法允许使用multipatternspliterator.matches(s，reTimeStamp，reHostName，reServiceTime)使用模式字符串创建流，但在flatmap操作中应该避免这种情况，因为该操作将为每一行输入重新编译每一个regex。这就是为什么上面的代码首先将所有模式编译成一个数组。这也是您的原始代码通过在流操作之外实例化PatternStreamer所做的事情。