通过Antlr 4 lexer/parser抓取所有空行

Question

尝试通过ANTLR4 lexer/parser抓取给定PHP文件的所有空行（仅行号）。我使用的语法在GitHub Antlr语法for PHP上可用。

定义为：

Whitespace:         [ \t\r\n]+ -> skip;

Whitespace: (
             ' ' 
             | '\t' 
             | '\r' '\n' { newline(); } 
             | '\n'       { newline(); }
            );

要测试的示例PHP：

<?php

//02-5002201-00001 5002201 - Machine hours test

	function test()
	{	
/*	Name:			Test.php

	Title:			Demo

	by:				XYZ
*/
		if (true && false)
		{		
			echo "aa";
		}

//TODO		
		echo <<<SEGDTA
		<link rel="stylesheet" type="text/css" href="ui.css"/>

		<script type="text/javascript" src="min.js"></script>
		SEGDTA;
	}

?>

Answer 1

试试下面这样的方法：

lexer grammar DemoLexer;

EmptyLine
 : {super.getCharPositionInLine() == 0}? [ \t]* '\r'? '\n'
 ;

Whitespace
 : [ \t\r\n] -> skip
 ;

Other
 : .
 ;

如果运行以下测试类：

import org.antlr.v4.runtime.CharStreams;
import org.antlr.v4.runtime.Token;

public class Main {

    public static void main(String[] args) {

        String source = "foo\n" +
                "\n" +
                "bar\n" +
                "    \n" +
                "   baz   \r\n" +
                " \t\t\n" +
                "\tend\n\n";

        DemoLexer lexer = new DemoLexer(CharStreams.fromString(source));

        for (Token t : lexer.getAllTokens()) {

            String name = lexer.getTokenNames()[t.getType()];
            String text = t.getText().replace("\r","\\r").replace("\n","\\n").replace("\t","\\t");

            System.out.printf("%-20s '%s'\n", name, text);
        }
    }
}

这将被打印：

Other                'f'
Other                'o'
Other                'o'
EmptyLine            '\n'
Other                'b'
Other                'a'
Other                'r'
EmptyLine            '    \n'
Other                'b'
Other                'a'
Other                'z'
EmptyLine            ' \t\t\n'
Other                'e'
Other                'n'
Other                'd'
EmptyLine            '\n'

参见:http://www.antlr.org/api/java/org/antlr/v4/runtime/tokensource.html#GetCharPositionInline()