TestNG重试失败的测试不会输出正确的测试结果

Question

安装程序：我有一个扩展IRetryAnalyzer的类，并实现了一个简单的重试逻辑，覆盖了以下方法：public boolean retry（ITestResult result）{

提前道谢。

Answer 1

请考虑以下与MAX的测试结果。2次重试：

1. 通过=>总体确定！
2. 失败，通过=>总体确定！
3. 失败，失败，通过=>总体确定！
4. 失败，失败，失败=>整体失败！

我所做的是创建一个TestNg侦听器，它扩展了默认行为，并在所有测试完成后执行一些魔法操作。

public class FixRetryListener extends TestListenerAdapter {

    @Override
    public void onFinish(ITestContext testContext) {
        super.onFinish(testContext);

        // List of test results which we will delete later
        List<ITestResult> testsToBeRemoved = new ArrayList<>();

        // collect all id's from passed test
        Set <Integer> passedTestIds = new HashSet<>();
        for (ITestResult passedTest : testContext.getPassedTests().getAllResults()) {
            passedTestIds.add(TestUtil.getId(passedTest));
        }

        Set <Integer> failedTestIds = new HashSet<>();
        for (ITestResult failedTest : testContext.getFailedTests().getAllResults()) {

            // id = class + method + dataprovider
            int failedTestId = TestUtil.getId(failedTest);

            // if we saw this test as a failed test before we mark as to be deleted
            // or delete this failed test if there is at least one passed version
            if (failedTestIds.contains(failedTestId) || passedTestIds.contains(failedTestId)) {
                testsToBeRemoved.add(failedTest);
            } else {
                failedTestIds.add(failedTestId);
            }
        }

        // finally delete all tests that are marked
        for (Iterator<ITestResult> iterator = testContext.getFailedTests().getAllResults().iterator(); iterator.hasNext(); ) {
            ITestResult testResult = iterator.next();
            if (testsToBeRemoved.contains(testResult)) {
                iterator.remove();
            }
        }

    }

}

null

public class TestUtil {

    public static int getId(ITestResult result) {
        int id = result.getTestClass().getName().hashCode();
        id = 31 * id + result.getMethod().getMethodName().hashCode();
        id = 31 * id + (result.getParameters() != null ? Arrays.hashCode(result.getParameters()) : 0);
        return id;
    }
}

@DataProvider(name = "dataprovider")
public Object[][] getData() {
    return new Object[][]{{System.currentTimeMillis()}};
}

对于失败的测试，将重新计算dataprovider。因此，您将获得一个新的时间戳，并且对于同一mehod的result1和result2将导致不同的哈希值。解决方案是在getId（）方法中包含parameterIndex而不是parameters，但ItestResult中似乎没有包含这样的值。

请参阅以下简单示例作为概念证明：

@Listeners(value = FixRetryListener.class)
public class SimpleTest {

    private int count = 0;

    @DataProvider(name = "dataprovider")
    public Object[][] getData() {
        return new Object[][]{{"Run1"},{"Run2"}};
    }

    @Test(retryAnalyzer = RetryAnalyzer.class, dataProvider = "dataprovider")
    public void teste(String testName) {
        count++;
        System.out.println("---------------------------------------");
        System.out.println(testName + " " + count);
        if (count % 3 != 0) {
            Assert.fail();
        }

        count = 0;
    }

}

将屈服于：

Total tests run: 2, Failures: 0, Skips: 0