< =比

Question

我在 SO 上发现了一些关于性能比较的问题

我为比较编写了一个程序（不太好使…复制到您的机器上运行它），其中我为（int i=0； i创建了两个循环

我运行每种方法100次;取了平均经过的时间，发现循环

我多次运行该程序，并且

3018.73, 2778.22

2816.87, 2760.62

2859.02, 2797.05

我的问题是:如果没有一个更快，为什么我会看到结果的差异？我的程序有什么问题吗？




Answer 1

首先，有很多很多理由可以看到基准的变化，即使它们做得很好。以下是我想到的几个：

• 您的计算机同时运行许多其他进程，在上下文中切换和切换内容，等等。操作系统不断接收和处理来自各种 I/O 设备等的中断。所有这些因素都可能导致计算机暂停一段时间，使正在测试的实际代码的运行时间相形见绌。
• JIT 进程可以检测函数何时运行了一定次数，并根据该信息对其应用其他优化。诸如循环展开之类的事情可以大大减少程序必须进行的跳转次数，这比典型的CPU操作要昂贵得多。重新优化指令在第一次发生时需要时间，然后在那之后加快速度。
• 您的硬件正在尝试进行其他优化，例如分支预测，以确保尽可能高效地使用其管道。（如果它猜对了，它基本上可以假装它会在等待查看是否

其次，要做好基准测试实际上真的很难。这是我已经使用了一段时间的基准模板。它并不完美，但它非常擅长确保任何新兴模式不太可能基于执行顺序或随机机会：

/* This is a benchmarking template I use in LINQPad when I want to do a
 * quick performance test. Just give it a couple of actions to test and
 * it will give you a pretty good idea of how long they take compared
 * to one another. It's not perfect: You can expect a 3% error margin
 * under ideal circumstances. But if you're not going to improve
 * performance by more than 3%, you probably don't care anyway.*/
void Main()
{
    // Enter setup code here
    var actions = new[]
    {
        new TimedAction("control", () =>
        {
            int i = 0;
        }),
        new TimedAction("<", () =>
        {
           for (int i = 0; i < 1000001; i++)
            {}
        }),
        new TimedAction("<=", () =>
        {
           for (int i = 0; i <= 1000000; i++)
            {}
        }),
        new TimedAction(">", () =>
        {
           for (int i = 1000001; i > 0; i--)
            {}
        }),
        new TimedAction(">=", () =>
        {
           for (int i = 1000000; i >= 0; i--)
            {}
        })
    };
    const int TimesToRun = 10000; // Tweak this as necessary
    TimeActions(TimesToRun, actions);
}


#region timer helper methods
// Define other methods and classes here
public void TimeActions(int iterations, params TimedAction[] actions)
{
    Stopwatch s = new Stopwatch();
    int length = actions.Length;
    var results = new ActionResult[actions.Length];
    // Perform the actions in their initial order.
    for(int i = 0; i < length; i++)
    {
        var action = actions[i];
        var result = results[i] = new ActionResult{Message = action.Message};
        // Do a dry run to get things ramped up/cached
        result.DryRun1 = s.Time(action.Action, 10);
        result.FullRun1 = s.Time(action.Action, iterations);
    }
    // Perform the actions in reverse order.
    for(int i = length - 1; i >= 0; i--)
    {
        var action = actions[i];
        var result = results[i];
        // Do a dry run to get things ramped up/cached
        result.DryRun2 = s.Time(action.Action, 10);
        result.FullRun2 = s.Time(action.Action, iterations);
    }
    results.Dump();
}

public class ActionResult
{
    public string Message {get;set;}
    public double DryRun1 {get;set;}
    public double DryRun2 {get;set;}
    public double FullRun1 {get;set;}
    public double FullRun2 {get;set;}
}

public class TimedAction
{
    public TimedAction(string message, Action action)
    {
        Message = message;
        Action = action;
    }
    public string Message {get;private set;}
    public Action Action {get;private set;}
}

public static class StopwatchExtensions
{
    public static double Time(this Stopwatch sw, Action action, int iterations)
    {
        sw.Restart();
        for (int i = 0; i < iterations; i++)
        {
            action();
        }
        sw.Stop();

        return sw.Elapsed.TotalMilliseconds;
    }
}
#endregion

以下是我在LINQPad中运行此程序时得到的结果：

所以你会注意到有一些变化，特别是在早期，但是在来回运行足够多的时间后，没有一个清晰的模式出现来表明一种方式比另一种方式快得多或慢得多。

Answer 2

基准测试是一门艺术。你所描述的在物理上是不可能的，即

x86抖动:

Less Than Equal To Method Time Elapsed: 0.5
Less Than Method Time Elapsed: 0.42
Less Than Equal To Method Time Elapsed: 0.36
Less Than Method Time Elapsed: 0.46
Less Than Equal To Method Time Elapsed: 0.4
Less Than Method Time Elapsed: 0.34
Less Than Equal To Method Time Elapsed: 0.33
Less Than Method Time Elapsed: 0.35
Less Than Equal To Method Time Elapsed: 0.35
Less Than Method Time Elapsed: 0.32
Less Than Equal To Method Time Elapsed: 0.32
Less Than Method Time Elapsed: 0.32
Less Than Equal To Method Time Elapsed: 0.34
Less Than Method Time Elapsed: 0.32
Less Than Equal To Method Time Elapsed: 0.32
Less Than Method Time Elapsed: 0.31
Less Than Equal To Method Time Elapsed: 0.34
Less Than Method Time Elapsed: 0.32
Less Than Equal To Method Time Elapsed: 0.31
Less Than Method Time Elapsed: 0.32

x64抖动:

Less Than Equal To Method Time Elapsed: 0.44
Less Than Method Time Elapsed: 0.4
Less Than Equal To Method Time Elapsed: 0.44
Less Than Method Time Elapsed: 0.45
Less Than Equal To Method Time Elapsed: 0.36
Less Than Method Time Elapsed: 0.35
Less Than Equal To Method Time Elapsed: 0.38
Less Than Method Time Elapsed: 0.34
Less Than Equal To Method Time Elapsed: 0.33
Less Than Method Time Elapsed: 0.34
Less Than Equal To Method Time Elapsed: 0.34
Less Than Method Time Elapsed: 0.32
Less Than Equal To Method Time Elapsed: 0.32
Less Than Method Time Elapsed: 0.35
Less Than Equal To Method Time Elapsed: 0.32
Less Than Method Time Elapsed: 0.42
Less Than Equal To Method Time Elapsed: 0.32
Less Than Method Time Elapsed: 0.31
Less Than Equal To Method Time Elapsed: 0.32
Less Than Method Time Elapsed: 0.35

您从这里得到的唯一真实信号是第一个DoIt()的缓慢执行，在您的测试结果中也可以看到，这是抖动开销。而最重要的信号，就是嘈杂。两个循环的中间值大致相等，标准偏差相当大。

否则，当你进行微优化时，你总会得到这样的信号，代码的执行不是很确定。短于。NET运行时开销通常很容易消除，但您的程序并不是唯一在您的机器上运行的程序。它必须共享处理器，只是WriteLine()调用已经产生了影响。由conhost.exe进程执行，在您的测试代码进入下一个for()循环时，与您的测试同时运行。在你的机器上发生的其他事情，内核代码和中断处理程序也有机会。

codegen可以发挥作用，例如，您应该做的一件事就是交换这两个调用。处理器本身通常非常不确定地执行代码。处理器缓存的状态以及分支预测逻辑收集了多少历史数据非常重要。

当我进行基准测试时，我认为15%或更少的差异没有统计学意义。找出小于这个数值的差异是相当困难的，你必须非常仔细地研究机器代码。一些愚蠢的事情，比如分支目标没有对齐或者变量没有存储在处理器寄存器中，都会对执行时间产生很大的影响。这不是你可以修复的，抖动没有足够的旋钮来调整。