chromium-cipd : golang net/http proxy代理

最近使用Ubuntu 18.04 LTS desktop版本，通过国外的proxy下载chromium，碰了不少壁，这里说说其中一个cipd更新失败的错误，回家使用windows下载也复现出来了：

C:\myGit\depot_tools>.cipd_client.exe selfupdate -version git_revision:521b9b64770d3b43a221b1e2bd277687616adace
[P2836 07:23:50.221 client.go:310 W] RPC failed transiently. Will retry in 1s    {"error":"failed to send request: Post https://chrome-infra-packages.appspot.com/prpc/cipd.Repository/ResolveVersion: dial tcp 69.171.245.49:443: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.", "host":"chrome-infra-packages.appspot.com", "method":"ResolveVersion", "service":"cipd.Repository", "sleepTime":"1s"}

通过网络，找到windows上的解决方案，就是set http_proxy和https_proxy的环境变量。

但是Ubuntu上怎么处理？因此下了个cipd的源代码来分析下。

发现cipd是用go语言写的，使用了luci的众多package：

go.chromium.org\luci

通过失败的命令行，走读代码，找到关键的流程：

main()->cmdSelfUpdate()->selfupdateRun.Run()->doSelfUpdate()->makeCipdClient()->NewAuthenticator()->

if opts.Transport == nil {
    opts.Transport = http.DefaultTransport
}

说明cipd使用了net/http中的transport来进行http通信，且使用的是DefaultTransport

net\http\transport.go

// DefaultTransport is the default implementation of Transport and is
// used by DefaultClient. It establishes network connections as needed
// and caches them for reuse by subsequent calls. It uses HTTP proxies
// as directed by the $HTTP_PROXY and $NO_PROXY (or $http_proxy and
// $no_proxy) environment variables.
var DefaultTransport RoundTripper = &Transport{
    Proxy: ProxyFromEnvironment,
    DialContext: (&net.Dialer{
        Timeout:   30 * time.Second,
        KeepAlive: 30 * time.Second,
        DualStack: true,
    }).DialContext,
    MaxIdleConns:          100,
    IdleConnTimeout:       90 * time.Second,
    TLSHandshakeTimeout:   10 * time.Second,
    ExpectContinueTimeout: 1 * time.Second,
}

这里我们看到了Proxy的代理相关的代码，注释中已经提到了可以通过添加HTTP_PROXY/http_proxy和HTTPS_PROXY和https_proxy的环境变量来使用http(s)代理。

进一步确认 ProxyFromEnvironment 的实现，注释中也有一大段代理的描述：

// ProxyFromEnvironment returns the URL of the proxy to use for a
// given request, as indicated by the environment variables
// HTTP_PROXY, HTTPS_PROXY and NO_PROXY (or the lowercase versions
// thereof). HTTPS_PROXY takes precedence over HTTP_PROXY for https
// requests.
//
// The environment values may be either a complete URL or a
// "host[:port]", in which case the "http" scheme is assumed.
// An error is returned if the value is a different form.
//
// A nil URL and nil error are returned if no proxy is defined in the
// environment, or a proxy should not be used for the given request,
// as defined by NO_PROXY.
//
// As a special case, if req.URL.Host is "localhost" (with or without
// a port number), then a nil URL and nil error will be returned.
func ProxyFromEnvironment(req *Request) (*url.URL, error) {
    var proxy string
    if req.URL.Scheme == "https" {
        proxy = httpsProxyEnv.Get()
    }
    if proxy == "" {
        proxy = httpProxyEnv.Get()
        if proxy != "" && os.Getenv("REQUEST_METHOD") != "" {
            return nil, errors.New("net/http: refusing to use HTTP_PROXY value in CGI environment; see golang.org/s/cgihttpproxy")
        }
    }

从代码段中能够看出，会先尝试获取https的代理变量，如果获取不到再尝试获取http的代理变量。

//
// Private implementation past this point.
//

var (
    httpProxyEnv = &envOnce{
        names: []string{"HTTP_PROXY", "http_proxy"},
    }
    httpsProxyEnv = &envOnce{
        names: []string{"HTTPS_PROXY", "https_proxy"},
    }
    noProxyEnv = &envOnce{
        names: []string{"NO_PROXY", "no_proxy"},
	}
)

另外，走读中还看到 type Transport struct 中关于支持sock5的代理的描述：

type Transport struct {
    ...
    // Proxy specifies a function to return a proxy for a given
    // Request. If the function returns a non-nil error, the
    // request is aborted with the provided error.
    //
    // The proxy type is determined by the URL scheme. "http"
    // and "socks5" are supported. If the scheme is empty,
    // "http" is assumed.
    //
    // If Proxy is nil or returns a nil *URL, no proxy is used.
    Proxy func(*Request) (*url.URL, error)

另外根据ProxyFromEnvironment的实现，说明cipd使用过程中，代理的使用优先级是https->http->sock5，如果proxy url中没有填schema(http://,https://,sock5://)，那么会使用http://的代理。

Ubuntu和其他linux上设置环境变量的方法网上有很多帖子，这里不再赘述。

通过上述的代码学习也认识到，网络程序的代理都需要自己通过代码实现。只是由于网络库属于基础的库，所以大多都已经实现了proxy的逻辑，不需要上层调用者再自行开发，使用其他人开发的工具时最好调研下。