golang-exec cmd data race

问题背景：bytes.Buffer grow panic

panic: runtime error: slice bounds out of range [:1024] with capacity 512
goroutine 428529 [running]:
bytes.(*Buffer).grow(0xc0001e86f0, 0x200, 0x0)
#011/usr/local/go/src/bytes/buffer.go:148 +0x297
bytes.(*Buffer).ReadFrom(0xc0001e86f0, 0x80e000, 0xc0006aa2e8, 0x7fdd0865d028, 0xc0001e86f0, 0xc000449701)
#011/usr/local/go/src/bytes/buffer.go:202 +0x48
io.copyBuffer(0x80d980, 0xc0001e86f0, 0x80e000, 0xc0006aa2e8, 0x0, 0x0, 0x0, 0x4073b5, 0xc000948ea0, 0xc0004497b0)
#011/usr/local/go/src/io/io.go:395 +0x2ff
io.Copy(...)
#011/usr/local/go/src/io/io.go:368
os/exec.(*Cmd).writerDescriptor.func1(0xc000948ea0, 0xc0004497b0)
#011/usr/local/go/src/os/exec/exec.go:311 +0x65
os/exec.(*Cmd).Start.func1(0xc000c398c0, 0xc000f6ad60)
#011/usr/local/go/src/os/exec/exec.go:441 +0x27
created by os/exec.(*Cmd).Start
#011/usr/local/go/src/os/exec/exec.go:440 +0x629

执行exec.Cmd 中 write bytes.Buffer grow, 出现panic，常理 标准库中应该不会出现这种错误；

google github 上让查data-race。比较符合slice 越界情况，除非出现多个线程同时操作buffer。

查看 writerDescripto 要么 写 stdout， 要么写 stderr，
再看代码：

多个线程 cmd.start 操作同一个 stderr，所以会出现竞态现象

// Pipe stdout of each command into stdin of next
func AssemblePipes(cmds []*exec.Cmd, stdin io.Reader, stdout io.Writer, stderr io.Writer) []*exec.Cmd {
	cmds[0].Stdin = stdin
	cmds[0].Stderr = stderr
	// assemble pipes
	for i, c := range cmds {
		if i < len(cmds)-1 {
			cmds[i+1].Stdin, _ = c.StdoutPipe()
			cmds[i+1].Stderr = stderr
		} else {
			c.Stdout = stdout
			c.Stderr = stderr
		}
	}
	return cmds
}
 
 

使用 go build -race （后面有必要做一个检测）执行：

WARNING: DATA RACE
Write at 0x00c0000121a0 by goroutine 82:
  bytes.(*Buffer).ReadFrom()
      /usr/local/go/src/bytes/buffer.go:200 +0x48
  io.copyBuffer()
      /usr/local/go/src/io/io.go:388 +0x3fa
  os/exec.(*Cmd).writerDescriptor.func1()
      /usr/local/go/src/io/io.go:364 +0x7a
  os/exec.(*Cmd).Start.func1()
      /usr/local/go/src/os/exec/exec.go:435 +0x34

Previous write at 0x00c0000121a0 by goroutine 75:
  bytes.(*Buffer).ReadFrom()
      /usr/local/go/src/bytes/buffer.go:200 +0x48
  io.copyBuffer()
      /usr/local/go/src/io/io.go:388 +0x3fa
  os/exec.(*Cmd).writerDescriptor.func1()
      /usr/local/go/src/io/io.go:364 +0x7a
  os/exec.(*Cmd).Start.func1()
      /usr/local/go/src/os/exec/exec.go:435 +0x34

Goroutine 82 (running) created at:

WARNING: DATA RACE
Read at 0x00c000012180 by goroutine 82:
  bytes.(*Buffer).grow()
      /usr/local/go/src/bytes/buffer.go:73 +0x53
  bytes.(*Buffer).ReadFrom()
      /usr/local/go/src/bytes/buffer.go:202 +0x7c
  io.copyBuffer()
      /usr/local/go/src/io/io.go:388 +0x3fa
  os/exec.(*Cmd).writerDescriptor.func1()
      /usr/local/go/src/io/io.go:364 +0x7a
  os/exec.(*Cmd).Start.func1()
      /usr/local/go/src/os/exec/exec.go:435 +0x34

Previous write at 0x00c000012180 by goroutine 75:
  bytes.(*Buffer).grow()
      /usr/local/go/src/bytes/buffer.go:144 +0x23c
  bytes.(*Buffer).ReadFrom()
      /usr/local/go/src/bytes/buffer.go:202 +0x7c
  io.copyBuffer()
      /usr/local/go/src/io/io.go:388 +0x3fa
  os/exec.(*Cmd).writerDescriptor.func1()
      /usr/local/go/src/io/io.go:364 +0x7a
  os/exec.(*Cmd).Start.func1()
      /usr/local/go/src/os/exec/exec.go:435 +0x34

能否确定是多个cmd 操作stderr 的问题呢？fmt 打印指针基本可以确定。
遗憾的是：不能用动态追踪追踪下 interface 的值：（研究课题）
https://medium.com/@mkevac
https://medium.com/bumble-tech/bpf-and-go-modern-forms-of-introspection-in-linux-6b9802682223
http://www.brendangregg.com/blog/2017-01-31/golang-bcc-bpf-function-tracing.html
https://github.com/grantseltzer/weaver/issues/15
https://riboseyim.github.io/2017/06/27/DTrace_bcc/

https://blog.pixielabs.ai/ebpf-function-tracing/post/
https://github.com/pixie-labs/pixie-demos/blob/main/simple-gotracing/http_trace_uprobe/utils.go
https://blog.pixielabs.ai/ebpf-http-tracing/
https://www.grant.pizza/blog/tracing-go-functions-with-ebpf-part-2/
https://github.com/iovisor/bcc/issues/1320
https://github.com/sematext/uprobe-http-tracer/blob/master/tracer/tracer.go
https://sematext.com/blog/ebpf-userland-apps/
https://www.slideshare.net/RayJenkins1/understanding-ebpf-in-a-hurry-149197981
https://blog.aquasec.com/intro-ebpf-tracing-containers
https://brunocalza.me/how-buffer-pool-works-an-implementation-in-go/
http://www.brendangregg.com/blog/2017-01-31/golang-bcc-bpf-function-tracing.html
https://pkg.go.dev/github.com/cilium/ebpf
https://medium.com/bumble-tech/bpf-and-go-modern-forms-of-introspection-in-linux-6b9802682223
https://www.linuxjournal.com/content/bpf-observability-getting-started-quickly
https://networkop.co.uk/post/2021-03-ebpf-intro/
https://chowdera.com/2021/04/20210401151937493O.html
https://news.ycombinator.com/item?id=24872974
https://speakerdeck.com/leodido/go-ebpf-superpowers
https://github.com/golang/go/issues/13398