weed-fs 压力测试

weed-fs 测试环境

ip	seaweed 版本	操作系统
192.168.8.16	2.88	CentOS Linux release 7.6.1810

测试方法

使用 benchmark 模拟读写测试
并发100，先写100万个1KB文件，后随机读取测试

./weedfs benchmark -c=100 -collection="benchmark" -size=1024 -n=1048576 -master="192.168.9.102:9333"

测试场景

• 本地不挂载nfs

----------- Writing Benchmark ----------
Completed 32519 of 1048576 requests, 3.1% 32502.6/s 32.7MB/s
Completed 69691 of 1048576 requests, 6.6% 37178.8/s 37.4MB/s
Completed 107048 of 1048576 requests, 10.2% 37358.2/s 37.6MB/s
Completed 143786 of 1048576 requests, 13.7% 36744.0/s 37.0MB/s
Completed 180857 of 1048576 requests, 17.2% 37077.8/s 37.3MB/s
Completed 218177 of 1048576 requests, 20.8% 37320.3/s 37.6MB/s
Completed 255767 of 1048576 requests, 24.4% 37588.2/s 37.8MB/s
Completed 292328 of 1048576 requests, 27.9% 36553.6/s 36.8MB/s
Completed 329049 of 1048576 requests, 31.4% 36730.0/s 37.0MB/s
Completed 365875 of 1048576 requests, 34.9% 36807.3/s 37.1MB/s
Completed 402848 of 1048576 requests, 38.4% 36992.4/s 37.2MB/s
Completed 439714 of 1048576 requests, 41.9% 36863.3/s 37.1MB/s
Completed 476813 of 1048576 requests, 45.5% 37101.0/s 37.3MB/s
Completed 513173 of 1048576 requests, 48.9% 36358.3/s 36.6MB/s
Completed 550031 of 1048576 requests, 52.5% 36849.4/s 37.1MB/s
Completed 587580 of 1048576 requests, 56.0% 37559.2/s 37.8MB/s
Completed 624796 of 1048576 requests, 59.6% 37216.6/s 37.5MB/s
Completed 663822 of 1048576 requests, 63.3% 39012.8/s 39.3MB/s
Completed 701189 of 1048576 requests, 66.9% 37367.9/s 37.6MB/s
Completed 737930 of 1048576 requests, 70.4% 36729.9/s 37.0MB/s
Completed 775515 of 1048576 requests, 74.0% 37607.7/s 37.9MB/s
Completed 810368 of 1048576 requests, 77.3% 34850.9/s 35.1MB/s
Completed 849244 of 1048576 requests, 81.0% 38875.9/s 39.1MB/s
Completed 885879 of 1048576 requests, 84.5% 36550.7/s 36.8MB/s
Completed 923218 of 1048576 requests, 88.0% 37425.0/s 37.7MB/s
Completed 960474 of 1048576 requests, 91.6% 37183.0/s 37.4MB/s
Completed 997335 of 1048576 requests, 95.1% 36936.9/s 37.2MB/s
Completed 1034254 of 1048576 requests, 98.6% 36916.4/s 37.2MB/s

Concurrency Level:      100
Time taken for tests:   28.356 seconds
Complete requests:      1048576
Failed requests:        0
Total transferred:      1106767077 bytes
Requests per second:    36978.83 [#/sec]
Transfer rate:          38116.20 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        0.3      2.6       20.8      1.6

Percentage of the requests served within a certain time (ms)
   50%      1.9 ms
   66%      2.7 ms
   75%      3.4 ms
   80%      4.0 ms
   90%      5.2 ms
   95%      5.8 ms
   98%      6.6 ms
   99%      7.1 ms
  100%     20.8 ms

------------ Randomly Reading Benchmark ----------
Completed 91137 of 1048576 requests, 8.7% 91116.7/s 91.7MB/s
Completed 197804 of 1048576 requests, 18.9% 106689.3/s 107.4MB/s
Completed 306640 of 1048576 requests, 29.2% 108832.6/s 109.6MB/s
Completed 414955 of 1048576 requests, 39.6% 108319.1/s 109.0MB/s
Completed 520581 of 1048576 requests, 49.6% 105625.8/s 106.3MB/s
Completed 626340 of 1048576 requests, 59.7% 105751.1/s 106.5MB/s
Completed 735383 of 1048576 requests, 70.1% 109041.0/s 109.8MB/s
Completed 842837 of 1048576 requests, 80.4% 107464.4/s 108.2MB/s
Completed 952149 of 1048576 requests, 90.8% 109311.0/s 110.0MB/s

Concurrency Level:      100
Time taken for tests:   9.898 seconds
Complete requests:      1048576
Failed requests:        0
Total transferred:      1106757712 bytes
Requests per second:    105941.49 [#/sec]
Transfer rate:          109199.03 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        0.1      0.8       14.0      0.9

Percentage of the requests served within a certain time (ms)
   50%      0.6 ms
   66%      0.9 ms
   75%      1.1 ms
   80%      1.3 ms
   90%      1.8 ms
   95%      2.5 ms
   98%      3.6 ms
   99%      4.8 ms
  100%     14.0 ms

• 本地挂载nfs

------------ Writing Benchmark ----------
Completed 25029 of 1048576 requests, 2.4% 25023.8/s 25.2MB/s
Completed 61170 of 1048576 requests, 5.8% 36144.0/s 36.4MB/s
Completed 97602 of 1048576 requests, 9.3% 36436.3/s 36.7MB/s
Completed 135023 of 1048576 requests, 12.9% 37391.4/s 37.6MB/s
Completed 171158 of 1048576 requests, 16.3% 36163.6/s 36.4MB/s
Completed 172511 of 1048576 requests, 16.5% 1351.9/s 1.4MB/s
Completed 182527 of 1048576 requests, 17.4% 10024.4/s 10.1MB/s
Completed 218790 of 1048576 requests, 20.9% 36245.1/s 36.5MB/s
Completed 256744 of 1048576 requests, 24.5% 37972.8/s 38.2MB/s
Completed 291007 of 1048576 requests, 27.8% 34261.8/s 34.5MB/s
Completed 327099 of 1048576 requests, 31.2% 36086.7/s 36.3MB/s
Completed 364972 of 1048576 requests, 34.8% 37864.7/s 38.1MB/s
Completed 402569 of 1048576 requests, 38.4% 37605.3/s 37.8MB/s
Completed 440599 of 1048576 requests, 42.0% 38036.8/s 38.3MB/s
Completed 478100 of 1048576 requests, 45.6% 37498.4/s 37.7MB/s
Completed 514831 of 1048576 requests, 49.1% 36728.1/s 37.0MB/s
Completed 551702 of 1048576 requests, 52.6% 36876.6/s 37.1MB/s
Completed 587823 of 1048576 requests, 56.1% 36118.3/s 36.4MB/s
Completed 624887 of 1048576 requests, 59.6% 37066.3/s 37.3MB/s
Completed 663885 of 1048576 requests, 63.3% 38998.2/s 39.3MB/s
Completed 701119 of 1048576 requests, 66.9% 37234.5/s 37.5MB/s
Completed 739619 of 1048576 requests, 70.5% 38498.1/s 38.7MB/s
Completed 777079 of 1048576 requests, 74.1% 37453.2/s 37.7MB/s
Completed 813902 of 1048576 requests, 77.6% 36825.4/s 37.1MB/s
Completed 851320 of 1048576 requests, 81.2% 37403.4/s 37.7MB/s
Completed 888874 of 1048576 requests, 84.8% 37571.6/s 37.8MB/s
Completed 927022 of 1048576 requests, 88.4% 38139.2/s 38.4MB/s
Completed 963956 of 1048576 requests, 91.9% 36943.3/s 37.2MB/s
Completed 1001169 of 1048576 requests, 95.5% 37180.4/s 37.4MB/s
Completed 1038169 of 1048576 requests, 99.0% 37030.7/s 37.3MB/s

Concurrency Level:      100
Time taken for tests:   30.286 seconds
Complete requests:      1048576
Failed requests:        0
Total transferred:      1106758292 bytes
Requests per second:    34622.16 [#/sec]
Transfer rate:          35686.76 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        0.3      2.8       1653.2      16.2

Percentage of the requests served within a certain time (ms)
   50%      2.0 ms
   66%      2.6 ms
   75%      3.4 ms
   80%      4.0 ms
   90%      5.2 ms
   95%      5.9 ms
   98%      6.6 ms
   99%      7.2 ms
  100%    1653.2 ms

------------ Randomly Reading Benchmark ----------
Completed 89353 of 1048576 requests, 8.5% 89344.2/s 89.9MB/s
Completed 198225 of 1048576 requests, 18.9% 108839.4/s 109.6MB/s
Completed 308330 of 1048576 requests, 29.4% 110147.0/s 110.9MB/s
Completed 418244 of 1048576 requests, 39.9% 109914.2/s 110.6MB/s
Completed 526718 of 1048576 requests, 50.2% 108463.6/s 109.2MB/s
Completed 634720 of 1048576 requests, 60.5% 108010.1/s 108.7MB/s
Completed 741321 of 1048576 requests, 70.7% 106603.2/s 107.3MB/s
Completed 850287 of 1048576 requests, 81.1% 108928.1/s 109.7MB/s
Completed 959126 of 1048576 requests, 91.5% 108877.8/s 109.6MB/s

Concurrency Level:      100
Time taken for tests:   9.862 seconds
Complete requests:      1048576
Failed requests:        0
Total transferred:      1106775701 bytes
Requests per second:    106330.02 [#/sec]
Transfer rate:          109601.28 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        0.0      0.8       14.5      0.9

Percentage of the requests served within a certain time (ms)
   50%      0.6 ms
   66%      0.9 ms
   75%      1.1 ms
   80%      1.3 ms
   90%      1.8 ms
   95%      2.5 ms
   98%      3.6 ms
   99%      4.8 ms
  100%     14.5 ms

• 容器化 不使用 nfs

------------ Writing Benchmark ----------
Completed 24847 of 1048576 requests, 2.4% 24846.4/s 25.0MB/s
Completed 60683 of 1048576 requests, 5.8% 35827.4/s 36.1MB/s
Completed 97599 of 1048576 requests, 9.3% 36883.9/s 37.1MB/s
Completed 135106 of 1048576 requests, 12.9% 37548.5/s 37.8MB/s
Completed 171712 of 1048576 requests, 16.4% 36607.0/s 36.9MB/s
Completed 207832 of 1048576 requests, 19.8% 36109.0/s 36.3MB/s
Completed 243725 of 1048576 requests, 23.2% 35904.0/s 36.1MB/s
Completed 280943 of 1048576 requests, 26.8% 37215.8/s 37.5MB/s
Completed 318745 of 1048576 requests, 30.4% 37770.9/s 38.0MB/s
Completed 356423 of 1048576 requests, 34.0% 37710.7/s 38.0MB/s
Completed 395104 of 1048576 requests, 37.7% 38681.4/s 38.9MB/s
Completed 433083 of 1048576 requests, 41.3% 37978.6/s 38.2MB/s
Completed 470901 of 1048576 requests, 44.9% 37818.3/s 38.1MB/s
Completed 506233 of 1048576 requests, 48.3% 35325.0/s 35.6MB/s
Completed 543124 of 1048576 requests, 51.8% 36886.4/s 37.1MB/s
Completed 581099 of 1048576 requests, 55.4% 37987.6/s 38.2MB/s
Completed 618376 of 1048576 requests, 59.0% 37276.7/s 37.5MB/s
Completed 653679 of 1048576 requests, 62.3% 35302.9/s 35.5MB/s
Completed 686724 of 1048576 requests, 65.5% 33036.5/s 33.3MB/s
Completed 722881 of 1048576 requests, 68.9% 36166.5/s 36.4MB/s
Completed 761828 of 1048576 requests, 72.7% 38939.0/s 39.2MB/s
Completed 800181 of 1048576 requests, 76.3% 38360.8/s 38.6MB/s
Completed 839296 of 1048576 requests, 80.0% 39053.9/s 39.3MB/s
Completed 877272 of 1048576 requests, 83.7% 38034.5/s 38.3MB/s
Completed 915227 of 1048576 requests, 87.3% 37930.2/s 38.2MB/s
Completed 954042 of 1048576 requests, 91.0% 38840.2/s 39.1MB/s
Completed 989903 of 1048576 requests, 94.4% 35852.6/s 36.1MB/s
Completed 1027635 of 1048576 requests, 98.0% 37736.5/s 38.0MB/s

Concurrency Level:      100
Time taken for tests:   28.550 seconds
Complete requests:      1048576
Failed requests:        0
Total transferred:      1106792146 bytes
Requests per second:    36727.60 [#/sec]
Transfer rate:          37858.09 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        0.3      2.6       250.0      2.7

Percentage of the requests served within a certain time (ms)
   50%      2.0 ms
   66%      2.6 ms
   75%      3.3 ms
   80%      3.8 ms
   90%      5.1 ms
   95%      5.8 ms
   98%      6.5 ms
   99%      7.1 ms
  100%    250.0 ms

------------ Randomly Reading Benchmark ----------
Completed 85977 of 1048576 requests, 8.2% 85976.3/s 86.5MB/s
Completed 189188 of 1048576 requests, 18.0% 103181.0/s 103.9MB/s
Completed 294827 of 1048576 requests, 28.1% 105669.9/s 106.4MB/s
Completed 398144 of 1048576 requests, 38.0% 103317.0/s 104.0MB/s
Completed 503945 of 1048576 requests, 48.1% 105800.1/s 106.5MB/s
Completed 610118 of 1048576 requests, 58.2% 106172.0/s 106.9MB/s
Completed 717164 of 1048576 requests, 68.4% 107044.8/s 107.8MB/s
Completed 822256 of 1048576 requests, 78.4% 105095.0/s 105.8MB/s
Completed 929762 of 1048576 requests, 88.7% 107504.0/s 108.2MB/s
Completed 1036110 of 1048576 requests, 98.8% 106341.7/s 107.1MB/s

Concurrency Level:      100
Time taken for tests:   10.116 seconds
Complete requests:      1048576
Failed requests:        0
Total transferred:      1106795837 bytes
Requests per second:    103652.52 [#/sec]
Transfer rate:          106843.36 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        0.1      0.9       24.5      0.9

Percentage of the requests served within a certain time (ms)
   50%      0.6 ms
   66%      0.9 ms
   75%      1.1 ms
   80%      1.2 ms
   90%      1.8 ms
   95%      2.4 ms
   98%      3.6 ms
   99%      4.8 ms
  100%     24.5 ms

容器化 nfs存储

------------ Writing Benchmark ----------
Completed 34023 of 1048576 requests, 3.2% 34022.2/s 34.2MB/s
Completed 70579 of 1048576 requests, 6.7% 36463.5/s 36.7MB/s
Completed 106427 of 1048576 requests, 10.1% 35938.4/s 36.2MB/s
Completed 142273 of 1048576 requests, 13.6% 35814.8/s 36.1MB/s
Completed 178955 of 1048576 requests, 17.1% 36703.0/s 36.9MB/s
Completed 215779 of 1048576 requests, 20.6% 36836.1/s 37.1MB/s
Completed 246561 of 1048576 requests, 23.5% 30781.8/s 31.0MB/s
Completed 283585 of 1048576 requests, 27.0% 37022.0/s 37.3MB/s
Completed 319791 of 1048576 requests, 30.5% 36204.0/s 36.4MB/s
Completed 355610 of 1048576 requests, 33.9% 35819.4/s 36.1MB/s
Completed 392916 of 1048576 requests, 37.5% 37307.3/s 37.6MB/s
Completed 430817 of 1048576 requests, 41.1% 37869.5/s 38.1MB/s
Completed 469295 of 1048576 requests, 44.8% 38509.7/s 38.8MB/s
Completed 507098 of 1048576 requests, 48.4% 37806.6/s 38.1MB/s
Completed 545584 of 1048576 requests, 52.0% 38482.4/s 38.7MB/s
Completed 581981 of 1048576 requests, 55.5% 36385.9/s 36.6MB/s
Completed 619513 of 1048576 requests, 59.1% 37511.1/s 37.8MB/s
Completed 656630 of 1048576 requests, 62.6% 37132.1/s 37.4MB/s
Completed 695668 of 1048576 requests, 66.3% 39037.8/s 39.3MB/s
Completed 733126 of 1048576 requests, 69.9% 37453.8/s 37.7MB/s
Completed 771071 of 1048576 requests, 73.5% 37969.4/s 38.2MB/s
Completed 809031 of 1048576 requests, 77.2% 37918.8/s 38.2MB/s
Completed 818004 of 1048576 requests, 78.0% 8978.2/s 9.0MB/s
Completed 818004 of 1048576 requests, 78.0% 0.0/s 0.0MB/s
Completed 818004 of 1048576 requests, 78.0% 0.0/s 0.0MB/s
Completed 818004 of 1048576 requests, 78.0% 0.0/s 0.0MB/s
Completed 818004 of 1048576 requests, 78.0% 0.0/s 0.0MB/s
Completed 818004 of 1048576 requests, 78.0% 0.0/s 0.0MB/s
Completed 820463 of 1048576 requests, 78.2% 2458.8/s 2.5MB/s
Completed 855290 of 1048576 requests, 81.6% 34837.5/s 35.1MB/s
Completed 892831 of 1048576 requests, 85.1% 37512.3/s 37.8MB/s
Completed 929307 of 1048576 requests, 88.6% 36504.0/s 36.7MB/s
Completed 966138 of 1048576 requests, 92.1% 36830.1/s 37.1MB/s
Completed 1002939 of 1048576 requests, 95.6% 36743.5/s 37.0MB/s
Completed 1039885 of 1048576 requests, 99.2% 36994.3/s 37.2MB/s

Concurrency Level:      100
Time taken for tests:   35.242 seconds
Complete requests:      1048576
Failed requests:        0
Total transferred:      1106772191 bytes
Requests per second:    29753.77 [#/sec]
Transfer rate:          30669.06 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        0.3      3.3       6040.2      57.1

Percentage of the requests served within a certain time (ms)
   50%      2.0 ms
   66%      2.6 ms
   75%      3.3 ms
   80%      3.9 ms
   90%      5.2 ms
   95%      5.9 ms
   98%      6.7 ms
   99%      7.3 ms
  100%    6040.2 ms

------------ Randomly Reading Benchmark ----------
Completed 89326 of 1048576 requests, 8.5% 89325.5/s 89.9MB/s
Completed 180890 of 1048576 requests, 17.3% 91491.9/s 92.1MB/s
Completed 283406 of 1048576 requests, 27.0% 102595.4/s 103.3MB/s
Completed 388306 of 1048576 requests, 37.0% 104896.3/s 105.6MB/s
Completed 494396 of 1048576 requests, 47.1% 106095.7/s 106.8MB/s
Completed 601020 of 1048576 requests, 57.3% 106623.0/s 107.3MB/s
Completed 704342 of 1048576 requests, 67.2% 103320.7/s 104.0MB/s
Completed 808570 of 1048576 requests, 77.1% 104230.1/s 104.9MB/s
Completed 913805 of 1048576 requests, 87.1% 105179.7/s 105.9MB/s
Completed 1019458 of 1048576 requests, 97.2% 105708.4/s 106.4MB/s

Concurrency Level:      100
Time taken for tests:   10.287 seconds
Complete requests:      1048576
Failed requests:        0
Total transferred:      1106769790 bytes
Requests per second:    101935.64 [#/sec]
Transfer rate:          105071.15 [Kbytes/sec]

Connection Times (ms)
              min      avg        max      std
Total:        0.1      0.9       15.1      0.9

Percentage of the requests served within a certain time (ms)
   50%      0.6 ms
   66%      0.9 ms
   75%      1.1 ms
   80%      1.3 ms
   90%      1.8 ms
   95%      2.4 ms
   98%      3.6 ms
   99%      4.8 ms
  100%     15.1 ms

测试结果汇总

测试环境	min	avg	max	std
本机 （写）	0.3	2.6	20.8	1.6
本机 （读）	0.1	0.8	14	0.9
本机 nfs （写）	0.3	2.8	1653.2	16.2
本机 nfs （读）	0	0.8	14.5	0.9
容器化 （写）	0.3	2.6	250	2.7
容器化 （读）	0.1	0.9	24.5	0.9
容器化 nfs （写）	0.3	3.3	6040.2	57.1
容器化 nfs （读）	0.1	0.9	15.1	0.9

测试结论

nfs 对weed 并发写入影响较大，本地和容器化相同条件下性能差别不大