linux bbcp路径不对,[20130109]使用bbcp传输文件.txt

[20130109]使用bbcp传输文件.txt

今天看了一些blog，发现一些blog提到了bbcp。使用它来传输大文件据说很快。

相关站点：

~abh/bbcp/bin/

~abh/bbcp/

--不行，可以自己下载编译：

git clone ~abh/bbcp/bbcp.git

cd bbcp

make

cp bin/amd64_linux26/bbcp /usr/local/bin/bbcp

1.开始测试：

一看文档，晕！这么多参数，使用简单一点看看:

~abh/bbcp/

bbcp -P 2 /data/orcl/users01.dbf oracle@remote_host:/tmp/users01.dbf

--提示，执行文件名明明在那里，为什么不行。直接进入/usr/local/bin目录执行，也一样。

bash: bbcp: command not found

bbcp: bbcp unexpectedly terminated on XXX.XXX.XXX.XXX

--仔细看了相关文档

bbcp assumes the remote system's non-interactive environment contains the path to the bbcp utility.

This can be determined by with the following command:

ssh remotesystem which bbcp

--也就是远端也需要这个一个执行文件。将bbcp拷贝过去。

#  bbcp -P 2 /data/orcl/users01.dbf oracle@remote_host:/tmp/users01.dbf

[e-mail]'s password:

bbcp: Creating /tmp/users01.dbf

bbcp: 130109 16:50:04  5% done; 55.7 MB/s

bbcp: 130109 16:50:06  15% done; 63.2 MB/s

bbcp: 130109 16:50:08  25% done; 66.1 MB/s

bbcp: 130109 16:50:10  36% done; 71.3 MB/s

bbcp: 130109 16:50:12  47% done; 72.8 MB/s

bbcp: 130109 16:50:14  55% done; 70.0 MB/s

bbcp: 130109 16:50:16  65% done; 70.7 MB/s

bbcp: 130109 16:50:18  74% done; 70.4 MB/s

bbcp: 130109 16:50:20  84% done; 70.4 MB/s

bbcp: 130109 16:50:22  95% done; 71.6 MB/s

--文件大小1.4G,大约20秒完成。

# du -sm /tmp/users01.dbf

1467    /tmp/users01.dbf

2.使用scp做一个比较:

scp     /data/orcl.20121016/users01.dbf oracle@192.168.101.209:/tmp/users01.dbf

users01.dbf 100% 1465MB  45.8MB/s   00:32

lftp也做了一个测试，稍微快一点，大约30秒。1536180224 bytes transferred in 30 seconds (48.95M/s)

3.另外bbcp有一个好处，如果拷贝文件已经存在，提示文件存在直接退出。

加入-f 参数

--force | -f

forces the copy by erasing the target prior to copying the source file. By default, if the target already exists for the

source file, the copy fails. See also the –K option modifier.

如果在拷贝文件时，在远程机器使用ps -elf | grep bbcp 看。

# ps -elf | grep bbcp | grep -v grep

0 S oracle    7398  7397  1  81   0 - 12251 wait   17:04 ?        00:00:00 bbcp SNK

1 S oracle    7427  7398 68  76   0 - 22495 184466 17:04 ?        00:00:13 bbcp SNK

--会有2个进程。

看了一些文档从调整性能讲重要的两个参数(我自己认为),我估计如果机器忙，带宽不足，也许会影响这个网络。

1.2.1        Window Size (-w)

The first and most important option is –w. This determines the TCP window size as well as the default I/O size.  The

default of 128Kis usually good enough for LAN and larger values are likely to hurt performance.

1.2.2        Streams (-s)

The -- s options is the second most important tuning parameter. This specifies the number of parallel TCP streams. A

na?ve explanation would say that streams can make up for not having a large enough window. The idea is that if you can’

t get enough packets moving in a single window, then create multiple windows and run them simultaneously. This is only

partially true. While multiple streams do provide multiple windows, multiple windows also parallelize traffic with

independent time-outs, re-transmissions, and greatly improved I/O overlapping. This is a cumulative effect that

dramatically increases the overall bandwidth utilization. But, too much of a good thing can also be bad, as you will see

below.

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/267265/viewspace-752395/，如需转载，请注明出处，否则将追究法律责任。