02-datax案例01 stream2stream

测试案例1

• 需求
  从Stream流中读取数据并且把数据打印到控制台.
• 操作

./datax.py -r streamreader -w streamwriter

获取模板

{
    "job": {
        "content": [
            {
                "reader": {
                    "name": "streamreader", 
                    "parameter": {
                        "column": [], 
                        "sliceRecordCount": ""
                    }
                }, 
                "writer": {
                    "name": "streamwriter", 
                    "parameter": {
                        "encoding": "", 
                        "print": true
                    }
                }
            }
        ], 
        "setting": {
            "speed": {
                "channel": ""
            }
        }
    }
}

修改为

{
    "job": {
        "content": [
            {
                "reader": {
                    "name": "streamreader", 
                    "parameter": {
	                    "column": [
							{
								"type":"long",
								"value":"10"
							},
							{
								"type":"string",
								"value":"hello,Datax"
						], 
                        "sliceRecordCount": "10"
                    }
                }, 
                "writer": {
                    "name": "streamwriter", 
                    "parameter": {
                        "encoding": "UTF-8", 
                        "print": true
                    }
                }
            }
        ], 
        "setting": {
            "speed": {
                "channel": "1"
            }
        }
    }
}

执行

datax.py ../job/stream2stream.json

结果

mptCount[1] is started
100	hello Datax
100	hello Datax
100	hello Datax
100	hello Datax
100	hello Datax
100	hello Datax
100	hello Datax
100	hello Datax
100	hello Datax
100	hello Datax

......
2020-11-23 15:11:20.821 [job-0] INFO  JobContainer - 
任务启动时刻                    : 2020-11-23 15:11:10
任务结束时刻                    : 2020-11-23 15:11:20
任务总计耗时                    :                 10s
任务平均流量                    :               12B/s
记录写入速度                    :              1rec/s
读出记录总数                    :                  10
读写失败总数                    :                   0