把你从写繁琐的Map-reduce Job中解放出来,写分布式跟写本地程序没两样,Scalding真真代表着先进生产力的方向啊
心动不如行动,赶紧装一个吧
1 安装JDK
2 安装Homebrew
ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"
brew update
brew doctor
方法1:
https://github.com/twitter/scalding/wiki/Scala-and-sbt-for-Homebrew-users
方法2:
3 安装sbt
brew install sbt
http://www.scala-lang.org/download/install.html
解压到 /usr/local/share目录下,重命名为scala
vi ~/.bash_profile
export SCALA_HOME=/usr/local/share/scala
export PATH=$PATH:$SCALA_HOME/bin
source ~/.bash_profile
然后再任意一个目录下都可以使用scala命令啦~\(≧▽≦)/~好了
scala> :quit
5 将Scalding源代码从git上弄下来
git clone https://github.com/twitter/scalding.git
查看源代码目录下的project/Build.scala中的scalaVersion,发现默认是支持
Scala 2.9.3的,更改
project/Build.scala的line17为
将将line153的java改成scala
下面这步可选,反正我没有成功
ln -s scripts/scald.rb $HOME/bin/
可以在.bash_profile中设置一下
export
SBT_OPTS=
"-XX:+CMSClassUnloadingEnabled -XX:PermSize=256M -XX:MaxPermSize=512M"
不然下面的过程可能会遇到OutOfMemoryError
Q) What happens if I get OutOfMemoryErrors when running "sbt assembly"?
A) Create ~/.sbtconfig with these options: SBT_OPTS="-XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:PermSize=256M -XX:MaxPermSize=512M"
下面可能是个十分之漫长的过程,第一次用了很久,第二次用校园网失败了,第三次校园网挂VPN等了一个下午update还没完,要哭了
./sbt update
#./sbt test # runs the tests; if you do 'sbt assembly' below, these tests, which are long, are repeated
./sbt assembly # creates a fat jar with all dependencies, which is useful when using the scald.rb script
4 安装Scala IDE for Eclipse with the Scala Worksheet
到这里下载Scala 2.10.x的Mac OS X Cocoa 64 bit版本
5 打开这个解压后伪装成正常Eclipse的家伙
挥洒下人生中第一个HelloWorld.scala
object HelloWorld extends App {
println("Hello, World!")
}
或者
object HelloWorld {
def main(args: Array[String]) {
println("Hello, World!")
}
}
6 添加Vim语法高亮支持
到https://github.com/scala/scala-dist下载工具,复制tool-support/src/vim里"indent", "syntax" and "ftdetect"文件夹到~/.vim目录下,给~/.vimrc加上一行syn on