Apache Toree试用

1. 安装Jupyter、notebook、toree

pip install toree
pip install notebook
pip install jupyter

notebook安装失败时执行

yum install gcc libffi-devel python-devel openssl-devel

2.初始化一个toree包装的Spark Shell

jupyter toree install --spark_home=/opt/spark/  --spark_opts='--driver-memory 8g --executor-cores 4 --executor-memory 12g --total-executor-cores 20 --jars /opt/hive/lib/mysql-connector-java-5.1.35-bin.jar'

SPARK_OPTS='--name "weiwenda" --driver-memory 4g --executor-cores 2 --executor-memory 8g --total-executor-cores 10 --jars /opt/hive/lib/mysql-connector-java-5.1.35-bin.jar' jupyter notebook

3.配置jupyter notebook

jupyter notebook --generate-config
vim /root/.jupyter/jupyter_notebook_config.py

添加以下内容：

c.NotebookApp.allow_origin = '*'
c.NotebookApp.notebook_dir = u'/root/Documents'
c.NotebookApp.open_browser = False
c.NotebookApp.port = 8888
c.NotebookApp.token = u''
c.NotebookApp.ip = '202.117.16.188'

4.启动jupyter notebook

jupyter notebook

5.启动到screen中

#查看当前screen窗口
screen -ls
#创建新的screen窗口
screen -S screen_name
#detach某个screen窗口
screen -d screen_name
ctrl+a -> d #detach当前
#return某个screen窗口
screen -r screen_name
#杀死当前screen窗口
ctrl+a -> k

6.在ipynb文件中添加类依赖

%AddJar file:///root/Documents/wwd/Find_IL_EdgeResultAnalysis.jar

7. 访问地址

http://202.117.16.188:8888

8.官方参考文档

https://toree.incubator.apache.org/documentation/user/quick-start