Gearmand学习1：安装及使用

最近因为项目需要，要用到任务分发框架。为了避免重复造轮子，gearmand就进入了视线。gearmand的工作原理，请参见gearman，或者中文版gearman使用。为了能够更好的使用gearmand，最好的办法就是能够了解它，深入的了解它。下面我们首先来看看gearmand如何安装以及使用。

1、gearmand安装

依赖库：boost（1.39版本以上）、libevent、gperf（不确定是否已经安装，最简单的方法就是直接安装gearmand，如果有失败提示在根据失败提示进行相应的操作）
安装过程
-下载最新版的gearmand源码包，源码地址gearmand-1.1.14.tar.gz，其他版本参见github
-解压缩安装
tar -xvzf gearmand-1.1.14.tar.gz
cd gearmand-1.1.14
./configure
make
sudo make install

祝你好运。
这里可能会遇到一个问题，就是boost已经安装，却依然提示boost未安装或者版本过低，不用惊慌，这里只需要加上相应的软连接即可（boost的安装路径根据实际情况进行调整）：
ln -s $BOOST_INCLUDE /usr/local/include/boost
ln -s $BOOST_LIB/libboost_program_options.so /usr/lib/libboost_program_options.so
ln -s $BOOST_LIB/libboost_program_options.so.1.53.0 /usr/lib/libboost_program_options.so.1.53.0

2.gearmand启动

到gearmand的bin目录下，这里是
cd /usr/local/gearman/sbin
gearmand的启动选项如下：

选项	选项含义
General options:
-b [ –backlog ] arg (=32)	Number of backlog connections for listen.
-d [ –daemon ]	Daemon, detach and run in the background.
–exceptions	Enable protocol exceptions by default.
-f [ –file-descriptors ] arg	Number of file descriptors to allow for the process (total connections will be slightly less). Default is max allowed for user.
-h [ –help ]	Print this help menu.
-j [ –job-retries ] arg (=0)	Number of attempts to run the job before the job server removes it. This is helpful to ensure a bad job does not crash all available workers. Default is no limit.
–job-handle-prefix arg	Prefix used to generate a job handle string. If not provided, the default “H:” is used.
–hashtable-buckets arg (=991)	Number of buckets in the internal job hash tables. The default of 991 works well for about three million jobs in queue. If the number of jobs in the queue at any time will exceed three million, use proportionally larger values (991 * # of jobs / 3M). For example, to accomodate 2^32 jobs, use 1733003. This will consume ~26MB of extra memory. Gearmand cannot support more than 2^32 jobs in queue at this time.
–keepalive	Enable keepalive on sockets.
–keepalive-idle arg (=-1)	If keepalive is enabled, set the value for TCP_KEEPIDLE for systems that support it. A value of -1 means that either the system does not support it or an error occurred when trying to retrieve the default value.
–keepalive-interval arg (=-1)	If keepalive is enabled, set the value for TCP_KEEPINTVL for systems that support it. A value of -1 means that either the system does not support it or an error occurred when trying to retrieve the default value.
–keepalive-count arg (=-1)	If keepalive is enabled, set the value for TCP_KEEPCNT for systems that support it. A value of -1 means that either the system does not support it or an error occurred when trying to retrieve the default value.
-l [ –log-file ] arg (=/usr/local/gearmanar/log/gearmand.log)	Log file to write errors and information to. If the log-file parameter is specified as ‘stderr’, then output will go to stderr. If ‘none’, then no logfile will be generated.
-L [ –listen ] arg	Address the server should listen on. Default is INADDR_ANY.
-P [ –pid-file ] arg (=/usr/local/gearman/var/gearmand.pid)	File to write process ID out to.
-r [ –protocol ] arg	Load protocol module.
-R [ –round-robin ]	Assign work in round-robin order per worker connection. The default is to assign work in the order of functions added by the worker.
-q [ –queue-type ] arg (=builtin)	Persistent queue type to use.
–config-file arg (=/usr/local/gearman/etc/gearmand.conf)	Can be specified with ‘@name’, too
–syslog	Use syslog.
–coredump	Whether to create a core dump for uncaught signals.
-t [ –threads ] arg (=4)	Number of I/O threads to use, 0 means that gearmand will try to guess the maximum number it can use. Default=4.
-u [ –user ] arg	Switch to given user after startup.
–verbose arg (=ERROR)	Set verbose level (FATAL, ALERT, CRITICAL, ERROR, WARNING, NOTICE, INFO,DEBUG).
-V [ –version ]	Display the version of gearmand and exit.
-w [ –worker-wakeup ] arg (=0)	Number of workers to wakeup for each job received. The default is to wakeup all available workers.
HTTP:
–http-port arg (=8080)	Port to listen on.
Gear:
-p [ –port ] arg (=4730)	Port the server should listen on.
–ssl	Enable ssl connections.
–ssl-ca-file arg	CA file.
–ssl-certificate arg	SSL certi ficate.
–ssl-key arg	SSL key for certificate.
builtin:
MySQL:
–mysql-host arg (=localhost)	MySQL host.
–mysql-port arg (=3306)	Port of server. (by default 3306)
–mysql-user arg	MySQL user.
–mysql-password arg	MySQL user password.
–mysql-db arg	MySQL database.
–mysql-table arg (=gearman_queue)	MySQL table name.

gearmand性能问题中使用到了多线程模型，其线程分为三种：
-Listening and management thread 监听和管理线程： 只有一个，主要负责接受连接请求，并分配给I/O线程
-I/O Thread I/O 线程 － 可以有很多个：主要负责读写系统调用，解析数据包
-Processing Thread 处理线程 ： 只有一个，负责管理列表和Hash表，管理唯一Key、任务的Handle、函数、任务队列
参考这些选项，我们就可以启动gearmand了，开启我们gearman学习之旅。下面我们先来练习一个小例子。
这里启动命令是 ./gearmand -d -p 4730

3.gearman使用实例

server端的服务已部署完毕，那么我们只需要搞定client和worker就可以了！这里将采用python作为开发语言，所以需要安装python-gearman。安装方式多样，pip、python包等，各位大神各显神通了。
-easy_install gearman
-pip install gearman
-源码安装：
wget https://pypi.python.org/packages/source/g/gearman/gearman-2.0.2.tar.gz –no-check-certificate
tar zxvf gearman-2.0.2.tar.gz
cd gearman-2.0.2
python setup.py install

在给出具体的例子之前，这里先简单回顾下gearman的工作流程：
一个Gearman请求的处理过程涉及三个角色：Client -> Job -> Worker。
-Client：请求的发起者，可以是 C，PHP, Perl，MySQL, UDF 等等。
-Job：请求的调度者，用来负责协调把 Client 发出的请求转发给合适的 Worker。
-Worker：请求的处理者，可以是 C，PHP，Perl 等等。

下面例子来自于gearman的安装启动以及python API入门例子
首先，编写worker端代码worker.py：

#!/usr/bin/env python

import os
import gearman
import math
class MyGearmanWorker(gearman.GearmanWorker):
    def on_job_execute(self, current_job):
        print "Job started"
        print "===================\n"
        return super(MyGearmanWorker, self).on_job_execute(current_job)

def task_callback(gearman_worker, gearman_job):
    print gearman_job.data
    print "-----------\n"
    return gearman_job.data

my_worker = MyGearmanWorker(['127.0.0.1:4730'])
my_worker.register_task("echo", task_callback)
my_worker.work()

然后编写client端代码client.py：

#!/usr/bin/env python
from gearman import GearmanClient

gearman_client = GearmanClient(['127.0.0.1:4730'])
gearman_request = gearman_client.submit_job('echo', 'test gearman')

result_data = gearman_request.result
print result_data

注意上面的ip和port的设置，根据实际情况设置就可以了。
这里对submit_job接口简单进行下说明，submit_job定义：
submit_job(task, data, unique=None, priority=None, background=False, wait_until_complete=True, max_retries=0, poll_timeout=None)
这里需要注意两个参数background和wait_until_complete。
当background设置为True时，client调用这个接口后将直接返回，并且不会关注该job的任何结果。
wait_until_complete设置为False时，client调用这个接口后虽然也会直接返回，但是后面我们可以通过wait_until_jobs_completed接口获取job的执行结果。
有关gearman python更多的细节，请移步python-gearman 2.x doc
现在开始最重要测试实现了,运行下面命令:(需要开启两个终端）
python worker.py
python client.py

看到client和worker的输出，恭喜您已经会简单的使用gearman了！