全docker化gitlab及CI

孙言

2023-12-01

##Docker安装

curl -fsSL https://get.docker.com/ | sudo sh  # 安装最新版的docker
sudo mkdir /etc/docker
echo '{"registry-mirrors": ["https://docker.mirrors.ustc.edu.cn"]}' | sudo cat > /etc/docker/daemon.json  # 更改为使用中科大的源 
sudo systemctl enable docker.service  # 设置开机启动
sudo systemctl start docker  # 立即启动服务

# docker用户组管理
# 通过将用户加入到docker组，可以省去后期该用户在docker命令时的sudo前缀（docker实现的便利机制）
sudo groupadd docker
sudo usermod -aG docker $(whoami)
exit  # 退出从而重新加载用户配置

# docker-compose安装
sudo curl -L "https://github.com/docker/compose/releases/download/1.9.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
docker-compose --version

df -hl  # 查看磁盘使用空间

创建HTTPS证书

权威CA机构仅对域名签发证书，这里由于备案尚未通过，临时使用**基于IP的自签名证书**来处理
sudo mkdir /etc/certs
cd /etc/certs

# 自签名证书需要有个备选的IP SANS校验信息
sudo sed -i '/\[ v3_ca \]/a\subjectAltName = IP:121.40.145.xxx' /etc/pki/tls/openssl.cnf

# 创建证书
sudo openssl req -newkey rsa:4096 -nodes -sha256 -keyout domain.key -x509 -days 365 -out domain.crt  # 其中Common Name输入ip地址：121.40.145.xxx
sudo openssl dhparam -out dhparam.pem 1024  # 生成DHE参数文件增强ssl加密

# 服务器自身对该证书的信任配
sudo mkdir -p /etc/docker/certs.d/121.40.145.xxx:4567
sudo cp /etc/certs/domain.crt  /etc/docker/certs.d/121.40.145.xxx:4567/


**客户端下做如下配置**
# 配置客户端docker信任服务端的自签名证书
# 服务端证书拷贝至客户端docker的根证书信任目录
sudo mkdir -p /etc/docker/certs.d/139.196.5.xxx:4567
sudo scp -P 8022 root@121.40.145.xxx:/etc/certs/domain.crt /etc/docker/certs.d/139.196.5.xxx:4567/

# 自签名证书网站地址栏被Chrome标记为不安全
# 我们需要手动将自签名证书导入Chrome的根证书信任列表
# 本地打开一个shell将服务器上的证书下载到本地
scp -P 8022 root@139.196.5.xxx:/etc/certs/domain.crt .
# 然后配置Chrome： 设置 》 高级 》 HTTPS/SSL 》 管理证书 》 受信任的根证书颁发机构 》 导入


#####################gitlab######################
version: '2'
services:
  Gitlab:
    image: 'twang2218/gitlab-ce-zh:8.17.4' //中文
   # image: 'gitlab/gitlab-ce:8.17.4-ce.0' //英文
    container_name: 'gitlab'
    hostname: 'repo.sharemeiti.com'
    restart: always
    ports:
      - '1022:22'
      - '1080:80'
      - '443:443'
      - '4567:4567'
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        # Add any other gitlab.rb configuration here, each on its own line
        external_url 'https://139.196.5.206'
        gitlab_rails['gitlab_shell_ssh_port'] = 22
        nginx['redirect_http_to_https'] = true
        nginx['ssl_dhparam'] = "/etc/gitlab/ssl/dhparam.pem"
        nginx['ssl_certificate'] = "/etc/gitlab/ssl/domain.crt"
        nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/domain.key"
        # nginx['custom_gitlab_server_config'] = "location ^~ /.well-known {\n alias /var/opt/gitlab/letsencrypt/.well-known;\n}\n"
        high_availability['mountpoint'] = ["/etc/gitlab", "/var/log/gitlab", "/var/opt/gitlab"]  # 严格限定gitlab服务启动前，指定文件系统挂载完毕

        #Registry配置
        registry_external_url "https://139.196.5.206:4567"  # ContainerRegistry的外部访问地址
        registry_nginx['ssl_certificate'] = "/etc/gitlab/ssl/domain.crt"
        registry_nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/domain.key"
        gitlab_rails['registry_host'] = "139.196.5.206"
        gitlab_rails['registry_port'] = "4567"
        gitlab_rails['registry_api_url'] = "http://localhost:5000" #docker官方5000
        gitlab_rails['gitlab_default_projects_features_builds'] = false #默认关闭CI
        gitlab_rails['gitlab_default_projects_features_container_registry'] = false #关闭Registry 
        #邮件配置
        gitlab_rails['smtp_enable'] = true
        gitlab_rails['smtp_address'] = "smtp.126.com"
        gitlab_rails['smtp_port'] = 25
        gitlab_rails['smtp_user_name'] = "hr3685930@126.com"
        gitlab_rails['smtp_password'] = "wyj15881894988"
        gitlab_rails['smtp_authentication'] = "login"
        gitlab_rails['smtp_enable_starttls_auto']= true
        gitlab_rails['gitlab_email_from']= 'hr3685930@126.com'
        gitlab_rails['gitlab_email_reply_to']= '599420081@qq.com'
    volumes:
      - /docker/gitlab/config:/etc/gitlab
      - /docker/gitlab/logs:/var/log/gitlab
      - /docker/gitlab/data:/var/opt/gitlab
      - /docker/gitlab/certs:/etc/gitlab/ssl


############################################

Gitlab调优 gitlab对内存资源的消耗比较厉害 其中尤以 sidekiq队列 及 unicorn服务 两个组件对内存消耗最多 可以再容器启动时对相关参数进行微调： 
        unicorn['worker_processes'] = 1 
        unicorn['worker_memory_limit_min'] = "300 * 1 << 20"
        unicorn['worker_memory_limit_max'] = "400 * 1 << 20"
        unicorn['worker_timeout'] = 15
        sidekiq['concurrency'] = 10
        sidekiq_cluster['enable'] = false
        sidekiq_cluster['ha'] = false
        redis['maxclients'] = "100"
        nginx['worker_processes'] = 2 
        nginx['worker_connections'] = 512 
        nginx['keepalive_timeout'] = 300 
        nginx['cache_max_size'] = '200m'
        mattermost['enable'] = false
        mattermost_nginx['enable'] = false
        gitlab_pages['enable'] = false
        pages_nginx['enable'] = false
        postgresql['shared_buffers'] = "256MB"
        postgresql['max_connections'] = 30
        postgresql['work_mem'] = "8MB"
        postgresql['maintenance_work_mem'] = "16MB"
        postgresql['effective_cache_size'] = "1MB"
        postgresql['checkpoint_timeout'] = "5min"
        postgresql['checkpoint_warning'] = "30s"


配置调整后需要重载一下
docker exec gitlab gitlab-ctl reconfigure
docker-compose down
docker-compose up -d

regitry 必须用root才能push 并且项目里配置该项目路径 和打开注册表

##日常维护命令

# Gitlab维护
docker exec gitlab gitlab-ctl status  # gitlab各组件服务状态
docker exec gitlab gitlab-ctl start/restart/stop [组件名]  # gitlab所有组件的统一控制（其中Unicorn组件重启完成前GitLab会报502）
docker exec gitlab gitlab-ctl tail [/var/log/gitlab下的某子目录]  # 实时查看日志

docker exec gitlab update-permissions  # 修复gitlab版本升级后出现的权限问题
docker exec gitlab gitlab-ctl reconfigure  # 重载配置
docker exec -t gitlab gitlab-rake gitlab:backup:create  # 创建备份

# ContainerRegistry维护
docker exec gitlab gitlab-ctl registry-garbage-collect  # 垃圾回收，清理废弃layer（registry停机）


# 账号密码若存在特殊字符则需要url编码
https://username:password@host:port/group/project.git

##RUNNER

###Gitlab Ci由三部分构成

gitlab-web版本库管理系统
gitlab-ci-server持续集成组件（已集成到gitlab-web）
gitlab-ci-runner持续集成运行机（默认执行build、test、deploy三阶段的job） ###Gitlab CI中build job主要工作流
Ci Runner运行机启动Ci Executor执行器
如有 before_script 则向 executor 发送该预处理脚本
在executor 中检出项目代码，切换至触发build的代码分支
执行build job中声明的script
检查构建的 exit status，若有非0返回码则build失败
销毁当前构建环境

###Ci-Runner 共享方式分类有如下2种

shared共享型runner
specific专享型runner Executor类型分类有如下6种
Shell：配置最简，直接以gitlab-runner账号在runner中检出项目代码build
docker：在build container中检出项目代码build, 和runner同级，都驻留在宿主机中
Docker Machine and Docker Machine SSH (auto-scaling)
Parallels
VirtualBox
SSH
Kubernetes

##############################
version: '2'

services:
  Ci-Runner:
    image: gitlab/gitlab-runner:alpine-v1.11.1
    container_name: ci_runner
    restart: always
    volumes:
      - /docker/gitlab-runner/config:/etc/gitlab-runner
      - /var/run/docker.sock:/var/run/docker.sock
###############################

注册Runner至Ci-Server

runner中可注册多个executor执行器给Ci-Server
具体注册信息保存在容器内 /etc/gitlab-runner/config/config.toml

##############toml#################
concurrent = 1
check_interval = 0

[[runners]]
  environment = ["DOCKER_AUTH_CONFIG={\"auths\":{\"139.196.5.206:4567\":{\"auth\":\"cm9vdDp3eWoxNTg4MTg5NDk4OA==\"}}}"]
  name = "test"
  url = "https://139.196.5.206:443"
  token = "5fd4b7a8f6dda1e96858a8ade94e5e"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "docker:latest"
    privileged = false
    disable_cache = false
    volumes = ["/cache","/var/run/docker.sock:/var/run/docker.sock"]
  [runners.cache]
##################################
自签名证书信任
# 如果Ci-Server使用了自签名证书，需要让runner信任Ci-Server的证书
sudo mkdir -p /srv/gitlab-runner/config/certs/
sudo scp -P 8022 root@121.40.xxx.yyy:/etc/certs/domain.crt /srv/gitlab-runner/config/certs/121.40.xxx.yyy.crt

###Runner-Executor注册

docker exec -it ci_runner gitlab-runner register  
#注册步骤
1. 填入gitlab-web的url地址，这里填 https://121.40.xxx.yyy:8443
2. 填入gitlab-web admin中提供的 Registration token
3. 填入runner介绍，这里填 executor1 in production
4. 填入runner的tags标签，这里 留空直接回车
5. 填入默认executor类型，这里填 docker
6. 填入镜像名称，这里填 docker:latest（具体构建时使用的executor镜像类型可在.gitlab-ci.yml中通过image指令调整）
7. 操作完成后，在gitlab-web的runner管理中，将显示当前创建的runner

# 若要删除runner
gitlab-ci-multi-runner unregister --url gitlab地址（带http和端口号） --token runner令牌

###Runner访问Private Registry的授权配置

runner宿主机上做一次登录操作 docker login https://121.40.xxx.yyy:4567 --username root --password my_password
拷贝文件~/.docker/config.json中的授权信息
在文件/srv/gitlab-runner/config/config.toml中runners节区下添加环境变量 environment = ["DOCKER_AUTH_CONFIG={"auths":{"registory服务地址":{"auth":"令牌"}}}"]
在config.toml中volumes变量中增加挂载 "/var/run/docker.sock:/var/run/docker.sock"
docker-compose重启

###Runner中SSH访问部署环境

ssh-keygen生成秘钥对，并手动ssh登录一次部署环境
部署环境上以root账号docker login一次我们的Container Registry
Project->CI/CD Pipelines->Secret Variables 添加变量
- id_rsa变量为 id_rsa文件内容
- known_hosts变量为known_hosts文件内容
.gitlab-ci.yml配置中增加ssh私钥导入处理before_script:
- mkdir ~/.ssh && cd ~/.ssh
- echo -e "$id_rsa" > id_rsa && chmod 600 id_rsa
- echo -e "$known_host" > known_hosts && chmod 644 known_hosts
- cd - 
将公钥添加至部署环境的ssh authorized_keys中
若build job中有git操作，则需将公钥添加至项目deploy key

Poject Settings -> Project Visibility -> Repository -> Pipelines 启用 Only team members
Poject Settings -> Runners 启用 Allow shared Runners

###自动化测试配置

phpcs配置
phpmd配置
phpunit配置

###持续集成工作流定义.gitlab-ci.yml 项目根目录的 .gitlab-ci.yml 文件可详细定义CI的 pipeline 构建流程

项目每次push或merge时，gitlab-ci会检测是否存在 .gitlab-ci.yml 文件，若存在则将派发ci-runner执行build job
pipeline分多个stage，每个stage包括多个build job
若runner足够数量，则stage下的不同build job是并行的（但每个runner中的executor是串行的）
runner完成job后会创建一个build
构建任务的结果状态可通过badge徽章（/namespace/project/badges/branch/build.svg）来动态显示，徽章可直观的嵌入README文档

IOS持续集成方案工具集：Gitlab-CI + fastlanes & pgyer + testflight

docker 强制删除none镜像 docker rmi -f <IMAGE ID>

转载于:https://my.oschina.net/u/2544629/blog/1023455

全docker化gitlab及CI

创建HTTPS证书

相关阅读

相关文章

相关问答

相关文档