持续集成

gitlab + jenkins + salt持续集成

目的：

主要是了解持续集成方面的知识，分析gitlab的对于svn的优势，了解gitlab和jenkins的原理，再配合脚本和salt做一个简易的持续集成。

环境

我使用三台机器做测试：
三台机器系统都是centos7，selinux disabled，iptables disabled
192.168.46.66 jenkins  salt-master
192.168.46.82 gitlab 
192.168.46.105 nginx    salt-minion

该测试大致过程

通过提交代码到gitlab触发jenkins构建，jenkins调用脚本打包程序，通过salt分发包，并通过salt部署包

安装

1.1 安装jenkins

官网安装文档：https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins#InstallingJenkins-Unix%2FLinuxInstallation

我装的稳定版本：
1.1.1 添加Jenkins的源（repository）:
wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo
rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key
yum install jenkins

1.1.2 安装/检查JDK
[root@localhost jenkins]# yum install java -y
[root@localhost jenkins]# java -version
openjdk version "1.8.0_71"
OpenJDK Runtime Environment (build 1.8.0_71-b15)
OpenJDK 64-Bit Server VM (build 25.71-b15, mixed mode)

1.1.3 安装Jenkins：
[root@localhost jenkins]#yum install jenkins

遇到错误
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
jenkins-1.653-1.1.noarch.rpm   FAILED                                         
http://pkg.jenkins-ci.org/redhat-stable/jenkins-1.653-1.1.noarch.rpm: [Errno 14] HTTP Error 404 - Not Found     ]  0.0 B/s |    0 B  --:--:-- ETA
Trying other mirror.
[root@localhost ~]# yum install -y deltarpm
继续安装jenkins
[root@localhost jenkins]#yum install jenkins

如果有防火墙
firewall-cmd --zone=public --add-port=8080/tcp --permanent
firewall-cmd --zone=public --add-service=http --permanent
firewall-cmd --reload
firewall-cmd --list-all

/usr/lib/jenkins/：jenkins安装目录，WAR包会放在这里。
/etc/sysconfig/jenkins：jenkins配置文件，“端口”，“JENKINS_HOME”等都可以在这里配置
/var/lib/jenkins/：默认的JENKINS_HOME。
/var/log/jenkins/jenkins.log：Jenkins日志文件。

启动Jenkins
[root@localhost jenkins]# /sbin/chkconfig jenkins on
[root@localhost jenkins]# systemctl  start jenkins
1.1.4 安装salt-master
  1.1.4.1 导入key
    rpm --import https://repo.saltstack.com/yum/redhat/7/x86_64/latest/SALTSTACK-GPG-KEY.pub
  1.1.4.2 编辑源/etc/yum.repos.d/saltstack.repo
    [saltstack-repo]
    name=SaltStack repo for RHEL/CentOS $releasever
    baseurl=https://repo.saltstack.com/yum/redhat/$releasever/$basearch/latest
    enabled=1
    gpgcheck=1
    gpgkey=https://repo.saltstack.com/yum/redhat/$releasever/$basearch/latest/SALTSTACK-GPG-KEY.pub
  1.1.4.3 安装master
    yum install salt-master -y
  1.1.4.4 启动
    systemctl enable salt-master
    systemctl start salt-master

访问 192.168.46.66:8080 [attach]991[/attach] 1.2 gitlab安装

官网安装文档：https://www.elastic.co/guide/index.html

1.2.1 安装相关软件
yum install curl policycoreutils openssh-server openssh-clients
systemctl enable sshd
systemctl start sshd
yum install postfix
systemctl enable postfix
systemctl start postfix
firewall-cmd --permanent --add-service=http
systemctl reload firewalld

1.2.2 下载安装gitlab包
curl -LJO https://packages.gitlab.com/gitlab/gitlab-ce/packages/el/7/gitlab-ce-8.5.5-ce.0.el7.x86_64.rpm/download
rpm -i  gitlab-ce-8.5.5-ce.0.el7.x86_64.rpm

1.2.3 配置启动gitlab
gitlab-ctl reconfigure
gitlab-ctl --help 查看帮助  
[root@localhost ~]# gitlab-ctl service-list
gitlab-workhorse*
logrotate*
nginx*
postgresql*
redis*
sidekiq*
unicorn*

1.2.4 登陆 192.168.46.82
Username: root 
Password: 5iveL!fe
第一次登陆会让你修改密码

[attach]992[/attach] 1.3 nginx安装

1.3.1 安装nginx
    yum install -y nginx
    启动nginx
    nginx
1.3.2 安装salt-minion
  1.3.2.1 导入key
    rpm --import https://repo.saltstack.com/yum/redhat/7/x86_64/latest/SALTSTACK-GPG-KEY.pub
  1.3.2.2 编辑源/etc/yum.repos.d/saltstack.repo
    [saltstack-repo]
    name=SaltStack repo for RHEL/CentOS $releasever
    baseurl=https://repo.saltstack.com/yum/redhat/$releasever/$basearch/latest
    enabled=1
    gpgcheck=1
    gpgkey=https://repo.saltstack.com/yum/redhat/$releasever/$basearch/latest/SALTSTACK-GPG-KEY.pub
  1.3.2.3 安装master
    yum install salt-master -y
  1.3.2.4 启动
    systemctl enable salt-minion
    systemctl start salt-minion

配置

2.1 配置46.105 nginx

2.1.1 配置nginx 
    我的代码会发布到/usr/share/nginx/html/demo目录中
    [root@nginx ~]# vim /etc/nginx/nginx.conf
     server {
    listen       80 default_server;
    listen       [::]:80 default_server;
    server_name  _;
    root         /usr/share/nginx/html/demo;
    # Load configuration files for the default server block.
    include /etc/nginx/default.d/*.conf;
    location / {
    }
    error_page 404 /404.html;
        location = /40x.html {
    }
    error_page 500 502 503 504 /50x.html;
        location = /50x.html {
    }
    }
重启nginx
nginx -s reload
2.1.2 配置salt-minion
    [root@nginx ~]# vim /etc/salt/minion
    master: 192.168.46.66
    id: nginx
    重启minion
    systemctl restart salt-minion 

    在46.66 salt-master 加入nginx
    salt-key -a nginx

2.2 配置46.82 gitlab

2.2.1 创建一个demo项目

[attach]993[/attach]

2.2.2 在46.66上创建ssh key  
把公钥拷贝过来，私钥会拷到jenkins上，就可以无密码通信了

[attach]994[/attach]

2.2.3 配置gitlab的web hooks
当有push时会触发jenkins构建
url为： http://192.168.46.66:8080/git/notifyCommit?url=git@192.168.46.82:root/demo.git
/git/notifyCommit?url=前为jenkins的地址和端口
/git/notifyCommit?url=后为git项目地址
2.2.4 配置salt master
vim /etc/salt/master
file_roots:
  base:
    - /srv/salt
重启salt-master
systemctl restart salt-master

[attach]995[/attach] 2.2 配置46.66 jenkins

2.2.1 配置jenkins 
系统设置

[attach]996[/attach] [attach]997[/attach] 创建项目 [attach]998[/attach] 添加私钥 [attach]999[/attach] [attach]1000[/attach] [attach]1001[/attach]

脚本放在/project
[root@jenkins01 project]# ls
code  config  deploy.sh  package
code 代码目录
config 配置文件目录
package 代码打包后存放目录

deploy.sh  jenkins构建时调用脚本
[root@jenkins01 project]# cat deploy.sh 
#!/usr/bin/bash
###########[size=16]#[/size]
#author:CM
#mail:kakaximeng@163.com
#########[size=16]#[/size]
[size=16]提前建立相应的目录[/size]
LOCK_FILE="/project/lock"
CODE_PATH="/project/code"
DEMO_PATH="/project/code/demo"
PACKAGE_PATH="/srv/salt/package"
SALT_PACKAGE_PATH="package"
CONFIG_PATH="/project/config"
LOG_FILE="/project/deploy.log"
DEST_PROJECT="/project"
CTIME=$(date "+%Y-%m-%d-%H-%M")


Usage(){
  echo "Usage: $0 ( deploy | list | rollback edition)"
}


Lock_judge(){
  if [ -f $LOCK_FILE ];then
    echo "warnning !!! someone else must deploy new version or rollback old version! please wait."
    exit 2
  else
    touch -f $LOCK_FILE
  fi    
}

Delete_lock_file(){
  if [ -f $LOCK_FILE ];then
    rm -f  $LOCK_FILE
  fi
}

Deploy_code(){
  Lock_judge
  tag=$?
  echo $tag
  if [ $tag == 0 ];then
    cd $DEMO_PATH
    git pull
    Commit=$(git log|head -n 1|cut -d ' ' -f 2)
    Com_ID=$(echo ${Commit:0:6})
    cp -r $DEMO_PATH ${PACKAGE_PATH}/demo_${Com_ID}_${CTIME}
    cp -r ${CONFIG_PATH}/* ${PACKAGE_PATH}/demo_${Com_ID}_${CTIME}/
    cd $PACKAGE_PATH
    tar -zcf demo_${Com_ID}_${CTIME}.tar.gz demo_${Com_ID}_${CTIME}
    salt "nginx" cp.get_file  salt://${SALT_PACKAGE_PATH}/demo_${Com_ID}_${CTIME}.tar.gz ${DEST_PROJECT}/demo_${Com_ID}_${CTIME}.tar.gz
    salt "nginx" cmd.run "cd ${DEST_PROJECT} && tar -zxf demo_${Com_ID}_${CTIME}.tar.gz && rm -f /usr/share/nginx/html/demo && ln -s ${DEST_PROJECT}/demo_${Com_ID}_${CTIME} /usr/share/nginx/html/demo && nginx -s reload"
  fi
  Delete_lock_file
}

Edition_list(){
  salt 'nginx' cmd.run "ls -l ${DEST_PROJECT}|grep ^d"
}

Rollback_code(){
  Lock_judge
  salt 'nginx' cmd.run "rm -f /usr/share/nginx/html/demo && ln -s ${DEST_PROJECT}/$1 /usr/share/nginx/html/demo"
  Delete_lock_file
}

main(){
  case $1 in
    deploy)
      Deploy_code
      ;;
    list)
      Edition_list
      ;;
    rollback)
      Rollback_code $2
      ;;
    *)
      echo $1
      Usage
      ;;
  esac
}

main $1 $2

#脚本没有完善，没有写日志

测试

我之前已经推上去代码

[attach]1002[/attach] [attach]1003[/attach]

vim index.html
Adam Jensen
  今晚上吃
  西红柿鸡
  西红牛腩
git add index.html 
git commit -m "牛腩"
git push origin master

[attach]1004[/attach] [attach]1005[/attach]

回滚
[root@jenkins01 project]# ls
code  config  deploy.sh  package
[root@jenkins01 project]# sh deploy.sh list
nginx:
    drwxr-xr-x 7 root root     129 Mar 23 09:28 demo_205d03_2016-03-23-09-28
    drwxr-xr-x 7 root root     129 Mar 23 17:04 demo_3c2a63_2016-03-23-17-04
    drwxr-xr-x 7 root root     129 Mar 22 09:21 demo_43fabe_2016-03-22-09-21
    drwxr-xr-x 7 root root     151 Mar 21 17:41 demo_4c5be1_2016-03-21-17-41
    drwxr-xr-x 7 root root     151 Mar 21 17:48 demo_4c5be1_2016-03-21-17-48
    drwxr-xr-x 7 root root     119 Mar 21 14:46 demo_574cdd_2016-03-21-14-46
    drwxr-xr-x 7 root root     119 Mar 21 14:48 demo_574cdd_2016-03-21-14-48
    drwxr-xr-x 7 root root     119 Mar 21 14:49 demo_574cdd_2016-03-21-14-49
    drwxr-xr-x 7 root root     119 Mar 21 14:56 demo_574cdd_2016-03-21-14-56
    drwxr-xr-x 7 root root     129 Mar 21 15:59 demo_8c7d37_2016-03-21-15-59
    drwxr-xr-x 7 root root     129 Mar 23 17:22 demo_8cc9b4_2016-03-23-17-22
    drwxr-xr-x 7 root root     129 Mar 21 15:41 demo_a2333d_2016-03-21-15-41
    drwxr-xr-x 7 root root     129 Mar 23 17:03 demo_d5f4f3_2016-03-23-17-03

[root@jenkins01 project]# sh deploy.sh rollback  demo_3c2a63_2016-03-23-17-04

[attach]1006[/attach]

0

2016-03-28

2 个评论

Budweiser小王子

能否提供demo项目的源代码，进行测试

欺壹世

写的很棒，借鉴了，只不过为什么我的jenkins构建那里没有你勾选的那个 build when a change....的选项，附近的几个也都没有，我的jenkins版本是2.32.2 是版本的问题么？？

要回复文章请先登录或注册

gitlab + jenkins + salt持续集成

Adam Jensen

今晚上吃

西红柿鸡

西红牛腩

2 个评论

发起人

OpenSkill 专业的开源技术学习问答平台

gitlab + jenkins + salt持续集成

Adam Jensen

今晚上吃

西红柿鸡

西红牛腩

2 个评论

发起人

相关问题

OpenSkill 专业的开源技术学习问答平台