Jenkins for PHP projects
Stephan Hochdörfer // June 23 2015
About me
Stephan Hochdörfer
Head of Technology , bitExpert AG (Mannheim, Germany)
S.Hochdoerfer@bitExpert.de
@shochdoerfer
#PHP, #DevOps, #Automation, #unKonf
Our story: Diversity
A lot of different projects with a lot of technology stacks.
Jenkins Use Cases
Running continuous build
Running nightly builds
Running integration tests
Running Satis builds
Continuous builds needs to run fast, just PHPUnit, CodeSniffer, PMD and such
Nightly builds check more things, e.g. run a full sencha app build
Integration builds test everything from the frontend up to the database
Satis is used for hosting our private Composer packages
Jenkins Project Overview
Jenkins Plugin: Custom Job Icon Plugin
Mostly used to better "organize" (categorize) our jobs
Icon Config Management
Jenkins Job Icon Config
Config File Management
Jenkins Plugin: Config File Provider Plugin
Jenkins Node Overview
Buildnodes for "unit tests" vs. Integrationnodes for "integration tests". Every job can run on a buildnode whiel just one particular job will tun on the integration build servers.
Code is checked out directly on the integration build servers, not copied around! Was easier this way.
Not easy to keep an overview, but there`s a solution to that...
Jenkins Node Config
The Jenkins law: If a job can run on multiple nodes Jenkins will always pick the node which has at least 1 of the required tools missing and thus the job will fail! Always."
Jenkins Node Environment
Did not want to whitelist the buildnodes in our Satis instance. Decided to configure username and password here.
Continuous build
« [...] environments automatically rebuild the project whenever changes are checked in - often several times a day - and provide more immediate feedback » - Wikipedia
Our typical (PHP) toolchain
Composer, Phing, PHPUnit,
PHP_CodeSniffer, PDepend, PHPCPD
Project structure
/ tmp/ myproject
|- build
|--- coverage
|--- logs
|--- pdepend
|- docs
|- src
|--- Vendor
|----- MyProject
|- tests
|--- Vendor
|----- MyProject
|- vendor
|- web
|- build.xml
|- composer.json
|- composer.lock
|- phpdoc.dist.xml
|- phpmd.xml
|- phpunit.xml.dist
Phing build.xml
<?xml version ="1.0" ?>
<project name ="ci" default ="ci:help" basedir ="." >
<target name ="-ci:clean"
depends ="-init"
hidden ="true" >
<delete dir ="${phing.dir}/build/logs" quiet ="true" />
</target>
<target name ="-ci:prepare"
depends ="-init, -ci:clean"
hidden ="true" >
<mkdir dir ="${phing.dir}/build/logs" />
</target>
<target name ="ci:build"
depends ="-init, -ci:prepare, ci:pdepend, ci:phpunit,
ci:phpcpd, ci:phpmd, ci:phpcs" />
</project>
Phing is Ant for PHP on steriods.
For most languages / tools we have a ci:build task which is triggered by Jenkins. The only exception are Puppet builds which are named ci_build. Consistency is very important for us!
Jenkins Job Config
Label Usage
Old: Jobs run on buildslave01 or buildslave02 or...
New: Jobs need a buildslave capable of providing featureA, featureB...
Environment configuration
Jenkins Plugin: Mask Passwords Plugin
Push config files to node
Jenkins Plugin: Config File Provider Plugin
We could have used Puppet or Chef for that but for me it made more sense to let Jenkins do the job as it is somehow part of the build process.
Validate composer.json
Installing dependencies
$COMPOSER_USER and $COMPOSER_PASS are populated from the node environment settings we defined before.
Jenkins Plugin: Mask Passwords Plugin
Running Phing
Will run PHP_CodeSniffer, PHPUnit, PDepend, ...
We do not use the Phing Jenkins plugin as the plugin is only capable of running a globally installed version of Phing. We like local dependencies and for that need to use the Shell commands.
Installing npm dependencies
Running Grunt
Grunt ist for Javascript what Phing is for PHP (but without the steriods)
Will run jscs, jshint, scss-lint
Process the build results
Checkstyle, PMD, Duplicate code results, Clover PHP Coverage Report, JUnit Report
Email notifications
Jenkins Plugin: Email-ext plugin
Email notifications
For the continious builds we send failure emails just on the 1st failure, but nightly builds or integration builds we send mails on every failure.
Trigger HHVM / PHP7 builds
For libraries we want to make sure that they are compatible with hhvm or the in-the-works version 7 of PHP.
Execute HHVM build
The Phing task will execute PHPUnit via the PHP interpreter which we do not want. Simple bugfix for the problem.
Trigger Satis build
Running nightly builds
« A nightly build is a neutral build that takes place automatically. These typically take place when no one is likely to be working in the office [...] » - Wikipedia
Runs the same tasks as the continous builds and some more, e.g. "compiling" js apps, or duing security checks.
Security issues?
Check composer.lock file
$ php security-checker.phar security: check ./ composer.lock
Security Check Report
~~~~~~~~~~~~~~~~~~~~~
Checked file : composer.lock
[ OK]
0 packages have known vulnerabilities
Security Checker Phing task
Automate with Jenkins
Running integration tests
« [...] is the phase in software testing in which individual software modules are combined and tested as a group. » - Wikipedia
-> Code is checked out directly on the integration build servers, not copied around!
Running Phing
Tests the service layer interaction with database access.
Running Grunt
Tests the frontend with Selenium connecting to the local instance.
Running Satis builds
Satis is a ultra-lightweight, static file-based version of packagist.
Installing Satis
$ composer.phar create- project composer/ satis -- stability= dev
{
"name" : "bitExpert Satis repo" ,
"homepage" : "https://satis.loc" ,
"repositories" : [
{ "type" : "vcs" , "url" : "/srv/repos/project-1.git" },
{ "type" : "vcs" , "url" : "/srv/repos/project-2.git" }
],
"require-all" : true ,
"archive" : {
"directory" : "dist" ,
"format" : "zip" ,
"skip-dev" : false
}
}
Git, Satis and Jenkis master run on the same box that`s why we use the local file paths to our git repos. No need to clone a remote.
Running Satis
$ php bin/ satis build config.json web/
$ php bin/ satis build config.json web/ customername/ project1
Jenkins Satis Job Config
Define Job Param (Package)
Jenkins Plugin: Parameterized Trigger Plugin
Define Job Param (Recipient)
Jenkins Plugin: Parameterized Trigger Plugin
Running Satis build
Rewrite repo urls
Git, Satis and Jenkis master run on the same box, that`s why we need to rewrite the paths here to be able to run composer with --prefer-source to clone the source repo.
Cleanup old files
Notify users after build
Jenkins Plugin: Email-ext plugin
Running the build
Console output
Jenkins Plugin: Mask Passwords Plugin
Satis repository
Run Composer
$ composer.phar require customername/ project1
Using version ^1.0 for customername/ project1
./ composer.json has been created
Loading composer repositories with package information
Updating dependencies ( including require- dev)
- Installing customername/ project1 ( 1.0.0 )
Downloading: 100 %
Writing lock file
Generating autoload files
Jenkins Plugins Reference