Composer for Corporate Use
Stephan Hochdörfer // October 19 2016
About me
Stephan Hochdörfer
Head of Technology , bitExpert AG (Mannheim, Germany)
S.Hochdoerfer@bitExpert.de
@shochdoerfer
#PHP, #DevOps, #Automation
#phpugffm, #phpugmrn, #unKonf
Who of you has heard of Composer? Who uses it?
This is no intro to Composer, I assume you already worked with it and you maybe already maintain your own packages.
Our story: Diversity
A lot of different project with a lot of technology stacks.
Back in the days: Subversion
Long time Subversion user. Stayed with it for a very long time, mostly for one reason: svn:externals - On the one hand a simple, easy solution but on the other hand caused a lot of trouble for us (access rights management, IDE did not really show externals, no one was really aware of the dependencies).
No fixed dependencies
To be honest: We did not really deal with dependencies, we linked in some folders. That`s sth. completly different, no support for transitive dependencies!
Big waste of time
Took about 2hrs to setup a new project. 2-3 people "almost" new how to set-up a new project. So in most cases it was me setting everything up. Over and over again. Even wrote an Eclipse plugin to reduce the work. Yes, an Eclipse plugin!
No way out?
I knew we had to change our strategy concerning dependency management especially when we wanted to move to Git. We almost went the Maven way. But this would have resulted in way more trouble. Composer was the only real option.
How can Composer help?
What are the everyday challenges we are dealing with?
Clear dependencies
just look into composer.json or composer.lock and you know what`s going on.
Transitive dependencies?
Transitive dependencies?
Security issues?
upload your composer.lock file
Jenkins Job as nightly build to check for problems
Turned into a FriendsOfPHP github project last autumn
Check composer.lock file
$ php security-checker.phar security: check ./ composer.lock
Security Check Report
~~~~~~~~~~~~~~~~~~~~~
Checked file : composer.lock
[ OK]
0 packages have known vulnerabilities
Check composer.lock file
$ curl - H "Accept: application/json"
https:// security.sensiolabs.org/ check_lock - F lock= @./ composer.lock
[]
Roave Security Advisories
Be aware of licensing issues
Make your developers aware of the licensing problem. It is so easy to pick any package from packagist but does it ahere to your company policy or the policy of your customer?
Tracking package licenses
$ composer.phar licenses
Name: customer/ project
Version: 1.0.0
Licenses: proprietary
Dependencies:
phpoffice/ phpexcel 1.8.0 LGPL
pimple/ pimple v1.1.1 MIT
psr/ log 1.0.0 MIT
silex/ silex v1.2.1 MIT
symfony/ debug v2.5.4 MIT
symfony/ event- dispatcher v2.5.4 MIT
symfony/ http- foundation v2.5.4 MIT
symfony/ http- kernel v2.5.4 MIT
symfony/ routing v2.5.4 MIT
Perspective change
As Anthony said in the keynote this morning: Composer made the Component Revolution possible. And this is true. Composer changed the way I think about our projects and their dependencies
I always try to ask the question if there`s functionality which we can reuse in multiple projects? If so, create a small, reusable (more or less independant) package
Small problem domains
Choose the "perfect" toolset
As Anthony said this morning: Do not always use a full stack framework just because you can. Pick the libs that make the most sense for the project. Maybe a full-stack framework, maybe a microframework. Get rid of what you do not need. Less code, less bugs
Install tools like PHPUnit, Phing locally within your project. They are dependencies just like Symfony or ZF
Local dependencies FTW!
Globally vs. locally installed dependencies: Install tools like PHPUnit, Phing locally within your project. They are dependencies just like Symfony or ZF
Local dependencies FTW!
# https://github.com/dflydev/just-run-phpunit
PHPUNIT_PATH= vendor/ bin: bin
function phpunit {
TEST_PATHS=( $PHPUNIT_PATH )
for TEST_PATH in "${TEST_PATHS[@]}"
do
if [ - x "${TEST_PATH}/phpunit" ]
then
"${TEST_PATH}/phpunit" "$@" ; return $?
fi
done
PHPUNIT_ON_PATH= "$(type -P phpunit )"
if [ - n "${PHPUNIT_ON_PATH}" ]; then
"${PHPUNIT_ON_PATH}" "$@" ; return $?
else
echo "phpunit not found!" ; return 127
fi
}
Build your own packages
Libs that take advantage of our infrastructure or the infrastructure of our customers
Package design challenges
Mathias Noback: Principles of PHP Package Design
1. Simplify your project setup
Back in the subversion days it took quite a while to setup a project and configure svn:externals. With Composer this whole process is much smoother and can be done by any dev.
Set-up a new project
$ composer.phar create- project symfony/ framework- standard- edition
myproject
$ composer.phar create- project -- repository- url= https:// satis.loc
my/ installer myproject
Custom project installer
{
"name" : "my/project-installer" ,
"type" : "project" ,
"license" : "MIT"
}
$ composer.phar create- project -- repository- url= https:// satis.loc
my/ project- installer myproject
Speeds up your project setup
One can not only create an installer for a framework but also to set-up a default project structure for your companies` projects incl. default dependencies and Vagrant/Puppet setup for a new project. Just a matter of running vagrant up and the virtual machine is up and running and the developers start to get their work done.
Customizing customers apps
We often build applications for our customers who sell these application to their customers, usually with customizations. To make things easier for the dev teams we create custom installers on a per project basis
2. Standardize pkg layouts
Guide the devlopers to make it super easy to bootstrap a new library. Create a sample filesystem layout incl. the required dependencies, e.g. setup Sphinx for documentation, configure PHPUnit and PHPDocumentor...
3. Split app, maintain core lib
This is what drives a German really crazy ;)
Build one monolithic app, split it into several apps and share common functionality via a "core" package
Example: Manage assets
Example: Copy assets to htdocs folder (e.g. dependent package contains php and javascript)
Custom package events
(pre|post)-install-cmd
(pre|post)-update-cmd
(pre|post)-status-cmd
(pre|post)-package-install
(pre|post)-package-update
(pre|post)-package-uninstall
(pre|post)-autoload-dump
...
Create custom scripts
{
"scripts" : {
"post-update-cmd" : "My \\ Composer \\ PostUpdate::execute" ,
"post-install-cmd" : [
"My \\ Composer \\ PostInstall::warmCache" ,
"phpunit"
]
}
}
Developing in parallel?
1. Modify files in vendor folder and copy/rsync them to "the other repo"
2. Symlink vendor folder to "the other repo"
3. prefer-source checkout, modify and commit directly
4. Use git submodules
It`s a marathon, not a sprint!
Recommendation: Split as late as possible!
4. Private packages?
Hosting your packages
Satis is a ultra-lightweight, static file-based version of packagist.
Toran acts as a proxy for Packagist and GitHub.
Installing Satis
$ composer.phar create- project composer/ satis -- stability= dev
Running Satis
{
"name" : "My own repo" ,
"homepage" : "http://satis.mydomain.loc" ,
"repositories" : [
{ "type" : "vcs" , "url" : "http://git.mydomain.loc/repo1.git" },
{ "type" : "vcs" , "url" : "http://git.mydomain.loc/repo2.git" }
],
"require-all" : true ,
"archive" : {
"directory" : "dist" ,
"format" : "zip" ,
"skip-dev" : false
}
}
$ php bin/ satis build config.json web/
$ php bin/ satis build config.json web/ mycompany/ mypackage
Mirror packages
{
"name" : "My own repo" ,
"homepage" : "http://satis.mydomain.loc" ,
"require-dependencies" : true ,
"require-dev-dependencies" : true ,
"repositories" : [
{ "type" : "vcs" , "url" : "https://github.com/silexphp/Silex.git" }
],
"archive" : {
"directory" : "dist" ,
"format" : "zip" ,
"skip-dev" : false
}
}
Mirror packages
{
"name" : "My own repo" ,
"homepage" : "http://satis.mydomain.loc" ,
"require-dependencies" : true ,
"require-dev-dependencies" : true ,
"repositories" : [
{ "type" : "composer" , "url" : "http://packagist.org" }
],
"require" : {
"silex/silex" : "v1.2.2"
},
"archive" : {
"directory" : "dist" ,
"format" : "zip" ,
"skip-dev" : false
}
}
Your Satis Repository
Using Satis
{
"repositories" : [{
"type" : "composer" ,
"url" : "http://satis.mydomain.loc/"
}
]
}
$ composer.phar require zendframework/ zend- mail
$ composer.phar require mycompany/ mypackage
Toran Proxy
Toran Proxy Setup
$ wget https:// toranproxy.com/ releases/ toran- proxy- v1.1.4.tgz
$ tar xvfz toran- proxy- v1.1.4.tgz
$ cp app/ config/ parameters.yml.dist app/ config/ parameters.yml
$ php app/ console server: run -- env= prod
Your Toran Proxy
Access Control?
Use whatever your webserver supports (e.g. ldap auth, http basic auth) or maybe a vpn connection.
Credential handling
$ composer.phar install
Loading composer repositories with package information
Authentication required (satis.mydomain.loc):
Username: myuser
Password:
Do you want to store credentials for satis.mydomain.loc in
~/.composer/auth.json ? [Yn]
Does not only work for the satis repository but also for git/svn repositories
Credential handling
{
"http-basic" : {
"satis.mydomain.loc" : {
"username" : "myuser" ,
"password" : "mypassword"
},
"git.mydomain.loc" : {
"username" : "mygituser" ,
"password" : "mypassword"
},
"svn.mydomain.loc" : {
"username" : "mysvnuser" ,
"password" : "mypassword"
}
}
}
(Side note)
Vagrant Cachier Plugin
$ vagrant plugin install vagrant- cachier
VAGRANTFILE_API_VERSION = "2"
Vagrant. require_version ">= 1.6.0"
Vagrant. configure ( VAGRANTFILE_API_VERSION) do | config|
if Vagrant. has_plugin?( "vagrant-cachier" )
config. cache. scope = :box
config. cache. enable :composer
end
config. vm. define "wheezy" do | wheezy|
wheezy. vm. box = 'debian-wheezy-64'
end
end
Jenkins Node Config
Jenkins Node Environment
Jenkins Job Config
Push Config files to node
Jenkins Plugin: Config File Provider Plugin
Validate composer.json
Installing dependencies
$COMPOSER_USER and $COMPOSER_PASS are populated from the node environment settings we defined before.
Jenkins Plugin: Mask Passwords Plugin
Running the build
Will run PHP_CodeSniffer, PHPUnit, PDepend, ...
Run satis build for package
Jenkins Satis Job Config
Define Job Parameters
Jenkins Plugin: Parameterized Trigger Plugin
Executing Satis
Notify users after build
Jenkins Plugin: Email-ext plugin
Running the build
Result