Skip to content

Introduction Into Docker and Continous Delivery with Docker - Docker Usergroup Hamburg 09.06.2014

Docker UserGroup Hamburg

Introduction Docker

General

By Johannes Ziemke from docker

What is Docker

  • version 1.0 released a few hours ago - api stable (a bit)
  • written in go
  • build, packs and ships applications as lightweight containers
  • build once an run almost everywhere (since kernel 3.8.*)
  • rewrite of docCloud PaaS code

Benefits

  • start always from clean slate
  • spawn up complete test infrastructures in seconds
  • run your code against multiple versions easily
  • easy up setup of software environment
  • distribute complex setups as self-contained containers
  • build own infrastructure
  • easy ups deployment per day
  • scaling and mantaining of infrastructure (state convergence, manage everything)
  • helps you to solve managing complexity
    • similar problems
      • modules, classes, plugins
      • human communication: named concepts like car, cat or container
      • shipping goods: intermodal containers
    • solution: create an abstraction
      • does not matter what is in the container nor where it runs
      • clear separation of concerns (reduce communication between developers and ops)
      • developer can take care about used libraries, packagemanager, ...
      • op can take care about the system (alos resource planning and monitoring)
  • running commandos on top of:
    • immutable, shipable, layered images (defined by dockerfile, built by builder and pushed/pulled to registry)
    • copy-on-write storage on top (provides writeable layer on top of read only images to persists changes done by running container)
      • aufs
      • btrfs
      • devicemapper
    • isolated environment
      • using kernal features
        • namespaces (isolation by scoping, available: pid, mnt, net, uts, jpc, user)
        • cgroups (limit, accound and isolated general devices, cou, memory, ...)
        • future (solaris zones ...)
    • via RESTish API
    • missing pieces
      • docker can not address service discovery or dynamic scheduling
      • projects to close the gap
        • openstack
        • coreOs
        • mesos + marathon
        • flynn.io

Demotime

  • --privileged - can break out of container but control host
  • images are read only and creating a container on each start
  • docker commit - creates new image out of running container
  • use boot2docker

Continous Delivery with Docker

By Tobias Schwab from Dynport GmbH

Philosophie

  • continous delivery
  • canarien releases
  • "never touch running system", create a new one when something has to change
  • immutable releases (once created, never change it)

Why They used Docker

  • privacy concerns, aws not an option
  • hoster they could not pick
  • unreliable hoster API
  • flash based structure management
  • limited capacity
  • they are the biggest customer

Learnings

  • image based deployments -> use dockerfiles (do not add dockerfile generators on top)
  • use syslog and rsyslog to log out of the container
  • >>CMD ["/sbin/init"]<< to start debian based image
  • caching
  • configuration management
    • store configuration in environment
    • dependency injected with start of container (use same container for testing, stageing, testing ...)
    • logfiles and data outside of the container
  • do not
    • full blown vm (best case, one service per container)
    • ssh deamon inside containers
    • syslog deamon inside containters (sometimes needed)
    • user management (everything can run as root)
    • chef/puppet makes caching useles
    • rely on external services (github, gem page)
  • build management tools
    • bundler, pip, carton, composer, ...
    • problem is, they are slow on "clean slate"
    • ways to solve
      • add mainifest (like composer.lock) before code and use cached build files
      • pre-bundles base images
  • multi host
    • image distribution via docker registry
    • weighted load balancing via HAProxy
    • SSL termination via HAProxy (container exists by Johannes Ziemke)
  • load balancing
    • HAProxy
    • pool configuration stored in redis/etcd
    • configuration update
      • created
      • uploaded via ssh to host
      • verify configuration
      • replace configuration (old with new)
      • reload configuration
  • logging
    • putting container id, revision of image (host, code, request) via rsyslog out of the container
  • metrics
    • openTSDB
      • distributed, scalable time series database
      • hbase
      • tags/diminsions
      • from syslog via udp
      • rickshaw.js for graphs
      • compare status cudes, counts and times between actions of two revisions
  • benefits (reduces)
    • external dependencies
    • "did work on my machine"
    • unused cpu cycles
    • number of hosts
    • feedback times (number of deploys per day)
    • time to get new host online
    • hosts are more flexible
    • controlled revision change (controlled on each host)
    • faster build

Reusability Of Symfony Component - Mindworks

By Oliver
Mindworks
Jarrestraße 42A
Take a look to knpbundles.

Hurdles

  • problem is solved in other language
  • problem is solved in other licence
  • can be solved by design patterns (abstract solution)
  • write a good bundle/component that is general enough takes much more time

Anti-Pattern

Reuseage by copy and paste.

  • decreases maintainability
  • decreases testability
  • violates dry principle

Scenario

  • community web application
  • users seek other users by interests
  • how to input data to easy up matching?

What Do We Need?

  • form widget (js and css based)
  • data transformer (converts format from storage to human readable presentation and back)
  • form type (provides name for form widget, called in FormTypes, gets TagManager injected and defines DataTransformer)
  • Entity Trait (avoid copy and paste)
  • Taggable Subscriber
  • Make it reusable (dedicated bundle, created composer project)
  • Use JavaScript and CSS via "composer component" (see web/bundles, RobLoach/component-installer)

Live Demo

Available on github

High-Performance Websites - PHPUG and WebPerformanceUG

Arne Blankerts, Stefan Priebsch - High-Performance Websites

This talk contains information about architecture.

Classical Architecture

When have done this for quite a while.

Browser->WebServer->PHP-Database

This works best in the past. General solution was to add more hardware and that leads to more errors.

  • monolithic architecture
  • normalized data (in database)
  • pull-principle (browser is build per request)
  • full page cache is bad
  • edge site include (for varnish/ESI) is also not the best solution

We are fast when:

  • no real workload per request
  • everything from memory
  • denomalized data
  • snippets

How Often Does Data Change (From Rare To Often)?

Based on the example of a shop system.

  • catalog
  • product information
  • price
  • availability

Who Changes Data?

  • editor
  • product manager
  • receipt of goods / outgoing goods

New Architecture (CQRS-Architecture)

  • images and static content is deliverd by static content webserver
  • only dynamic part is done by php
  • snippets are deliverd, not full pages

PHP->Key-Value-Store->Backend-Process->Database
or
PHP->Web-Service->Database

Benefit of key-value-store (for example redis) is, that the database itself can die and the only thing that happens is, that the entries are becoming obsolete.

But What About Filternavigation?

Use a searchengine that returns simple a collection of product keys. Use this product keys and ask the key value store to fetch product data.

But What About Personalization?

  • use snippets with variables and default values
  • fetch this snippet in the key value store
  • thanks to "time to death" feature provided by many key-value-stores, you can easily define "special offers" per day and so on

Nice meetup, incredible how many people are attending already.

Define The Unspeakable - PHPUG and WebPerformanceUG

2013-09-10 Hamburg, Otto, Loft 06

Judith Andresen - Make Mistakes The Right Way

Define The Unspeakable

Result

  • establishment
  • project
  • uniform

moral concepts (negotiable)

  • hanseatic
  • freaked out / nerdic
  • cutting edge

basic assumption

They are teached/ already at school level (errors are bad, don't do something wrong).

  • what can i do
  • who am i able to add critic to
  • natur
  • fellow men
  • industry

From the school already, we get trained to:

  • it is bad to say "that was my fault"
  • shift the know error to others
  • there is one majority out there who decides if this is a bad error
  • be perfect
  • always have a fallback (not to lose the face)

Define A Good Error Culture

This goals can be achived by exemplify the following steps (best by upper management but important for all).

  • "good" is enough
  • no fear of mistakes
  • many/fast update cycles (don't try to do big jumps, doing small steps results in small mistakes)
  • straightforwardness
  • to take the blame is not important
  • do not repeat your mistakes
  • do small and usefull steps (validate if your change contains a "only if")
  • use and understand the Twelve Principles of Agile Software
  • celebrate reached goals

Slides are available on judiths page. Again a nice talk and incredible what kind of easy mistakes we are doing in our regular live.