By Johannes Ziemke
What is Docker
- version 1.0 released a few hours ago - api stable (a bit)
- written in go
- build, packs and ships applications as lightweight containers
- build once an run almost everywhere (since kernel 3.8.*)
- rewrite of docCloud PaaS code
- start always from clean slate
- spawn up complete test infrastructures in seconds
- run your code against multiple versions easily
- easy up setup of software environment
- distribute complex setups as self-contained containers
- build own infrastructure
- easy ups deployment per day
- scaling and mantaining of infrastructure (state convergence, manage everything)
- helps you to solve managing complexity
- similar problems
- modules, classes, plugins
- human communication: named concepts like car, cat or container
- shipping goods: intermodal containers
- solution: create an abstraction
- does not matter what is in the container nor where it runs
- clear separation of concerns (reduce communication between developers and ops)
- developer can take care about used libraries, packagemanager, ...
- op can take care about the system (alos resource planning and monitoring)
- running commandos on top of:
- immutable, shipable, layered images (defined by dockerfile, built by builder and pushed/pulled to registry)
- copy-on-write storage on top (provides writeable layer on top of read only images to persists changes done by running container)
- isolated environment
- using kernal features
- namespaces (isolation by scoping, available: pid, mnt, net, uts, jpc, user)
- cgroups (limit, accound and isolated general devices, cou, memory, ...)
- future (solaris zones ...)
- via RESTish API
- missing pieces
- docker can not address service discovery or dynamic scheduling
- projects to close the gap
- mesos + marathon
- --privileged - can break out of container but control host
- images are read only and creating a container on each start
- docker commit - creates new image out of running container
- use boot2docker
By Tobias Schwab
from Dynport GmbH
- continous delivery
- canarien releases
- "never touch running system", create a new one when something has to change
- immutable releases (once created, never change it)
Why They used Docker
- privacy concerns, aws not an option
- hoster they could not pick
- unreliable hoster API
- flash based structure management
- limited capacity
- they are the biggest customer
- image based deployments -> use dockerfiles (do not add dockerfile generators on top)
- use syslog and rsyslog to log out of the container
- >>CMD ["/sbin/init"]<< to start debian based image
- configuration management
- store configuration in environment
- dependency injected with start of container (use same container for testing, stageing, testing ...)
- logfiles and data outside of the container
- do not
- full blown vm (best case, one service per container)
- ssh deamon inside containers
- syslog deamon inside containters (sometimes needed)
- user management (everything can run as root)
- chef/puppet makes caching useles
- rely on external services (github, gem page)
- build management tools
- bundler, pip, carton, composer, ...
- problem is, they are slow on "clean slate"
- ways to solve
- add mainifest (like composer.lock) before code and use cached build files
- pre-bundles base images
- multi host
- image distribution via docker registry
- weighted load balancing via HAProxy
- SSL termination via HAProxy (container exists by Johannes Ziemke)
- load balancing
- pool configuration stored in redis/etcd
- configuration update
- uploaded via ssh to host
- verify configuration
- replace configuration (old with new)
- reload configuration
- putting container id, revision of image (host, code, request) via rsyslog out of the container
- distributed, scalable time series database
- from syslog via udp
- rickshaw.js for graphs
- compare status cudes, counts and times between actions of two revisions
- benefits (reduces)
- external dependencies
- "did work on my machine"
- unused cpu cycles
- number of hosts
- feedback times (number of deploys per day)
- time to get new host online
- hosts are more flexible
- controlled revision change (controlled on each host)
- faster build
Take a look to knpbundles.
- problem is solved in other language
- problem is solved in other licence
- can be solved by design patterns (abstract solution)
- write a good bundle/component that is general enough takes much more time
Reuseage by copy and paste.
- decreases maintainability
- decreases testability
- violates dry principle
- community web application
- users seek other users by interests
- how to input data to easy up matching?
What Do We Need?
- form widget (js and css based)
- data transformer (converts format from storage to human readable presentation and back)
- form type (provides name for form widget, called in FormTypes, gets TagManager injected and defines DataTransformer)
- Entity Trait (avoid copy and paste)
- Taggable Subscriber
- Make it reusable (dedicated bundle, created composer project)
Available on github
This talk contains information about architecture.
When have done this for quite a while.
This works best in the past.
General solution was to add more hardware and that leads to more errors.
- monolithic architecture
- normalized data (in database)
- pull-principle (browser is build per request)
- full page cache is bad
- edge site include (for varnish/ESI) is also not the best solution
We are fast when:
- no real workload per request
- everything from memory
- denomalized data
How Often Does Data Change (From Rare To Often)?
Based on the example of a shop system.
- product information
Who Changes Data?
- product manager
- receipt of goods / outgoing goods
- images and static content is deliverd by static content webserver
- only dynamic part is done by php
- snippets are deliverd, not full pages
Benefit of key-value-store (for example redis) is, that the database itself can die and the only thing that happens is, that the entries are becoming obsolete.
But What About Filternavigation?
Use a searchengine that returns simple a collection of product keys. Use this product keys and ask the key value store to fetch product data.
But What About Personalization?
- use snippets with variables and default values
- fetch this snippet in the key value store
- thanks to "time to death" feature provided by many key-value-stores, you can easily define "special offers" per day and so on
Nice meetup, incredible how many people are attending already.
2013-09-10 Hamburg, Otto, Loft 06
Define The Unspeakable
moral concepts (negotiable)
- freaked out / nerdic
- cutting edge
They are teached/ already at school level (errors are bad, don't do something wrong).
- what can i do
- who am i able to add critic to
- fellow men
From the school already, we get trained to:
- it is bad to say "that was my fault"
- shift the know error to others
- there is one majority out there who decides if this is a bad error
- be perfect
- always have a fallback (not to lose the face)
Define A Good Error Culture
This goals can be achived by exemplify the following steps (best by upper management but important for all).
- "good" is enough
- no fear of mistakes
- many/fast update cycles (don't try to do big jumps, doing small steps results in small mistakes)
- to take the blame is not important
- do not repeat your mistakes
- do small and usefull steps (validate if your change contains a "only if")
- use and understand the Twelve Principles of Agile Software
- celebrate reached goals
Slides are available on judiths page. Again a nice talk and incredible what kind of easy mistakes we are doing in our regular live.