Skip to content

FrOSCon - Test Proxies: How Problems Became Features

By Sebastian Bergman and Arne Blankerts.
Keep your eyes up and stay tuned for "patch coverage".
Second time, thanks for the book - handed over by sebastian himself. Mister HowTrueIsFalse, you will get back your book soon :-).

What Is The Talk About

Integration tests closing the gap between unit tests and system tests. This is resolved by keep the focus on interfaces between components and testing that the interaction between the components is working as expected.

Arne Blankerts and Sebastian Bergman showing how a problem by setting up the integration tests became a feature aka "Test Proxies" for PHPUnit.

They also discuss problems and provide solutions for implementing mock objects in php unit and how to ship them as features.

How To Visualize

How lower you go, the faster and the exact the tested code behaves. How higher you go, the more you can provide a stable and general testcase.

  • business
  • technology

  • ui

  • service
  • unit

Isolation, speed (top down) vs Confidence in whole system (bottom up) from how google tests software * large * medium * small

Terms

Test one unit is a unit test.

Write a test between two units is more a integration test than a unit test. Current idea is to stub or mock one of the two units and see how other unit acts.

Push the idea of stubbing and mocking more to the edge will lead to "edge to edge test" like "client to server test". In general, try to test components on the same level (layer).
Try to monitor systems for load tests and use appdynamcis or xdebug and xhprof.

Example

<?php
namespace Company\Project

class SampleWorkflow
{
    private $backend;
    private $service;

    public function __construct(Backend $backend, Service $service)
    {
        $this->backend = $backend;
        $this->service = $service;
    }

    public function execute(Request $request)
        $this->service->workflow(
            $this->backend->getObjectById($request->getValue('id));
        );
    }
}

How to test this?

Would be cool if we can log that code was executed like test proxies by adding an extra flag to the mock builder.
Provides way to see where dataflow changed or drifts from expected way (will be in PHPUnit 3.8).

<?php
namespace Company\Project

use PHPUnit_Framework_TestCase;

class SampleWorkflowTest extends PHPUnit_Framework_TestCase
{
    public function testServiceCallUpdatesObjects()
    {
        $service = $this->getMockBuilder('Service')
            ->enableProxyingToOriginalMethods()
            ->getMock();

        $service->expects($this->once())
            ->method('doWork');

        $backend = new Backend();
        $workflow = new SampleWorkflow($backend, $service);

        $workflow->execute(new Request(array('id' => 42)));
    }
}

FrOSCon - Marring Fron With Back End

By Bastian Hofmann and Volker Dusch The talk is about application architecture from the code site. Both are from ResearchGate Speak available on speakersdeck

What You Want

  • good code review
  • rapid development (even with big teams)
  • handle large code bases
  • frameworks to the rescue

Most Framework Are Dealing With Backend Or Frontend

  • backend
    • symfony
    • zend framework
  • frontend
    • backbone.js
    • jQuery
    • ExtJS

Frameworks Out There To Join Both Worlds

How To Achieve The Goal

  • incremental refactoring (take a look to flight)
  • separate your code into components
    • clean interface
    • clean architecture
    • small teams can work on that

How Reasearchgate Have Done It

Classicla Webapplication Architecture

  • loadbalancer
  • webservers
  • database
    • pgsql
    • memcached
    • mongodb
  • services (rest services)

Webserver

  • php templates
  • lazy db queries in view
  • yui
  • duplicated code

We Could Do It Better

  • try to compose your page out of containers
    • every component has to be self contained
    • can be addressed and rendered separately (by pure html response or json/xml data that is rendered by javascript on the client)
    • javascript is part of the component (easy up refactoring)
  • initial page request is returning a simple html page as response that is triggering a lot of ajax calls
  • if ajax request returns a ok, the return contains the updated data or handles the error (hint, on the backend, a "read from master" flag is added to retrieve the most accurate data without running into master/slave replication problems)
  • share code between server and client
  • templates, validation, entities
  • create "a widget"
    • php controller (providing data)
    • ...
  • do not fetch data directly (because it is painfully slow)
  • require stuff (search for "evolution of code design at facebook" at infoq)
    • fetch requirements in a preparing step
    • batch requirements by calling resolvers
    • resolvers talks with services (multi-GET)
    • services calls connector interface (databases for example)
    • fetching data from connector
    • distribute back all the fetched data
  • entity requirement
  • service requirement
  • request cache
  • Multi-Get (get multiple data via one request for fetching a lot of data)
  • requirement vs. optional
    • optional, template will be rendered without that data
  • data dependencies within a widget by using callbacks (to create a dependencies and dataload workflow)
  • php 5.5 generators by using yield
  • data dependecies between widgets by using data requirements (fetched from the require stuff above)
  • looks like this is complicated to debug by using symfony debugging tools (for example ;-))
  • rendering (by using mustache)
    • [x]html
    • json
    • v8js for rendering templates on the server (rendering mustache templates is faster in javascript then in php) and to provide a single place to render templates (cool!)
    • use models in javascript to stop messings things up
  • enables developers to only focus on their components
    • single place for refactoring
    • moving ugly code to one point
  • easies up testing (also with faked data)
  • error handling is simplified
    • remove component from the view
    • try to rerender it later
    • display a generic component
  • a/b testing is easied up (removing "long hanging fruites" - that just made my date :-D))
    • create multiple components and simple render then via a/b testing on the frontend
    • read bandit algorithm on untyped
  • by usgin "" you can use varnish to speed up the page
  • easy reusing of components ("take that button and move it or use to somewhere else")
  • mark components to load them asynchronous
  • use "placeholder" with "placeholder data" for search engines
  • try to use BigPipe (published by facebook)
    • compute as much informations as needed (like navigation) and then flush this to the client
    • render and flush some javascript logic
    • start computing the first body (kind of heavy work) that holds javascript logic and data to replace rendered and flushed content from the top
    • repeat that until you have finished creating the initial datas
    • send the final flush with the html end tag
  • fast changes on the frontend by replaceing only the components that have to be updated and use javascript push to change the url

FrOSCon - Unbreakable Domain Models

Mathias Varreas Mathias is helping teams escaping from survival mode. Cofounder of DDDBE Domain driven design Introduces me with a great upcomming term called "modelstorm". He also told me that a friend of him is writing on a paper about "modelstorm" so i'm looking forward to find more informations about that term on the web.

What Is This Talk About

  • Where to put business logic?
  • How to protect our code from abuse?
  • Where to put queries, and how test them?

Defining Terms

  • Domain is a problem space (let customers buy more)
  • Domain model is solution space

Domain Model

  • contains data model - the models state
  • protect your invariants (example: a customer must always have a email address)
    • create unittest with something like assertTrue($customer->hasValidEmail())
  • a less to abuse model is harder to break
    • like adding it to the constructor
  • use objects as consistency boundaries
    • "if i give you this object, you don't have to take care if it is valid and contains consistence data"
  • make the implicit the explicit
    • "a user must have a valid email address"
    • keep in mind, if you can add an "and" to describe a method, the method is doing to much
  • do not violate single responsibility principle
    • for the example, create email class
  • try to encapsulate state and behavior with (simple) value objects

Customer business process is: "A customer orders products and pays for them.".

  • try to encapsulate operations

  • use objects as consistency boundaries
    • "if i give you this object, you don't have to take care if it is valid and contains consistence data"
  • make the implicit the explicit
    • "a user must have a valid email address"
    • keep in mind, if you can add an "and" to describe a method, the method is doing to much
  • do not violate single responsibility principle
    • for the example, create email class
  • try to encapsulate state and behavior with (simple) value objects
  • Customer business process is: "A customer orders products and pays for them.".

    • try to encapsulate operations

    Customer business process is: "Order three times to become a premium member.".

    • try to create specification

      !php CustomerIsPremium::public function isSatisfiedBy(Customer $customer)

    Customer business process is: "Different rules apply for different tenants.".

    • use specifications to encapsulate rules about object selection

    Customer business process is: "Get a list of all premium customers.".

    • for big collections, try to duplicate the logic in the database (even if it breaks the single response philosophy)
    • test it by comparing (different implementation) object model code with database logic code in the unittest

    How To Persit A Model

    • implemented by a storage manager
    • special or generic storage manager has a save/add and delete/remove method

    Modelstorm

    The idea behind modelstorm (if i get it correctly) can be told as "a room of people, a bunch of magic markers, an area where you can write on". Prepare a wall with white papers. Each person can have an own marker with its own color. Each person is allowed to write on the paper. Then you start writing/creating the models. Add lines with actions to display process that handles different domain stages.

    FrOSCon - Database DevOps With Flyway, Git, Maven, MySQL And Jenkins

    By Michael Hüttermann.
    Thanks for the book again, i will try to write a review about it this year :-).

    What Is This Talk About

    • what is a DevOp
    • the goal
    • the problem
    • the solution
    • recipes and pitfalls
    • examples - not in this documentation
    • demo (MySQL, Flyway, Jenkins, Git, Maven, Gradle, Sonar, Vagrant) - not in this documentation

    How Is Part Of The Team

    • everybody how is envolved in the process of creating the software

    What Is Not A DevOp

    • is already a buzz word
    • it is not a new project role
    • its also not a new tool-suite
    • devOps is not a new department / business unit
    • *

    What Is A DevOp

    • development and operations
    • better communication
    • improved collaboration

    The Goal

    • shared incentives
    • holistic metrics
    • common processes (like kanban)
    • shared tools (same deployment tools for development and production for example)
    • high automation degree
    • improve and accelerate delivery - downsize the batch size
      • small releases
      • downsize the changesize to keep up the frequency
      • improve the cycle time (from bug finding/feature request to release the patch/feature)

    The Problem

    • a lot of completed functions/features facing a slow release cycle
      • release small, release often
    • problem between development (want feature online) against operation (want stable and "relaiable" software)

    The Solution

    • value stream map
    • continuous integration
    • continuous delivery (each commit moves system into stable and releaseable version)
    • continuous deployment
    • development and operation is merged together to create a infinie loop of release and feedback (both directions)
    • share knowledge and experience

    Define clearly what is a release (new feature, bugfix, and so one). Again DevOps is not a new layer between the two areas. Responsibilites is still clear to reproduce scenarios (automate as much). It is about people, not about tools. Implement culture of communication. Try to find conceptual deficits (also software design problems) as early as possible. Add a version number to your database layout (most likly same release number than code) - flyway, liquibase, self-made

    The Area Matrix

    • area 1 - extend development to operations
      • practice
        • use tools like puppet/chef/cfengine to provision environments from versioned code
      • goal
        • fast feedback through automations
        • reuse of code and tools
        • reliability of delivery process and provisioning
    • area 2 - extend operations to development
      • practice
        • provide monitoring and log files to development (or automated database snapshots to reproduce bug on development machine)
      • goal
        • share information about state in production
        • enable development to improve
        • enable development to trace production incidents
    • area 3 - embed development into operations (end user experience is goal to reach)
      • practice
        • set stability and capacity as development goals
      • goal
        • align goals, share incentives
    • area 4 - embed operations into development
      • practice
        • operations gives feedback about the design of the application that is under development, early and often
      • goal
        • avoid not releaseable design

    Recipes And Pitfalls

    • automate to ensure repeatability and foster collaboration (build once on a build server and release)
    • consider kanban (scrum is more for development but bad for organisation)
    • consider thorough change and config management - content management
    • track to ensure traceability (which build/release contains which tickets/bugs - artifactory)
    • monitor to support accountability (nagios and everything)
    • dashboard what you are doing (talk to each other what you are doing or what you have done)
    • use version control to ensure reproducibility (no manual tweaking after checkout)
    • consider pipelines/process from "put in feature/bugfix to release"
    • create (executable) documentation to, well, document (try to define goals and what it needs to reach it)
    • align goals and incentives of devs and ops (small cycle time)
    • ask and help, open culture of communication
    • respect your colleagues (stability against features, both leads to great user experience)