Skip to content

FrOSCon - Marring Fron With Back End

By Bastian Hofmann and Volker Dusch The talk is about application architecture from the code site. Both are from ResearchGate Speak available on speakersdeck

What You Want

  • good code review
  • rapid development (even with big teams)
  • handle large code bases
  • frameworks to the rescue

Most Framework Are Dealing With Backend Or Frontend

  • backend
    • symfony
    • zend framework
  • frontend
    • backbone.js
    • jQuery
    • ExtJS

Frameworks Out There To Join Both Worlds

How To Achieve The Goal

  • incremental refactoring (take a look to flight)
  • separate your code into components
    • clean interface
    • clean architecture
    • small teams can work on that

How Reasearchgate Have Done It

Classicla Webapplication Architecture

  • loadbalancer
  • webservers
  • database
    • pgsql
    • memcached
    • mongodb
  • services (rest services)

Webserver

  • php templates
  • lazy db queries in view
  • yui
  • duplicated code

We Could Do It Better

  • try to compose your page out of containers
    • every component has to be self contained
    • can be addressed and rendered separately (by pure html response or json/xml data that is rendered by javascript on the client)
    • javascript is part of the component (easy up refactoring)
  • initial page request is returning a simple html page as response that is triggering a lot of ajax calls
  • if ajax request returns a ok, the return contains the updated data or handles the error (hint, on the backend, a "read from master" flag is added to retrieve the most accurate data without running into master/slave replication problems)
  • share code between server and client
  • templates, validation, entities
  • create "a widget"
    • php controller (providing data)
    • ...
  • do not fetch data directly (because it is painfully slow)
  • require stuff (search for "evolution of code design at facebook" at infoq)
    • fetch requirements in a preparing step
    • batch requirements by calling resolvers
    • resolvers talks with services (multi-GET)
    • services calls connector interface (databases for example)
    • fetching data from connector
    • distribute back all the fetched data
  • entity requirement
  • service requirement
  • request cache
  • Multi-Get (get multiple data via one request for fetching a lot of data)
  • requirement vs. optional
    • optional, template will be rendered without that data
  • data dependencies within a widget by using callbacks (to create a dependencies and dataload workflow)
  • php 5.5 generators by using yield
  • data dependecies between widgets by using data requirements (fetched from the require stuff above)
  • looks like this is complicated to debug by using symfony debugging tools (for example ;-))
  • rendering (by using mustache)
    • [x]html
    • json
    • v8js for rendering templates on the server (rendering mustache templates is faster in javascript then in php) and to provide a single place to render templates (cool!)
    • use models in javascript to stop messings things up
  • enables developers to only focus on their components
    • single place for refactoring
    • moving ugly code to one point
  • easies up testing (also with faked data)
  • error handling is simplified
    • remove component from the view
    • try to rerender it later
    • display a generic component
  • a/b testing is easied up (removing "long hanging fruites" - that just made my date :-D))
    • create multiple components and simple render then via a/b testing on the frontend
    • read bandit algorithm on untyped
  • by usgin "" you can use varnish to speed up the page
  • easy reusing of components ("take that button and move it or use to somewhere else")
  • mark components to load them asynchronous
  • use "placeholder" with "placeholder data" for search engines
  • try to use BigPipe (published by facebook)
    • compute as much informations as needed (like navigation) and then flush this to the client
    • render and flush some javascript logic
    • start computing the first body (kind of heavy work) that holds javascript logic and data to replace rendered and flushed content from the top
    • repeat that until you have finished creating the initial datas
    • send the final flush with the html end tag
  • fast changes on the frontend by replaceing only the components that have to be updated and use javascript push to change the url

FrOSCon - Unbreakable Domain Models

Mathias Varreas Mathias is helping teams escaping from survival mode. Cofounder of DDDBE Domain driven design Introduces me with a great upcomming term called "modelstorm". He also told me that a friend of him is writing on a paper about "modelstorm" so i'm looking forward to find more informations about that term on the web.

What Is This Talk About

  • Where to put business logic?
  • How to protect our code from abuse?
  • Where to put queries, and how test them?

Defining Terms

  • Domain is a problem space (let customers buy more)
  • Domain model is solution space

Domain Model

  • contains data model - the models state
  • protect your invariants (example: a customer must always have a email address)
    • create unittest with something like assertTrue($customer->hasValidEmail())
  • a less to abuse model is harder to break
    • like adding it to the constructor
  • use objects as consistency boundaries
    • "if i give you this object, you don't have to take care if it is valid and contains consistence data"
  • make the implicit the explicit
    • "a user must have a valid email address"
    • keep in mind, if you can add an "and" to describe a method, the method is doing to much
  • do not violate single responsibility principle
    • for the example, create email class
  • try to encapsulate state and behavior with (simple) value objects

Customer business process is: "A customer orders products and pays for them.".

  • try to encapsulate operations

  • use objects as consistency boundaries
    • "if i give you this object, you don't have to take care if it is valid and contains consistence data"
  • make the implicit the explicit
    • "a user must have a valid email address"
    • keep in mind, if you can add an "and" to describe a method, the method is doing to much
  • do not violate single responsibility principle
    • for the example, create email class
  • try to encapsulate state and behavior with (simple) value objects
  • Customer business process is: "A customer orders products and pays for them.".

    • try to encapsulate operations

    Customer business process is: "Order three times to become a premium member.".

    • try to create specification

      !php CustomerIsPremium::public function isSatisfiedBy(Customer $customer)

    Customer business process is: "Different rules apply for different tenants.".

    • use specifications to encapsulate rules about object selection

    Customer business process is: "Get a list of all premium customers.".

    • for big collections, try to duplicate the logic in the database (even if it breaks the single response philosophy)
    • test it by comparing (different implementation) object model code with database logic code in the unittest

    How To Persit A Model

    • implemented by a storage manager
    • special or generic storage manager has a save/add and delete/remove method

    Modelstorm

    The idea behind modelstorm (if i get it correctly) can be told as "a room of people, a bunch of magic markers, an area where you can write on". Prepare a wall with white papers. Each person can have an own marker with its own color. Each person is allowed to write on the paper. Then you start writing/creating the models. Add lines with actions to display process that handles different domain stages.

    FrOSCon - Database DevOps With Flyway, Git, Maven, MySQL And Jenkins

    By Michael Hüttermann.
    Thanks for the book again, i will try to write a review about it this year :-).

    What Is This Talk About

    • what is a DevOp
    • the goal
    • the problem
    • the solution
    • recipes and pitfalls
    • examples - not in this documentation
    • demo (MySQL, Flyway, Jenkins, Git, Maven, Gradle, Sonar, Vagrant) - not in this documentation

    How Is Part Of The Team

    • everybody how is envolved in the process of creating the software

    What Is Not A DevOp

    • is already a buzz word
    • it is not a new project role
    • its also not a new tool-suite
    • devOps is not a new department / business unit
    • *

    What Is A DevOp

    • development and operations
    • better communication
    • improved collaboration

    The Goal

    • shared incentives
    • holistic metrics
    • common processes (like kanban)
    • shared tools (same deployment tools for development and production for example)
    • high automation degree
    • improve and accelerate delivery - downsize the batch size
      • small releases
      • downsize the changesize to keep up the frequency
      • improve the cycle time (from bug finding/feature request to release the patch/feature)

    The Problem

    • a lot of completed functions/features facing a slow release cycle
      • release small, release often
    • problem between development (want feature online) against operation (want stable and "relaiable" software)

    The Solution

    • value stream map
    • continuous integration
    • continuous delivery (each commit moves system into stable and releaseable version)
    • continuous deployment
    • development and operation is merged together to create a infinie loop of release and feedback (both directions)
    • share knowledge and experience

    Define clearly what is a release (new feature, bugfix, and so one). Again DevOps is not a new layer between the two areas. Responsibilites is still clear to reproduce scenarios (automate as much). It is about people, not about tools. Implement culture of communication. Try to find conceptual deficits (also software design problems) as early as possible. Add a version number to your database layout (most likly same release number than code) - flyway, liquibase, self-made

    The Area Matrix

    • area 1 - extend development to operations
      • practice
        • use tools like puppet/chef/cfengine to provision environments from versioned code
      • goal
        • fast feedback through automations
        • reuse of code and tools
        • reliability of delivery process and provisioning
    • area 2 - extend operations to development
      • practice
        • provide monitoring and log files to development (or automated database snapshots to reproduce bug on development machine)
      • goal
        • share information about state in production
        • enable development to improve
        • enable development to trace production incidents
    • area 3 - embed development into operations (end user experience is goal to reach)
      • practice
        • set stability and capacity as development goals
      • goal
        • align goals, share incentives
    • area 4 - embed operations into development
      • practice
        • operations gives feedback about the design of the application that is under development, early and often
      • goal
        • avoid not releaseable design

    Recipes And Pitfalls

    • automate to ensure repeatability and foster collaboration (build once on a build server and release)
    • consider kanban (scrum is more for development but bad for organisation)
    • consider thorough change and config management - content management
    • track to ensure traceability (which build/release contains which tickets/bugs - artifactory)
    • monitor to support accountability (nagios and everything)
    • dashboard what you are doing (talk to each other what you are doing or what you have done)
    • use version control to ensure reproducibility (no manual tweaking after checkout)
    • consider pipelines/process from "put in feature/bugfix to release"
    • create (executable) documentation to, well, document (try to define goals and what it needs to reach it)
    • align goals and incentives of devs and ops (small cycle time)
    • ask and help, open culture of communication
    • respect your colleagues (stability against features, both leads to great user experience)

    FrOSCon - ownCloud

    By Björn Schießle.
    I am the owner of my data!
    Use and implements the User Data Manifesto.

    Back In The Past

    • everybody owned a pc at home
    • data is stored localy
    • simple install open source software to ensure you are the owner of your data

    Today

    • multiple pcs
    • access should be possible from everywhere

    Open Questions In New World

    • where is my data stored
    • how has access to it
    • what happens if i change the service provider
    • what happens if the service provider changes core functions

    User Data Manifesto In Short Terms

    • user has full control who has access to the data
    • user has to know where date is stored (server location, how is responsible of the server)
    • user has the right and the possiblility to switch to an other service provider (data are exportable, open standards are implemented)

    What Makes OwnCloud Different?

    • you decide what happens to your data
    • simple runs of every webspace (php driven)
    • simple user interface
    • quite easy to sync your data
    • based on AGPLv3 and open standards
    • integration in GNU/Linux desktops
    • extendable, ownCloud is a plattform

    Lets Talk About Files

    • files are available everywhere (web, desktop, mobile via webdav, browser, sync clint)
    • slide show for pictures included
    • multiple viewers available for text, pdf, odf, image, video, audio etc.
    • allows you to share files with others (scoped by groups with different rights)
    • provides mounting of external storaged like WebDAV, Dropbox, GDrive, FTP etc.
    • server based encryption
    • simple kind of versioning - last save wins
    • thrashbin implemented (to restore deleted files)
    • intelligent storage management included
    • conflict solving implemented in desktop sync client

    Contacts

    • address book via webinterface or client
    • share address books with others (same permission handling as with files)
    • sync address book via CardDAV (?)

    Calender

    • appointments via webinterface or clinet
    • show birthdays
    • share calender
    • sync calender via CalDAV

    Newsreader

    • full workable newsreader included
    • google reader import
    • different clients (web, android, ios, qt-app, html 5)

    Notes

    • app to write notes and tasks
    • android ap for working with tasks via CardDAV

    And A Lot More

    • music player
    • url shorter
    • full text search
    • presentation via ownCloud
    • bookmarks (a firefox bookmark plugin is currently in development)
    • cards via OpenStreetMap
    • and many more

    Desktop Integration

    Wrap Up

      • file sync with keeping control for rights
      • free software and open standards
      • clients for major free desktops or mobile devices
      • integration in free desktops
      • plattform for many more applications
      • it is enterprise ready