Skip to content

FrOSCon - Database DevOps With Flyway, Git, Maven, MySQL And Jenkins

By Michael Hüttermann.
Thanks for the book again, i will try to write a review about it this year :-).

What Is This Talk About

  • what is a DevOp
  • the goal
  • the problem
  • the solution
  • recipes and pitfalls
  • examples - not in this documentation
  • demo (MySQL, Flyway, Jenkins, Git, Maven, Gradle, Sonar, Vagrant) - not in this documentation

How Is Part Of The Team

  • everybody how is envolved in the process of creating the software

What Is Not A DevOp

  • is already a buzz word
  • it is not a new project role
  • its also not a new tool-suite
  • devOps is not a new department / business unit
  • *

What Is A DevOp

  • development and operations
  • better communication
  • improved collaboration

The Goal

  • shared incentives
  • holistic metrics
  • common processes (like kanban)
  • shared tools (same deployment tools for development and production for example)
  • high automation degree
  • improve and accelerate delivery - downsize the batch size
    • small releases
    • downsize the changesize to keep up the frequency
    • improve the cycle time (from bug finding/feature request to release the patch/feature)

The Problem

  • a lot of completed functions/features facing a slow release cycle
    • release small, release often
  • problem between development (want feature online) against operation (want stable and "relaiable" software)

The Solution

  • value stream map
  • continuous integration
  • continuous delivery (each commit moves system into stable and releaseable version)
  • continuous deployment
  • development and operation is merged together to create a infinie loop of release and feedback (both directions)
  • share knowledge and experience

Define clearly what is a release (new feature, bugfix, and so one). Again DevOps is not a new layer between the two areas. Responsibilites is still clear to reproduce scenarios (automate as much). It is about people, not about tools. Implement culture of communication. Try to find conceptual deficits (also software design problems) as early as possible. Add a version number to your database layout (most likly same release number than code) - flyway, liquibase, self-made

The Area Matrix

  • area 1 - extend development to operations
    • practice
      • use tools like puppet/chef/cfengine to provision environments from versioned code
    • goal
      • fast feedback through automations
      • reuse of code and tools
      • reliability of delivery process and provisioning
  • area 2 - extend operations to development
    • practice
      • provide monitoring and log files to development (or automated database snapshots to reproduce bug on development machine)
    • goal
      • share information about state in production
      • enable development to improve
      • enable development to trace production incidents
  • area 3 - embed development into operations (end user experience is goal to reach)
    • practice
      • set stability and capacity as development goals
    • goal
      • align goals, share incentives
  • area 4 - embed operations into development
    • practice
      • operations gives feedback about the design of the application that is under development, early and often
    • goal
      • avoid not releaseable design

Recipes And Pitfalls

  • automate to ensure repeatability and foster collaboration (build once on a build server and release)
  • consider kanban (scrum is more for development but bad for organisation)
  • consider thorough change and config management - content management
  • track to ensure traceability (which build/release contains which tickets/bugs - artifactory)
  • monitor to support accountability (nagios and everything)
  • dashboard what you are doing (talk to each other what you are doing or what you have done)
  • use version control to ensure reproducibility (no manual tweaking after checkout)
  • consider pipelines/process from "put in feature/bugfix to release"
  • create (executable) documentation to, well, document (try to define goals and what it needs to reach it)
  • align goals and incentives of devs and ops (small cycle time)
  • ask and help, open culture of communication
  • respect your colleagues (stability against features, both leads to great user experience)

FrOSCon - ownCloud

By Björn Schießle.
I am the owner of my data!
Use and implements the User Data Manifesto.

Back In The Past

  • everybody owned a pc at home
  • data is stored localy
  • simple install open source software to ensure you are the owner of your data

Today

  • multiple pcs
  • access should be possible from everywhere

Open Questions In New World

  • where is my data stored
  • how has access to it
  • what happens if i change the service provider
  • what happens if the service provider changes core functions

User Data Manifesto In Short Terms

  • user has full control who has access to the data
  • user has to know where date is stored (server location, how is responsible of the server)
  • user has the right and the possiblility to switch to an other service provider (data are exportable, open standards are implemented)

What Makes OwnCloud Different?

  • you decide what happens to your data
  • simple runs of every webspace (php driven)
  • simple user interface
  • quite easy to sync your data
  • based on AGPLv3 and open standards
  • integration in GNU/Linux desktops
  • extendable, ownCloud is a plattform

Lets Talk About Files

  • files are available everywhere (web, desktop, mobile via webdav, browser, sync clint)
  • slide show for pictures included
  • multiple viewers available for text, pdf, odf, image, video, audio etc.
  • allows you to share files with others (scoped by groups with different rights)
  • provides mounting of external storaged like WebDAV, Dropbox, GDrive, FTP etc.
  • server based encryption
  • simple kind of versioning - last save wins
  • thrashbin implemented (to restore deleted files)
  • intelligent storage management included
  • conflict solving implemented in desktop sync client

Contacts

  • address book via webinterface or client
  • share address books with others (same permission handling as with files)
  • sync address book via CardDAV (?)

Calender

  • appointments via webinterface or clinet
  • show birthdays
  • share calender
  • sync calender via CalDAV

Newsreader

  • full workable newsreader included
  • google reader import
  • different clients (web, android, ios, qt-app, html 5)

Notes

  • app to write notes and tasks
  • android ap for working with tasks via CardDAV

And A Lot More

  • music player
  • url shorter
  • full text search
  • presentation via ownCloud
  • bookmarks (a firefox bookmark plugin is currently in development)
  • cards via OpenStreetMap
  • and many more

Desktop Integration

Wrap Up

    • file sync with keeping control for rights
    • free software and open standards
    • clients for major free desktops or mobile devices
    • integration in free desktops
    • plattform for many more applications
    • it is enterprise ready

FrOSCon - Open Source Storage Management With openATTIC

My first german talk today so hopefully the human "just in time translation" has a less error state today ;-).

openATTIC with David Breitung.
it-novum
openATTIC

it-novum GmbH
Edelzeller Str. 44
36043 Fulda

The Two Faces Of Cloud

User View

  • flexibility
  • rapid deployment
  • costs
  • high availability

Producer View

  • ...

What Defindes A Unified Storage

  • high availability
  • freedrom of used protocol
  • storage efficiency
  • centraliced management
  • process integration
  • scalable

High Availablility

  • data mirroring
    • block based
    • file based
    • syncron
    • asyncron
  • cluster failover
    • automatated
    • ...

Freedom Of Protocol

  • NAS protocol
    • CIFS/SMB
    • NFS
    • AFP
  • SAN protocol
    • iSCSI
    • Fibrechannel
    • FCoE
    • Infiniband

storage efficiency

  • deduplication
  • compression
  • over commitment
  • thin provisioning
  • snapshotting

Centraliced Management

  • administration
    • web frontend
    • command line tool
    • api
  • openATTIC is a master/slave system
  • minitoring (by using nagios, check itcockpit)
    • availability
    • ressources
    • performance

Process Integration

  • automated backup process
    • intelligent backups by SnapApps (for consistent snapshots)
    • to backup a database in good health (consistent) state
  • automated server deployment
    • request by user, helpdesk or api
    • ...
    • deployment
    • feedback
  • Snapshotting
    • check snapshot
    • backup healthy snapshot

Scalable

  • disc scaling
    • more controllers
    • more shelfs
    • more discs
  • node scaling
    • openATTIC Hosts
    • other hosts

Costs

  • benefits by freedom of choice for server choosing
  • software and license cost
  • running environment since a lot of tools are community driven

Why Open Source

  • vendor lock
  • security
  • flexibility
  • cost (...)
  • innovation

OpenATTIC

  • community edition
    • supported by community
    • basis functionality
    • open source
    • central web console
    • nas protocol
    • san protocol
    • deduplication
  • enterprise edition

Costs

  • benefits by freedom of choice for server choosing
  • software and license cost
  • running environment since a lot of tools are community driven

Basic Framework (Down To Up Layer)

  • service and tools and vendor plugins
  • backend
  • core (deamon, python django based))
  • frontend (XML RPC API, extJS)
  • interfaces

Next Steps

  • fibrechannel, fcoe
  • more interfaces to vendors
  • more snap apps
  • content adressable storage (cas, hadoop)
  • glusterFS
  • object storeage planned in the far future

FrOSCon - How To Make Money From Open Source Today

Gosh, one of the fastes talks i joined in the past. Nevertheless, the talk was full of informations. Mr. Widenius is an incredible likeable person since you feel the force he is up to presenting open source as a workable business concept.

Based on the speed of the slide switches and the enormous number of slides, there are a lot of gabs in my blog entry (aka talk record).

By Michael "Monty" Widenius, one of the creator of mysql and now part of the mariaDB team.

What Is The Talk About

  • how to make a good open source project
  • how to create money of that
  • understand how open source works
  • create an open source project
  • participate, drive or fork an open source project
  • try to create a company around an open source code

Good And Bad About Open Source

Good

  • more developers
  • more spread
  • much better code
  • works well for a lot of projects

Bad

  • not enough money to pay developers (most at the beginning)
  • hard to get money and investors for projects (you have to make consulting and so on to get investors)
  • investors, in general, don't belive that a open source company can make enough money

Things To Clear Up

Top Reasons For Creating An Open Source Project

  • solve one of ones personal problems (mysql/php)
  • relicense and existing closed source porject to open source to get the benefits of open source (netscape)
  • get more developers on an internal company tool/project
  • wanting to earn money and at the same time do something good
  • wanting to give something back to the comunity
  • ...

Developing Software Open Source Is In Human Nature

  • you solve your own problems and get free help and development efforts from others while doing it
  • you participate to increase your reputation
  • you use open source because its less expensive
  • ...

Before Starting A New Open Source Project

  • check if there is already an existing actively developed project (it is always better to participate then to do new project or fork it)
  • do some reseach why old and dead projects are failed
  • find a company or a group of users that wants to work with you to define the scope of the project

Its Not Just Software

  • most importent part is to create a communiy
  • you need to interact with that community
    • web pages
    • forum or knowledgebase, email lists, bug system
    • documentation and localization
    • packages, build systems
    • respository

You Need A Good Team And Active Community

  • mark a designated active leader (linux) or leaders (MariaDB/PostgreSQL) that have the respect from the community
  • ...

Transparency Is Critical For Long Time Success

  • a open development model
  • clear guidelines
  • clear license and business model
  • extensive documentation
  • be clear about your roadmap and release schdule
  • good open review process of patches that enforces quality
  • be transparent with your plans and let users influence them
  • be open about your bugs

Communicate With Your Community

  • attend conferences and talk about your product
  • listen to your users and either do it or help them to help themselves
  • make it easy for people to communicate with you and find informations about your product
  • ensure that your faq is up to date
  • ...

Be Good Open Source Citizens

  • building a supportive community later in the game is extreamly hard - you have to do it at day 1
  • ...

You Need To Get The Product Out There And Used

  • release early, release often
  • aim that each release should be bug free enought to be usable in production
  • work with the distributions and cloud providers to get your product in there
  • use a good open toolset ...

Doing Business With Open Source

Open source is a philosophy and a development model.

Open source does not gurantee that you will make enough money.

Different Kind Of Open Source Products

  • products developed by the community
  • products that are tools for the company and released as open source to get more community development
  • small team or company that is developing and driving ...

Questions To Consider When Creating A Company

  • do you plan for a virtual company (no offices)
  • are you creating a company of equals (hacker companies where company is owned by employes)
  • do you want to concentrate on services or development
  • do you plan th have a big community or work with a few big companies
  • do you plan to take in investors (if yes, you need an exit plan)

What Business Model To Choose

  • service company
    • man powered services (support, training, consulting)
    • valuation 2 times revenue
  • software company
    • licensing, software as a service (SAAS), subscription
    • valuation 10 times revenue plus x number of users

What Is The Final Goal With The Company

  • sold on the market (quick and high profit for owners, unpredictable future for employees)
  • go public on the stock market
  • owned by founders, employees
  • create an open source foundation

Why Go Open Source

  • spread the product more quickly
  • get some part of the development done elsewhere
  • get things more tested and more bug reports
  • possible to get development done in "not business critical" directions
  • easier to find good developers, partners and customers

In general, more market recognition, feedbacks ...

Reasons For Users To Trust A Open Source Vendor

  • more trustworthy as they depend on trust to survive
  • no vendor lock in (can fork it or publish patches each time)
  • little risk for trap doors

Benefits For Developers Using Open Source Vendor

  • easy to get access ...

Benefits For Big Business / Countries Using Open Source

  • no license costs
  • not depending on vendors (from different countries)
  • ...

When Go Open Source

  • ...

How To Choose An OS License

  • what is your business idea ...

Business Models To Use With Open Source

  • open core model (SugarCRM)
  • dual licensing model
  • service models
  • subscriptions

Open Core

  • its a closed source business model (like oracle and mysql)
  • ...
  • for cummunity developers, the worst possible offer

Dual Licensing

  • used first by ghostscript and mysql
  • can only be used if you have full rights to all the code
  • give same code under two licenses
  • companies that can not use the gpl have to buy the closed source version from you
  • only works well for infrastructure, easily embeddable produts like libraries or databases

Business Source (Delayed Open Source)

  • not an open source license
  • source code is available from start but using it commercially you have to pay
  • after x years, the code automaticly converts to some open source/free license
  • better than open core
  • investor friendly (early adaptors have to pay)
  • forces good behaviour
  • you to do frequently releases
  • you to innovate and fix bugs to ensure users wants it
  • ensures that company is bought by "bad entity" that just wants to close down the project
  • project can be take over if something happens to the company

Recommendatins For Business Source

  • 3 to 5 years before it becomes open source
  • target that 1/100 or 1/1000 users have to pay
  • free version and commercial should be identical

The Importance Of Selling Licensing

  • mysql was only possibly with the licenses
  • very hard to get companies to pay
  • ...

Business Differnces Between MySQL and MariaDB

  • ...