Skip to content

FrOSCon - Beyond LAMP

First talk finished :-). No proofreading done so far. Next talk will start in a few seconds.

Its All About Scale

  • size
  • features
  • non-functional requirements
  • Language agnostic - language you are using doesn't matter

Background Tasks

User can't wait and server can't handle it

  • sending mail
  • calling 3rd party APIs / services
  • converting media (image, video, audio)
  • updating caches

Direct Approach

Don't do that since your are losing control

Slightly Less Bad Idea

  • Write jobs into sql database
  • have some workers that poll the database for jobs

Don't use it

Queus To The Rescue

  • write jobs into message queue
  • have some workers for the queue
  • many services available (ZeroMQ, RabbitMQ, Redis Pub/Sub, Amazon SQS ...)

Good idea but lot of work and hard to get right

Beanstalkd

  • create a job
  • process a job

RQ

  • simple job queue in python
  • backed by redis
  • enqeueu function call
  • run worker how stores result in database

Celery

  • python
  • multiple brokers
  • primarily python with clients for ruby and php
  • highly available (HA)
  • fast
  • flexible
  • monitoring
  • workflows
  • time and rate limits
  • scheduling
  • autoscaling

Search

Like query

  • no lingustic suport
  • no ranking
  • not indexed (mysql has fulltext index and postgresql)

Fulltext Search Engines - Solr Vs. Elastic Search

  • based on lucene
  • full linguistic support
  • faceted search
  • result highlighting
  • HTTP API
  • similar but different
  • easy to set up
  • clients available like PHP: solarium, symfony component or PHP Elastica
  • integrate search into framework (map objects into documents and back) or handeling updates and deletes

NoSQL

  • key-value stores
  • column based
  • document stores
  • graph

Errors

  • don't let it happen
  • send an email (overload mailboxes)
  • log errors
    • php symfony provides a component that you log only all messages if error occures (so only info logging if something happens)
    • properly configure logging
    • use event aggregator
      • sentry
      • errbit

Sentry (event aggregator)

  • started as OSS
  • now available as SaSS
  • udp
  • written in django
  • php client available

Errbit (event aggretator)

  • still OSS

Deployment

  • ftp
  • git pull / app server by ssh
  • but you want a automated deployment like fab deploy
  • automated deployment with webistrano (just a single button)

Wrap Up

  • use background tasks
  • use full text search
  • try to use nosql where needed
  • do error logging
  • think about deployment