Skip to content

FrOSCon - Beyond LAMP

First talk finished :-). No proofreading done so far. Next talk will start in a few seconds.

Its All About Scale

  • size
  • features
  • non-functional requirements
  • Language agnostic - language you are using doesn't matter

Background Tasks

User can't wait and server can't handle it

  • sending mail
  • calling 3rd party APIs / services
  • converting media (image, video, audio)
  • updating caches

Direct Approach

Don't do that since your are losing control

Slightly Less Bad Idea

  • Write jobs into sql database
  • have some workers that poll the database for jobs

Don't use it

Queus To The Rescue

  • write jobs into message queue
  • have some workers for the queue
  • many services available (ZeroMQ, RabbitMQ, Redis Pub/Sub, Amazon SQS ...)

Good idea but lot of work and hard to get right

Beanstalkd

  • create a job
  • process a job

RQ

  • simple job queue in python
  • backed by redis
  • enqeueu function call
  • run worker how stores result in database

Celery

  • python
  • multiple brokers
  • primarily python with clients for ruby and php
  • highly available (HA)
  • fast
  • flexible
  • monitoring
  • workflows
  • time and rate limits
  • scheduling
  • autoscaling

Search

Like query

  • no lingustic suport
  • no ranking
  • not indexed (mysql has fulltext index and postgresql)

Fulltext Search Engines - Solr Vs. Elastic Search

  • based on lucene
  • full linguistic support
  • faceted search
  • result highlighting
  • HTTP API
  • similar but different
  • easy to set up
  • clients available like PHP: solarium, symfony component or PHP Elastica
  • integrate search into framework (map objects into documents and back) or handeling updates and deletes

NoSQL

  • key-value stores
  • column based
  • document stores
  • graph

Errors

  • don't let it happen
  • send an email (overload mailboxes)
  • log errors
    • php symfony provides a component that you log only all messages if error occures (so only info logging if something happens)
    • properly configure logging
    • use event aggregator
      • sentry
      • errbit

Sentry (event aggregator)

  • started as OSS
  • now available as SaSS
  • udp
  • written in django
  • php client available

Errbit (event aggretator)

  • still OSS

Deployment

  • ftp
  • git pull / app server by ssh
  • but you want a automated deployment like fab deploy
  • automated deployment with webistrano (just a single button)

Wrap Up

  • use background tasks
  • use full text search
  • try to use nosql where needed
  • do error logging
  • think about deployment

Trackbacks

No Trackbacks

Comments

Display comments as Linear | Threaded

No comments

Add Comment

E-Mail addresses will not be displayed and will only be used for E-Mail notifications.
To leave a comment you must approve it via e-mail, which will be sent to your address after submission.

To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly.
CAPTCHA

Markdown format allowed
Form options