Skip to content

FrOSCon - Beyond LAMP

First talk finished :-). No proofreading done so far. Next talk will start in a few seconds.

Its All About Scale

  • size
  • features
  • non-functional requirements
  • Language agnostic - language you are using doesn't matter

Background Tasks

User can't wait and server can't handle it

  • sending mail
  • calling 3rd party APIs / services
  • converting media (image, video, audio)
  • updating caches

Direct Approach

Don't do that since your are losing control

Slightly Less Bad Idea

  • Write jobs into sql database
  • have some workers that poll the database for jobs

Don't use it

Queus To The Rescue

  • write jobs into message queue
  • have some workers for the queue
  • many services available (ZeroMQ, RabbitMQ, Redis Pub/Sub, Amazon SQS ...)

Good idea but lot of work and hard to get right

Beanstalkd

  • create a job
  • process a job

RQ

  • simple job queue in python
  • backed by redis
  • enqeueu function call
  • run worker how stores result in database

Celery

  • python
  • multiple brokers
  • primarily python with clients for ruby and php
  • highly available (HA)
  • fast
  • flexible
  • monitoring
  • workflows
  • time and rate limits
  • scheduling
  • autoscaling

Search

Like query

  • no lingustic suport
  • no ranking
  • not indexed (mysql has fulltext index and postgresql)

Fulltext Search Engines - Solr Vs. Elastic Search

  • based on lucene
  • full linguistic support
  • faceted search
  • result highlighting
  • HTTP API
  • similar but different
  • easy to set up
  • clients available like PHP: solarium, symfony component or PHP Elastica
  • integrate search into framework (map objects into documents and back) or handeling updates and deletes

NoSQL

  • key-value stores
  • column based
  • document stores
  • graph

Errors

  • don't let it happen
  • send an email (overload mailboxes)
  • log errors
    • php symfony provides a component that you log only all messages if error occures (so only info logging if something happens)
    • properly configure logging
    • use event aggregator
      • sentry
      • errbit

Sentry (event aggregator)

  • started as OSS
  • now available as SaSS
  • udp
  • written in django
  • php client available

Errbit (event aggretator)

  • still OSS

Deployment

  • ftp
  • git pull / app server by ssh
  • but you want a automated deployment like fab deploy
  • automated deployment with webistrano (just a single button)

Wrap Up

  • use background tasks
  • use full text search
  • try to use nosql where needed
  • do error logging
  • think about deployment

bash function to search and replace in all files with special extension recursively

Pretty much, the code is below. It is a part of my function collection for the bash.

####
# Replaces a string in all files in given path and below
# taken from: http://www.cyberciti.biz/faq/unix-linux-replace-string-words-in-many-files/
# taken from: http://stackoverflow.com/questions/4437901/find-and-replace-string-in-a-file
# taken from: http://stackoverflow.com/questions/7450324/how-do-i-replace-a-string-with-another-string-in-all-files-below-my-current-dir
#
# @author stev leibelt 
# @since 2013-7-30
####
function net_bazzline_replace_string_in_files ()
{
    if [[ $# -lt 3 ]]; then
        echo 'invalid number of arguments provided'
        echo 'command search replace fileextension [path]'
        return 1
    fi

    if [[ $# -eq 4 ]]; then
        find "$4" -name "*.$3" -type f -exec sed -i 's/'"$1"'/'"$2"'/g' {} \;
    else
        find . -name "*.$3" -type f -exec sed -i 's/'"$1"'/'"$2"'/g' {} \;
    fi
}

VI/VIM - Search

date: 111025 search

:set [ic|noic]
sets globaly the search to ["ignore case"|"not ignore case"] (not to be or be a case sensitive searcher ;-) ).
%s/myMixedCaseWord/ReplacedWord/gi
let you search/replace with ignored case
/thIsISmyCaseInsensitVSearch\c

searchs for your term with ignored case

Links yolinux.com - tutorial - vim vim.wikia.com

date: 111229 search and replace

:$rs/searchterm/replaceterm/$a
$r (the range) can be: % - the whole file 23 - works on line 23 $ - nothing, only for current line. $a (the argument) can be: g - whole line i - be case insensitive I - be case sensitive c - commit every change [y|n] $ - nothing, only first searchterm will be replaced. tuxfiles.org