Skip to content

web - living in a country with a telecom indsutry monopoly? Try to build your own network

[...]
Despite Mexico’s reputation for horrendous, Slim-driven telecom service and policy, it’s far from the only country that struggles with providing rural cell phone access. According to the GSM (for Global System for Mobile Communications, the standard technology behind a 2G network) Association, a consortium of commercial mobile providers from all over the world, 1.6 billion people in rural parts of developing countries don’t have access to mobile networks. That’s why Bloom and his collaborators at Rhizomatica say that if you really want to make the benefits of cell phones available to the people who need them most, it’s not enough to democratize the hardware by making the phones themselves super cheap. You have to democratize the infrastructure, the network itself. And that’s a lot harder to do.
[...]

source

I have no idea how much I have to honor the people around rhizomatica. They are doing things right and making the world better. In germany, we only have problems bring the internet to the people (and with internet, I mean something with higher bandwith) and even this is not an easy task (special thanks to the politics and the nearby telecom monopol). The good part about our internet problem, the hardware is there. To build up a mobile phone network (that is stable and scalls well) is something different.
Super cool that I was able to read that article. Faith in humanity restored, you know :-).

FrOSCon - Beyond LAMP

First talk finished :-). No proofreading done so far. Next talk will start in a few seconds.

Its All About Scale

  • size
  • features
  • non-functional requirements
  • Language agnostic - language you are using doesn't matter

Background Tasks

User can't wait and server can't handle it

  • sending mail
  • calling 3rd party APIs / services
  • converting media (image, video, audio)
  • updating caches

Direct Approach

Don't do that since your are losing control

Slightly Less Bad Idea

  • Write jobs into sql database
  • have some workers that poll the database for jobs

Don't use it

Queus To The Rescue

  • write jobs into message queue
  • have some workers for the queue
  • many services available (ZeroMQ, RabbitMQ, Redis Pub/Sub, Amazon SQS ...)

Good idea but lot of work and hard to get right

Beanstalkd

  • create a job
  • process a job

RQ

  • simple job queue in python
  • backed by redis
  • enqeueu function call
  • run worker how stores result in database

Celery

  • python
  • multiple brokers
  • primarily python with clients for ruby and php
  • highly available (HA)
  • fast
  • flexible
  • monitoring
  • workflows
  • time and rate limits
  • scheduling
  • autoscaling

Search

Like query

  • no lingustic suport
  • no ranking
  • not indexed (mysql has fulltext index and postgresql)

Fulltext Search Engines - Solr Vs. Elastic Search

  • based on lucene
  • full linguistic support
  • faceted search
  • result highlighting
  • HTTP API
  • similar but different
  • easy to set up
  • clients available like PHP: solarium, symfony component or PHP Elastica
  • integrate search into framework (map objects into documents and back) or handeling updates and deletes

NoSQL

  • key-value stores
  • column based
  • document stores
  • graph

Errors

  • don't let it happen
  • send an email (overload mailboxes)
  • log errors
    • php symfony provides a component that you log only all messages if error occures (so only info logging if something happens)
    • properly configure logging
    • use event aggregator
      • sentry
      • errbit

Sentry (event aggregator)

  • started as OSS
  • now available as SaSS
  • udp
  • written in django
  • php client available

Errbit (event aggretator)

  • still OSS

Deployment

  • ftp
  • git pull / app server by ssh
  • but you want a automated deployment like fab deploy
  • automated deployment with webistrano (just a single button)

Wrap Up

  • use background tasks
  • use full text search
  • try to use nosql where needed
  • do error logging
  • think about deployment