******
Celery
******
.. highlight:: python

Logging
=======
Mission: Log to syslog with a single logging config (``logging.yaml``):

.. code-block:: yaml

    version: 1
    disable_existing_loggers: True

    root:
      level: INFO
      handlers:
        - console
        - syslog

    loggers:
      celery:
        level: INFO

    handlers:
      console:
        class: logging.StreamHandler
        formatter: generic
        stream: ext://sys.stdout
      syslog:
        class: logging.handlers.SysLogHandler
        formatter: generic
        address: '/dev/log'

    formatters:
      generic:
        format: '%(name)s[%(process)d] [%(levelname)s] [%(module)s:%(lineno)d] %(message)s'
        datefmt: '%Y-%m-%d %H:%M:%S'

Tip with '/dev/log' from http://stackoverflow.com/questions/6205254/how-to-setup-sysloghandler-with-django-1-3-logging-dictionary-configuration.

In Celery there are two different loggers:

- global logger for the main daemon
- task loggers


Global Logger
-------------
To configure the global logger::

    import yaml
    import logging.config
    logging_config_dict = yaml.load(open('/etc/my-service/logging.yaml'))
    logging.config.dictConfig(logging_config_dict)


Task Logger
-----------
Whereever you put your tasks::

    from celery.utils.log import get_task_logger
    log = get_task_logger(__name__)

    @after_setup_task_logger.connect
    def configure_task_logger(**kwargs):
        import yaml
        import logging.config
        logging_config_dict = yaml.load(open('/etc/service/logging.yaml'))
        logging.config.dictConfig(logging_config_dict)


See Also
--------
- http://docs.celeryproject.org/en/latest/whatsnew-3.0.html#logging-improvements
- http://echorand.me/2012/08/14/celery-and-python-logging/
- http://stackoverflow.com/questions/18995020/django-celery-task-logging