Celery#

Logging#

Mission: Log to syslog with a single logging config (logging.yaml):

version: 1
disable_existing_loggers: True

root:
  level: INFO
  handlers:
    - console
    - syslog

loggers:
  celery:
    level: INFO

handlers:
  console:
    class: logging.StreamHandler
    formatter: generic
    stream: ext://sys.stdout
  syslog:
    class: logging.handlers.SysLogHandler
    formatter: generic
    address: '/dev/log'

formatters:
  generic:
    format: '%(name)s[%(process)d] [%(levelname)s] [%(module)s:%(lineno)d] %(message)s'
    datefmt: '%Y-%m-%d %H:%M:%S'

Tip with ‘/dev/log’ from http://stackoverflow.com/questions/6205254/how-to-setup-sysloghandler-with-django-1-3-logging-dictionary-configuration.

In Celery there are two different loggers:

global logger for the main daemon
task loggers

Global Logger#

To configure the global logger:

import yaml
import logging.config
logging_config_dict = yaml.load(open('/etc/my-service/logging.yaml'))
logging.config.dictConfig(logging_config_dict)

Task Logger#

Whereever you put your tasks:

from celery.utils.log import get_task_logger
log = get_task_logger(__name__)

@after_setup_task_logger.connect
def configure_task_logger(**kwargs):
    import yaml
    import logging.config
    logging_config_dict = yaml.load(open('/etc/service/logging.yaml'))
    logging.config.dictConfig(logging_config_dict)

See Also#