How to setup celery worker to log all task functio

2019-08-22 21:16发布

I have Django application with such logging configuration.

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'default': {
            'format': '%(asctime)s [%(levelname)s] %(filename)s:%(lineno)s: %(message)s'
        },
    },
    'handlers': {
        'cron': {
            'class': 'logging.FileHandler',
            'filename': 'cron.log',
            'formatter': 'default',
        },
        'admin': {
            'class': 'logging.FileHandler',
            'filename': 'admin.log',
            'formatter': 'default',
        },
        'app': {
            'class': 'logging.FileHandler',
            'filename': 'app.log',
            'formatter': 'default',
        },
        'core': {
            'class': 'logging.FileHandler',
            'filename': 'app.log',
            'formatter': 'default',
        },
        'metrics': {
            'class': 'logging.FileHandler',
            'filename': 'metrics.log',
            'formatter': 'default',
        },
        'pixel': {
            'class': 'logging.FileHandler',
            'filename': 'pixel.log',
            'formatter': 'default',
        },
    },
    'loggers': {
        'cron_api': {
            'handlers': ['cron'],
            'level': 'DEBUG',
        },
        'main': {
            'handlers': ['app'],
            'level': 'INFO',
        },
        'admin': {
            'handlers': ['admin'],
            'level': 'INFO',
        },
        'spa_api': {
            'handlers': ['app'],
            'level': 'INFO',
        },
        'metrics': {
            'handlers': ['metrics'],
            'level': 'INFO',
        },
        'pixel': {
            'handlers': ['pixel'],
            'level': 'INFO',
        },
    },
}

Also I have a celery worker. I want to log every activity that happens inside the task to celery_worker.log, even if task calls a function from main or cron_api application.

For example: If main.celery_tasks.task1.task1 calls main.service.google.get_campaign, I want all logs from get_campaign to be in worker.log. Also I want no logs related to this task to be in app.log.

How to achieve this? P.S.: Also I'd like to have a scalable solution. If number of worker increases - I'd like each worker to write all logs related to task call to it's file.

0条回答
登录 后发表回答