How to use (django-celery,RQ) worker to execute a

2019-07-25 11:55发布

问题:

One part of my website includes a form that allows users to upload video. I use ffmpeg to convert the video to flv. My media and static files are stored on Amazon S3. I can get everything to work perfectly locally, however I can't seem to figure out how to use a worker to run the video conversion subprocess in production. I have dj-celery and rq installed in my app. The code in my view that I was able to get to work locally is:

#views.py
def upload_broadcast(request):
    if request.method == 'POST':
        form = VideoUploadForm(request.POST, request.FILES)
        if form.is_valid():
            new_video=form.save()
            def convert_to_flv(video):
                filename = video.video_upload
                sourcefile = "%s%s" % (settings.MEDIA_ROOT, filename)
                flvfilename = "%s.flv" % video.id
                imagefilename = "%s.png" % video.id
                thumbnailfilename = "%svideos/flv/%s" % (settings.MEDIA_ROOT, imagefilename)
                targetfile = "%svideos/flv/%s" % (settings.MEDIA_ROOT, flvfilename)
                ffmpeg = "ffmpeg -i %s -acodec mp3 -ar 22050 -f flv -s 320x240 %s" % (sourcefile, targetfile)
                grabimage = "ffmpeg -y -i %s -vframes 1 -ss 00:00:02 -an -vcodec png -f rawvideo -s 320x240 %s" % (sourcefile, thumbnailfilename)
                print ("SOURCE: %s" % sourcefile)
                print ("TARGET: %s" % targetfile)
                print ("TARGET IMAGE: %s" % thumbnailfilename)
                print ("FFMPEG TASK CODE: %s" % ffmpeg)
                print ("IMAGE TASK CODE: %s" % grabimage)
                try:
                    ffmpegresult = subprocess.call(ffmpeg)
                    print "---------------FFMPEG---------------"
                    print ffmpegresult
                except:
                    print "Not working."
                try:
                    videothumbnail = subprocess.call(grabimage)
                    print "---------------IMAGE---------------"
                    print videothumbnail
                except:
                    print "Not working."
                video.flvfilename = flvfilename
                video.videothumbnail = imagefilename
                video.save()

            convert_to_flv(new_video)

            return HttpResponseRedirect('/video_list/')
    else:
...

This is my first time trying to use a worker (or ever pushing a project to production), so even with the documentation it is still unclear to me what I need to do. I have tried several different things but nothing seems to work. Is there just a simple way to tell celery to run the ffmpegresult = subprocess.call(ffmpeg)? Thanks in advance for any help or insight.

EDIT- Added heroku logs

2013-01-10T20:58:57+00:00 app[web.1]: TARGET: /media/videos/flv/8.flv
2013-01-10T20:58:57+00:00 app[web.1]: IMAGE TASK CODE: ffmpeg -y -i /media/videos/practice.wmv -vframes 1 -ss 00:00:02 - an -vcodec png -f rawvideo -s 320x240 /media/videos/flv/8.png
2013-01-10T20:58:57+00:00 app[web.1]: SOURCE: /media/videos/practice.wmv
2013-01-10T20:58:57+00:00 app[web.1]: FFMPEG TASK CODE: ffmpeg -i /media/videos/practice.wmv -acodec mp3 -ar 22050 -f fl v -s 320x240 /media/videos/flv/8.flv
2013-01-10T20:58:57+00:00 app[web.1]: TARGET IMAGE: /media/videos/flv/8.png
2013-01-10T20:58:57+00:00 app[web.1]: Not working.
2013-01-10T20:58:57+00:00 app[web.1]: Not working.

NEWER EDIT

I tried adding a tasks.py and added the task:

celery = Celery('tasks', broker='redis://guest@localhost//')

@celery.task
def ffmpeg_task(video):
    converted_file = subprocess.call(video)
    return converted_file

then I changed the relevant section of my view to:

...
try:
    ffmpeg_task.delay(ffmpeg)
    print "---------------FFMPEG---------------"
    print ffmpegresult
except:
    print "Not working."
...

My new logs are:

2013-01-15T13:19:52+00:00 app[web.1]: TARGET IMAGE: /media/videos/flv/12.png
2013-01-15T13:19:52+00:00 app[web.1]: SOURCE: /media/videos/practice.wmv
2013-01-15T13:19:52+00:00 app[web.1]: FFMPEG TASK CODE: ffmpeg -i /media/videos/practice.wmv -acodec mp3 -ar 22050 -f fl v -s 320x240 /media/videos/flv/12.flv
2013-01-15T13:19:52+00:00 app[web.1]: IMAGE TASK CODE: ffmpeg -y -i /media/videos/practice.wmv -vframes 1 -ss 00:00:02 -an -vcodec png -f rawvideo -s 320x240 /media/videos/flv/12.png
2013-01-15T13:19:52+00:00 app[web.1]: TARGET: /media/videos/flv/12.flv
2013-01-15T13:20:17+00:00 app[web.1]: 2013-01-15 13:20:17 [2] [CRITICAL] WORKER TIMEOUT (pid:12)
2013-01-15T13:20:17+00:00 app[web.1]: 2013-01-15 13:20:17 [2] [CRITICAL] WORKER TIMEOUT (pid:12)
2013-01-15T13:20:17+00:00 app[web.1]: 2013-01-15 13:20:17 [19] [INFO] Booting worker with pid: 19

Am I completely missing something? I'll keep trying, but will be very appreciative of any direction or assistance.