2008-12-31

桑梓贴吧的临时急救方案

  桑梓贴吧在教育网内还是比较不错的在线视频站点,但是由于完成的比较早,并且采用的核心程序Mplayer和FFMpeg都是一个还没有很完美的release版本的项目,因此在服务器环境不断升级的过程中,后台程序上自然遇到一些问题。

  最近修复的桑梓贴吧由于之前系统严重错误,重新部署了新版的操作系统,在重新构架编码环境的时候就遇到了问题:

my string: MPlayer 1.0rc2-4.2.3 (C) 2000-2007 MPlayer Team
my string: CPU: Intel(R) Xeon(TM) CPU 2.40GHz (Family: 15, Model: 2, Stepping: 5)
my string: CPUflags: MMX: 1 MMX2: 1 3DNow: 0 3DNow2: 0 SSE: 1 SSE2: 1
my string: Compiled with runtime CPU detection.
mplayer: could not connect to socket
mplayer: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.
my string:
my string: Playing /813f77fa8bbd055850d1b84d0e95f7e2.wmv.
my string: ASF file format detected.
my string: ID_AUDIO_ID=1
my string: [asfheader] Audio stream found, -aid 1
my string: ID_VIDEO_ID=2
my string: [asfheader] Video stream found, -vid 2
my string: VIDEO: [WMV3] 640x480 24bpp 1000.000 fps 736.0 kbps (89.8 kbyte/s)
my string: Clip info:
my string: name: 鎴戞剾鍙板
my string: ID_CLIP_INFO_NAME0=name
my string: ID_CLIP_INFO_VALUE0=鎴戞剾鍙板
my string: author: MC Hot Dog
my string: ID_CLIP_INFO_NAME1=author
my string: ID_CLIP_INFO_VALUE1=MC Hot Dog
my string: copyright: 婊剧煶鍞辩墖
my string: ID_CLIP_INFO_NAME2=copyright
my string: ID_CLIP_INFO_VALUE2=婊剧煶鍞辩墖
my string: ID_CLIP_INFO_N=3
my string: ID_FILENAME=/813f77fa8bbd055850d1b84d0e95f7e2.wmv
my string: ID_DEMUXER=asf
my string: ID_VIDEO_FORMAT=WMV3
my string: ID_VIDEO_BITRATE=736000
my string: ID_VIDEO_WIDTH=640
my string: ID_VIDEO_HEIGHT=480

Is OK:480


0:-of,1:lavf,2:-lavfopts,3:i_certify_that_my_video_stream_does_not_use_b_frames,4:-oac,5:mp3lame,6:-lameopts,7:abr:br=56,8:-ovc,9:lavc,10:-lavcopts,11:vcodec=flv:mbd=2:mv0:trell:v4mv:cbp:last_pred=3:dia=4:cmp=6:vb_strategy=1:vbitrate=500,12:-vf,13:scale=320:240,14:-ofps,15:12,16:-srate,17:22050,18:/8bbd055850d1b84d0e95f7e2.wmv,19:-o,20:/813f77fa8bbd055850d1b84d0e95f7e2.flv
MEncoder 2:1.0~rc2-0ubuntu13 (C) 2000-2007 MPlayer Team
CPU: Intel(R) Xeon(TM) CPU 2.40GHz (Family: 15, Model: 2, Stepping: 5)
CPUflags: Type: 15 MMX: 1 MMX2: 1 3DNow: 0 3DNow2: 0 SSE: 1 SSE2: 1
Compiled with runtime CPU detection.
Option lavfopts: Unknown suboption i_certify_that_my_video_stream_does_not_use_b_frames
Error parsing option on the command line: -lavfopts

Exiting... (error parsing command line)
  其实这里的问题在于 i_certify_that_my_video_stream_does_not_use_b_frames这个参数错误,mencoder在新版中不支持此参数,因为mencoder已经能够正确的生成带有b帧的视频文件了。这一点可以在我前几天发的文章中看到解释。
  其实只需要去除掉这个参数即可,不过严重的问题在于,后台程序转换的参数是写在代码里面的,而后台程序的源代码又遗失了,所以现在只能从外部着手了,于是,我写了个fakemencoder程序,接受原来程序传递个mencoder的参数,之后替换其中无效的参数,再将替换后的参数传递个mencoder进行编码。这样就解决了这个问题。

  事情本来到这里就结束了,不过没想到桑梓贴吧的后台程序出现异常,所有视频都无法成功转换,经过严密的排查,最后定位到,程序在移动视频文件的时候出现异常,导致视频文件无法正常被移动,或者是后台程序在删除临时文件的时候把目标文件也删除了。这样一来,结果变成了转换完成的视频丢失了。
  至此,问题大发了。。。
  由于联系不到其他前辈,现在的问题无法查出原因,所以目前,只能采取临时的应急方案了。
  还是从fakemencoder程序入手,我们在之前的测试中发现,其实桑梓贴吧判断视频转换成功与否的原理只是扫描目标文件夹下是否存在对应的flv文件而已,因此,我的做法是,在fakemencoder文件里面执行完成mencoder的命令后立刻把目标文件从临时文件夹拷贝的目标文件夹,这样以来居然真的成功使得贴吧出现了转换完成的状态。
  但是,果然还是没有这么容易就搞定,结果又发现问题:转换成功的视频无法播放,再次经过测试,发现,贴吧后台程序转换视频用的mencoder的命令是:
mencoder. -vf scale=320:192 -ffourcc FLV1 -lavdopts er=1 -pp 6 -sws 2 -of lavf -ovc lavc -lavcopts vcodec=flv:vhq:turbo:vbitrate=500 -srate 22050 -oac mp3lame -lavcopts acodec=mp3:abitrate=128 /tempvideo/2801493.avi -o /flvtemp/edab85785c473ad7e0d6a6a46b000673.flv
  经过测试发现把--ffourcc FLV1去除我的mplayer即可正常播放,但是google出来的结果称,flash播放器可能不能正常播放,于是,搜索了youtube的原理之后发现只要增加一个metadata即可。
  要增加metadata只需要用一个开源工具flvtool就能轻松搞定:
flvtool2 -U targetfile.flv

这样解决方案就出炉了,看我源代码:



#include <iostream>
#include <stdlib.h>
#include <string.h>
using namespace std;
/*
*
*/
int main(int argc , char** argv) {

char *s=" ";
char *st="mencoder.bak ";
char *oadd = new char[100];
char *mencoder;
mencoder = new char[1000];
*mencoder = '\0';
mencoder = strcat(mencoder, st);
for (int i = 1 ; i < argc ; i++) {
if(strcmp(argv[i],"-lavfopts")==0 || strcmp(argv[i], "i_certify_that_my_video_stream_does_not_use_b_frames")==0 || strcmp(argv[i],"-ffourcc")==0 || strcmp(argv[i],"FLV1")==0 ) {
continue;
}
mencoder = strcat(mencoder, argv[i]);
mencoder = strcat(mencoder , s);
}

system(mencoder);
//copy to flv/ directory
char *sss=new char[50];
*sss='\0';
sss = strcat(sss,"cp ");
sss = strcat(sss,argv[argc-1]);
sss = strcat(sss," ");
oadd = strcpy(oadd,argv[argc-1]);
*strrchr(oadd,'/')='\0';
*strrchr(oadd,'/')='\0';
sss= strcat(sss,oadd);
sss= strcat(sss,"/flv");
sss= strcat(sss, strrchr(argv[argc-1],'/'));
cout <<"\nEncoder command Line:"<<"\n"<<mencoder <<"\n: copy file : "<<sss << endl;

system(sss);

//write mate-data into flv file
oadd = strcpy(oadd,"flvtool2 -U ");
oadd = strcat(oadd,argv[argc-1]);
*strrchr(oadd,'/')='\0';
*strrchr(oadd,'/')='\0';
oadd = strcat(oadd,"flv");
oadd = strcat(oadd, strrchr(argv[argc-1],'/'));
cout<< "\nWriting META_DATA by:" << oadd << endl;
system(oadd);
return (EXIT_SUCCESS);
}


  如果需要了解这种在线视频网站的原理,可以看这篇文章,我把主要部分引用过来:

Converting the video for use in the site

The users are asked to upload AVI videos to the site, but we cannot play AVI videos directly in the browser (at least not in a browser-independent manner). A good way to publish videos for viewing in a browser is using FLV (Flash(tm) Video) format. This is what YouTube, Google Video and a whole host of other sites use. If it’s good enough for them, it’s good enough for me!

Converting to FLV

So, how to convert the AVI video the user uploaded to a usable FLV format? Luckily, the OSS package ffmpeg [2] provides this conversion functionality (plus a wide range of other video conversion features one of which we’ll come to later on). The good thing about ffmpeg is that it is controlled entirely from the command-line and can be run in headless environments — this is vital for using it on an application server. Most other FLV conversion tools were either for Windows or came with some form of Gnome or KDE gui which wouldn’t have worked on my hosted Linux box.

The basic command for converting a video into FLV format is (see [1] in resources):

ffmpeg -i [sourcefile.avi] -acodec mp3 -ar 22050 -ab 32 -f flv -s 320×240 [destfile.flv]

Adding FLV metadata

This command creates a simple FLV format file, containing the video and audio streams. In addition, FLV files need meta-information such as duration, frames, etc. FLV movie players use this information to calculate progress bar sliders and allow the user to fast-forward or reverse through the video. For reasons which I didn’t bother to research, ffmpeg does not add this information. But there is a package which can: flvtool2 (see [3]). Using this tool, we can add FLV meta-information with the following command:

flvtool2 -U [flvfile]

(Warning, Djangoists — flvtool2 is written in Ruby. Please check your religious language preferences at the door and pick them up as you leave. Thank you).

Adding a video thumbnail

Blog entries in trogger.de can include pictures uploaded by the users. One of these pictures is displayed as a small preview when showing the blog posting (e.g. in the blog overview, or in the list of the latest blog submissions). Wouldn’t it be nice if we could also add a thumbnail for a video submission, so that the blog’s reader can get a first idea of what to expect? I think it would. And, again, ffmpeg comes to the rescue.

ffmpeg can extract single frames from a video stream, storing them in still image format. The command for doing this is:


ffmpeg -y -i [videofile] -vframes 1 -ss 00:00:02 -an -vcodec png -f rawvideo -s 320×240 [thumbnailimage.png]

Putting it all together

With these individual steps, it’s EASY to put together a video conversion function which kicks in once a user has uploaded a video file. Since we have the information which video the user uploaded with the form, we convert this video into FLV format, add metadata and create a thumbnail image:

def convertvideo (video):
if video is None:
return "Kein Video im Upload gefunden"
filename = video.videoupload
print "Konvertiere Quelldatei: %s" + filename
if filename is None:
return "Video mit unbekanntem Dateinamen"
sourcefile = "%s%s" % (settings.MEDIA_ROOT,filename)
flvfilename = "%s.flv" % video.id
thumbnailfilename = "%svideos/flv/%s.png" % (settings.MEDIA_ROOT, video.id)
targetfile = "%svideos/flv/%s" % (settings.MEDIA_ROOT, flvfilename)
ffmpeg = "ffmpeg -i %s -acodec mp3 -ar 22050 -ab 32 -f flv -s 320x240 %s" % (sourcefile, targetfile)
grabimage = "ffmpeg -y -i %s -vframes 1 -ss 00:00:02 -an -vcodec png -f rawvideo -s 320x240 %s " % (sourcefile, thumbnailfilename)
flvtool = "flvtool2 -U %s" % targetfile
print ("Source : %s" % sourcefile)
print ("Target : %s" % targetfile)
print ("FFMPEG: %s" % ffmpeg)
print ("FLVTOOL: %s" % flvtool)
try:
ffmpegresult = commands.getoutput(ffmpeg)
print "-------------------- FFMPEG ------------------"
print ffmpegresult
# Check if file exists and is > 0 Bytes
try:
s = os.stat(targetfile)
print s
fsize = s.st_size
if (fsize == 0):
print "File is 0 Bytes gross"
os.remove(targetfile)
return ffmpegresult
print "Dateigroesse ist %i" % fsize
except:
print sys.exc_info()
print "File %s scheint nicht zu existieren" % targetfile
return ffmpegresult
flvresult = commands.getoutput(flvtool)
print "-------------------- FLVTOOL ------------------"
print flvresult
grab = commands.getoutput(grabimage)
print "-------------------- GRAB IMAGE ------------------"
print grab
except:
print sys.exc_info()
return sys.exc_info[1]
video.flvfilename = flvfilename
video.save()
return None

没有评论: