Get webrtc-like latency with ffmpeg

ffmpegvideo streaming

I've been using this between Chrome and my phone:

http://www.webrtc.org/demo

And the latency is really good – less than 1 second.

I've been trying to replicate that on my computer with no success.

ffmpeg -f video4linux2 -i /dev/video0  -s 320x200 -r 50 -deadline realtime -vcodec libvpx -f webm -fflags nobuffer udp://10.0.0.55:9002

And then using ffplay on the other side.

It's still got a couple of seconds of lag to it.

Eventually I'd like to stream from my computer to the Android phone, but the latency has got to be good.

Edit – this works significantly better. If I could shave just a bit off of this, I'd be happy:

ffmpeg -vcodec rawvideo -f video4linux2 -i /dev/video0  -s 320x200 -r 25 -vcodec libvpx -f rtp -deadline realtime rtp://10.0.0.55:9002

Best Answer

The problem is mostly from the fact that you are using software transcoding, instead of hardware transcoding.

As a rule of thumb, if the conversion uses the hardware acceleration, the latency will be of less-than-a-second order (usually milliseconds). If it is done in software, then the latency will be of more-than-a-second order.

FFmpeg supports hardware acceleration, but it is usually tricky to make it work for you.

https://trac.ffmpeg.org/wiki/HWAccelIntro

On the other hand, Google Chrome supports VP8 and H264 (where it is available) hardware encoding/decoding, both on your computer and your Android phone:

http://code.google.com/p/chromium/issues/detail?id=428223

Related Solutions

Command encoding h264 baseline profile, level 1, with FFmpeg and libx264

Make sure you've got the latest FFmpeg and libx264 – this is really important ^* – and then try something like:

ffmpeg -i … -c:v libx264 -profile:v baseline -level 1 …

This will result in:

[libx264 @ 0x10180fa00] profile Constrained Baseline, level 1.0

Of course, you can supply the -preset and -tune options to x264 as well. Refer to x264 --fullhelp for more info.

Here are the x264 options used:

cabac=0 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1
psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=0 
cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 
sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0
constrained_intra=0 bframes=0 weightp=0 keyint=250 
keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 
rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 
qpstep=4 ip_ratio=1.40 aq=1:1.00

_{* It works with FFmpeg 0.9 and x264 core 118. The -vprofile option was undocumented in FFmpeg, now it's -profile:v. It now basically works just like -profile, which again is supposed to replace -vpre. It uses x264's built-in profiles instead of relying on .ffpreset files.}

Low-latency streaming using ffmpeg

Streaming from desktopA to desktopB is an interesting problem. If you don't need video/audio, then you might want to check into the NX protocol instead. It is much more efficient than VNC. FreeNX is a server and NoMachine and others make NX clients. There are other servers - free and commercial.

There is also work being performed on a high performance remote desktop, SPICE, capability in as a F/LOSS project. I think Redhat is taking the lead. http://spice-space.org/

I don't know if these are useful answers, but perhaps just leads for something more useful?

For streaming video, avoiding the overhead of a full desktop protocol is probably needed.

Best Answer

Related Solutions

Command encoding h264 baseline profile, level 1, with FFmpeg and libx264

Low-latency streaming using ffmpeg

Related Question