The video is rendered with ffmpeg with the "faststart" flag added meaning the metadata should be at the start of the file, and the server appears to be handling partial content requests correctly, so why does it need to have downloaded so much of the video before the player becomes enabled and can play the video? I am testing it in Google Chrome.
Once the player becomes enabled I can seek around to various points in the video pretty instantly and see the new partial content requests being made.
Here is a link to the video: http://iptv.la1tv.co.uk/unibrass.mp4
Here is a jsbin with the video tag: https://jsbin.com/rahewidoru . It takes a few minutes but does work after it loads.
Any suggestions?
I realise there are other methods like HLS and dash which use chunks, but I would like to know why it isn't working this way because I can't find anywhere that provides a reason as to why it doesn't work well.
The problem lies in the duration of the video, I don't know the exact duration of the video, but I can estimate based in the video size.
The reason is the video player implementation reads some pieces around the video to take some information information for the playback and forward and to ensure that the header is well formed.
For this reason is very important make use of streaming protocols such HLS o DASH because the situation of time stamps is different and it is optimized for streaming.
So it looks like it is just how h.264 works. The metadata contains pointers to all the keyframes in the file, and therefore as the duration of the file increases, so does the number of keyframes, and hence the size of metadata at the beginning.
More information at http://www.adobe.com/devnet/video/articles/mp4_movie_atom.html
If anyone can provide a more in depth answer as to what the metadata in a h.264 encoded mp4 contains then let me know and I'll mark that as the answer, or just edit this.