Comment 'I did notice when te...' to 'I still need insight on how to encode videos larger than 2gb.'
Comment to I still need insight on how to encode videos larger than 2gb.

  • I did notice when testing different servers for our UNA Hosting that when using UNA in a Shared Linux environment, large videos up to 1.5GB - 2GB it can be hit and miss if a video gets converted.

    I found this is for a number of reasons.

    • 2 Cores or less can take FFMPEG over 30-40 minutes for a 1.5-2GB file can take. 
      • Most shared Linux Plans use up to 2 cores and restricted.
      • This can cause the server to see it as abuse depending on the server setup and keep on killing the process.
      • If running CSF by default memory and CPU usage is low and set to kill processes over 128MB RAM.
      • Also shared hosting also has low limits set for max_execution_time, max_input_time, memory_limit, post_max_size, upload_max_filesize and can effect encoding.
    • I found that more than 1 video being uploaded over 1.5GB, UNA will only encode the first video.
      • It seems that because the first video takes so long on a shared environment that it forgets to covert the other videos in the upload.
        • I watched the FFMPEG process closely. It also stopped encoding during points and took 10-20 mins to restart encoding.
      • I have noticed once or twice that sometimes the second video will be converted the next day if a big file.

    Compared to a dedicated or a vps that has high CPU resources. 

    • 4 Cores + is best for minimum usage of FFMPEG
      • 2GB video can encode in less than 10-20 minutes depending on HD and SD videos.
    • You can upload up to 3 large videos and not have an issue. More then 3 videos you would need more processor power.
      • I found while testing that the more CPU power the faster FFMPEG converts, the more videos you can upload at the time of upload.
    • RAM really has no relevance to FFMPEG as it is CPU it depends on more.

    For my busy community, I personally use a Dedicated server with 6 cores, and video uploading is nice and smooth. 

    I would prefer for the following though.

    1. Videos to start uploading as soon as upload of the video is complete and not when all uploads are complete and the album has been submitted. 
      1. This would decrease encoding time.
    2. For 2 videos to encode at a time instead of one at a time.
      1. Of course, you would need 6+ Cores.
      2. It would decrease the time to encode all the videos.

    A simple approach of running them all at the same time would be to call them as background processes:

    #!/bin/bash 

    ffmpeg -i input1.mp4 -vcodec mpeg4 -qscale:v 1 -aspect 16:9 output1.avi & 

    ffmpeg -i input2.mp4 -vcodec mpeg4 -qscale:v 1 -aspect 16:9 output2.avi & 

    ffmpeg -i input3.mp4 -vcodec mpeg4 -qscale:v 1 -aspect 16:9 output3.avi & 

    ffmpeg -i input4.mp4 -vcodec mpeg4 -qscale:v 1 -aspect 16:9 output4.avi & 

    ffmpeg -i input5.mp4 -vcodec mpeg4 -qscale:v 1 -aspect 16:9 output5.avi & 

    ffmpeg -i input6.mp4 -vcodec mpeg4 -qscale:v 1 -aspect 16:9 output6.avi &

    • Thank you for the suggestions and your investigation.

      1. Video transcoding already starts as soon as it's uploaded, you don't need to even submit the form, however since conversion is run under cron, it's run in 1 minute interval, so it starts in 1-59 sec interval, so in 30 sec on average.

      2. https://github.com/unaio/una/issues/3085

      • Hi Alex,

        Sorry but I have tested the following

        • Posting Update To Timeline
          • Encoding starts on cron.
          • Encoding will not start unless "Post" has been clicked. 
            • Uploaded a video and pressed "Post" straight after upload and it started to encode when cron ran after posting.
            • Uploaded a video a waited 3 minuets for encoding to start. Cron ran 3 times and the video did not encode. One pressed "Post" the video started encoding straight away once the next cron ran.
          • Have to wait for video to finish encoding before watching.
          • Do not get an update of video encoding.
        • Posting a Post
          • Encoding starts on cron and before clicking "Post".
          • Can start watching before the video is encoded.
          • Do not get a update that the video is finished encoding.
        • Discussions
          • Video don't encode at all. And if it don't encode and we can watch it straight away can't that be done for Updates, Posts and Albums.
          • Attachment has no thumbnail.
          • Video works straight away.
          • On timeline it don't show that the discussion has a video attached. Might be handy to show it has. 
          • On Post you can insert media, maybe do this for videos in discussions.
        • Albums
          • Encoding starts on cron and before clicking submit.
          • Have to wait for video to finish encoding before watching.
          • Do not get an update of video encoding.

        I do like that discussions bypass encoding, to be honest, I wish videos did not need encoding at all, it would save on CPU resources allowing for UNA to be run on low-end servers. And also videos would be instant. 

        I understand it's for compression and encryption, but I would be happy to have the media stored on a secure data storage server or similar to keep the data safe and as for compression I would be happy to sacrifice space to bypass FFMPEG. 

        • Thank you for checking, yes, it looks like videos start to transcode upon submit only in some modules, thank you for stating this. 

          Actually Discussions don't have video encoding at all, video file can have different codecs and most of them aren't playable in web browser, so transcoding is required to make videos playable in browser, also it makes sure that videos have consistent resolution and quality to be playable using average internet connection speed.