Concatenating videos with FFMpeg

If you end up here, you’re either:

• Tearing your hair out with FFMpeg, trying to do something clever and cursing – a lot!! 
• Or you’ve followed my saga of the last few days on Twitter.

A bit of background:

I’m a long term user of the 1 Second Everyday app on Android, I’ve got 3 years worth of 1-1.5 second clips or images of our lives as a family taken daily that I mash up around New Year time as a review of the previous year. It’s become a pretty special family tradition to us and I’ve even managed to transfer the clips between phones etc in previous years.

This year however the app stopped working, at least for mashing up the video clips, I could still cut down a clip and get it down to one second, or grab a screenshot but the build would spend a LONG time saying it was building only to announce a failure. For those of you who remember loading tapes into computers to play a game and hoping you’d set the volume correctly you’ll know what I mean here.

I chatted with 1 Second Everyday support and it transpires that the original app was built by a third-party which meant that they were struggling to get it updated to run with the current Android build on my Pixel 3. However, they gave me a way to get out my 1 second clips so I could do with them what I wanted, whilst they built a new app with an in-house developer.

The big sticking point however, was that old clips could not be transferred to the new app 🙁 I was offered the chance to Alpha and Beta the new app, but I wasn’t willing to lose my year part way through, so I carried on until we got to January the 1st.

Concatenating films with FFmpeg

This is the part where I hope I can help someone, I don’t profess to know everything about these commands and how they work )but they certainly helped me) and as such this is more of a story than a tutorial.

I copied all the clips to my Mac and set about trying to build the film:

Initially I started in Laravel using this package: https://github.com/pascalbaljetmedia/laravel-ffmpeg

However, long timeouts quickly convinced me that this was not going to be a way forward to build this video, and actually some of what I was going to need when we got further down the line was going to need a MASSIVE command line that Laravel would end up building for me.

Concat Demuxer

I moved on to using the Concat Demuxer, creating a file list and injecting this into this command:

mylist.txt:

# this is a comment
file '/path/to/file1.mp4’
file '/path/to/file2.mp4’
file '/path/to/file3.mp4’

ffmpeg -f concat -safe 0 -i mylist.txt -c copy 2019_review.mp4

This built a large (correctly sized) video that ultimately froze the video part way through (around April) whilst the audio continued to run in the background and at some point later (around September) it returned to normal, after a lot of reading I came to the conclusion this was either different codecs or screen resolutions (most likely the latter give the way VLC behaved when playing)

Concat Filter

It was clear that this was going to have to be done the hard way, with the dreaded concat filter 🙂

I started building a couple of films using this command

ffmpeg \
	 -i ‘./20190001/clip.mp4' \
	 -i ‘./20190002/clip.mp4' \
	 -i ‘./20190003/clip.mp4' \
	 -i ‘./20190005/clip.mp4' \
	-filter_complex "\
\
	[0:v:0]scale=1920:1080[c1];\
	[1:v:0]scale=1920:1080[c2]; \
	[2:v:0]scale=1920:1080[c3]; \
	[3:v:0]scale=1920:1080[c4], \
	[c1] [0:a:0] \
	[c2] [1:a:0] \
	[c3] [2:a:0] \
	[c4] [3:a:0] \
	concat=n=4:v=1:a=1 [v] [a]" -map "[v]" -map "[a]" ‘./videos/output_2019.mp4' -y

This would re-encode the clips to 1920×1080 and join them together, and initial results were very positive, until we hit an image rather than a video clip. The problem here was the parts [?:a:0] towards the end of the command –  these were telling FFmpeg to use the first audio track alongside the image, images didn’t have an audio track!

Removing the audio command part just resulted in an error about mismatches, so I went down the route of looping through all the directories and using the following command to detect ones with no audio

 ffprobe -i INPUT -show_streams -select_streams a -loglevel error | grep index 

Videos with an audio track would return index=1, where images and screenshots without would return an error, where I could use the following command:

ffmpeg -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=44100 -i ./20190001/clip.mp4  -shortest -c:v copy -c:a aac ./20190001/audio_fix.mp4

I essentially built this out in Laravel because it made the traversal of directories easier, and made sure that if I hadn’t needed to add audio I still copied the file to audio_fix.mp4 to make it easier to build the file later, I then had 325 files all with audio and ready to be scaled to 1920×1080. 

ffmpeg \
  -i ‘./20190001/clip.mp4' \
  -i ‘./20190002/clip.mp4' \
  -i ‘./20190003/clip.mp4' \
  -i ‘./20190005/clip.mp4' \

   …

 -filter_complex "\
 [0:v:0]scale=1920:1080[c1];\
 [1:v:0]scale=1920:1080[c2]; \
 [2:v:0]scale=1920:1080[c3]; \
 [3:v:0]scale=1920:1080[c4], \

 …

[c1] [0:a:0] \
[c2] [1:a:0] \
[c3] [2:a:0] \
[c4] [3:a:0] \

…
concat=n=325:v=1:a=1 [v] [a]" -map "[v]" -map "[a]" ‘./videos/output_2019.mp4' -y

Success!! After what felt like an unbelievable amount of time, and with my Macbook Air sounding like it was going to take off I had a full video containing all of my videos!!

Improvements

The only annoyance was that unlike those videos built in the app, each clip didn’t have a date title so it was hard to tell how far through the year we were or what an important event was. Fortunately each clip was in a directory that we could use to calculate the date in PHP.

// $time_input is a PHP timestamp based on the directory name.

“ffmpeg -i INPUT -vf drawtext=\"fontfile=./Fonts/Raleway-Regular.ttf: text='".date('jS F Y', $time_input)."': fontcolor=white: fontsize=50: box=1: boxcolor=black@0.5: boxborderw=3: x=10: y=(h-text_h-10)\" -codec:a copy OUTPUT”

This uses the Raleway font to draw a black box of 50% opacity in the bottom left corner with white text with the date on, I had to send it to yet another file, but then the final command was easy!

ffmpeg \
  -i ‘./20190001/date_audio_fixed_clip.mp4' \
  -i ‘./20190002/date_audio_fixed_clip.mp4' \
  -i ‘./20190003/date_audio_fixed_clip.mp4' \
  -i ‘./20190005/date_audio_fixed_clip.mp4' \

   …

 -filter_complex "\
 [0:v:0]scale=1920:1080[c1];\
 [1:v:0]scale=1920:1080[c2]; \
 [2:v:0]scale=1920:1080[c3]; \
 [3:v:0]scale=1920:1080[c4], \

 …

[c1] [0:a:0] \
[c2] [1:a:0] \
[c3] [2:a:0] \
[c4] [3:a:0] \

…
concat=n=325:v=1:a=1 [v] [a]" -map "[v]" -map "[a]" ‘./videos/output_2019.mp4' -y

Finally I had my video – I really hope this helps somebody!