- entry 2021/01/31

How to download TS chunked videos

These days, however, if you look to the video, you'll see it is a bunch of little pieces each with a ts extension. What the heck is this? How can it be downloaded?

What it is, is exactly what it looks like. Chunked video. I think the point is that each chunk can be adapted to your available bitrate (like how Netflix gets really blocky in poor reception, which is often better than pausing to buffering...buffering...buffering...).

The obligatory warning

Copyright is complicated, so one can assume that films made and released prior to 1927 are out of copyright in the United States. It used to be 75 years. The CTEA (in 1998) added twenty years, but notably did not re-copyright movies whose copyrights had expired, so some movies made prior to 1946 may be out of copyright. This covers the "Goldern Era of Hollywood". But, note, that while a film's copyright might have lapsed, it isn't public domain if it is based upon a book or play what is still in copyright. Then there's the dubious question of automatic renewal of copyright. And, in the US at least, some films might never have had copyright if they didn't state it or didn't state it correctly (the US was really late in implementing automatic copyright in 1978). Yeah, it's complicated!

PS: Note that colourised movies date from the release of the colourised version, not the original, so most are not out of copyright.
PPS: Don't watch colourised versions of black and white movies. They're generally awful.

Google for public domain movies. You'll find such treats as Night Of The Living Dead (1968, the movie was retitled and the new title omitted the copright notice, so it immediately became public domain!), Little Shop Of Horrors (the original, 1960), Charade (1963, misworded copyright notice, immediately became public domain!), House on Haunted Hill (1959), Carnival of Souls (1962, missing copyright notice), Plan 9 from Outer Space (1959, the very definition of "so bloody crap it's almost good"), various Shirley Temple films if you can stomach them, Santa Claus Conquers the Martians (1968, and with a title like that...), A Bucket of Blood (1959, costing $50K and shot in five days, the ultimate in low budget filmmaking), and of course It's a Wonderful Life (1946).

And, of course, I am not responsible for what you may or may not choose to download. It's up to you to have the fun of trying to work out copyright eligibility in your country, and accept consequences of downloading.

Prerequisites

• Firefox with UBlock Origin installed.
  Note - UBlock Origin is nothing to do with UBlock, don't mix them up!
• FFMPEG with command line support.
• Plenty of free disc space!
  Expect something in the order of 700MB to 1GB for an average movie.

This tutorial will concentrate on Android in order to demonstrate that you can do it quickly and easily on the go. The same principles will apply to desktop versions (Windows, Linux...).

There are several incarnations of FFMPEG for Android. This is the one I recommend. FFMPEG is free software, so you shouldn't be paying for the app, and you'll need the ability to customise the command line that you give to the application.

Finding a video

Once you have found a video of your liking, and made sure that it is actually playing and is actually the video that you think it should be...

Tap the three dots on the right of the URL bar to open the menu, and then tap on the entry for UBlock Origin.
In the tab that appears, tap the third icon along, it's a diagram a little bit like a window.

Getting UBlock Origin ready.

Ignore the tab that opens. Switch back to the tab with your video and reload it. Then start playing the video again.
When the video begins playing, pause it.

Getting the correct link

The list of requests.

What that window is showing is a list of everything that was requested in fetching the page. Things in red were blocked, and things in blue are canonical names (that means, a name that points to something else - this isn't always a weird thing, on this site, heyrick.co.uk points to heyrick.eu).

Reading the list backwards, you will see a group of entries listed in the sixth column (before the URL) marked as xhr. This means it is an XMLHttpRequest - and these are your video chunks.

There is, however, a little trick to how these work. Your browser cannot just begin playing, it needs to know what it has to play, and this is determined by a special file sent before the video.

Look down the list for the first xhr link, and when you see it, tap the second column for that entry. A dialogue should pop up giving further information on that request.

Request information.

What you are looking for here is a file that ends with .m3u8, as shown in this screenshot.

If you find that, copy the entire URL (all that gibberish).

Now, quickly check the second-to-last xhr entry. Some sites like to send a bogus m3u8 file first, with the real one following. I suspect this is to defeat extensions that automatically scan for media to grab. If the second one is also an m3u8 file, copy that.

Prepare your device

How this is done depends upon your version of Android and whatever tweaks the manufacturer might have made. Generally, drag down to open the notification panel and look for a little cog wheel symbol to get into Settings.
When in Settings, search for "timeout". You're looking for "Screen timeout". If that doesn't work, try searching for "sleep" in case it is labelled Screen -> Sleep.

When you find it, remember the setting (to restore later on), and then set it to Never.

Setting up FFMPEG

When you first open the app, it will look like this:

FFMPEG pokey-tappey GUI.

The GUI is useful for setting up local conversions quickly, but it isn't what we want today. So tap on the >_ icon indicated.

The screen will change to a command line. Erase whatever is there, you don't want it.

Replace it with the following:

fmpeg -y -i "" -c copy "/storage/emulated/0/videoname.mp4"

In between the first set of quotes, paste that long link that you copied from UBlock's logger, so things should look like this:

Take control with the FFMPEG command line.

When you are happy with that, tap the green arrow at the bottom right to begin.

Downloading

FFMPEG in action.

FFMPEG will get on with the task of retrieving the video. Every so often it will briefly pause while opening a URL. This is normal, each chunk is a different file that needs to be fetched individually.

IMPORTANT: FFMPEG is extremely sensitive to disturbances. DO NOT switch to another task. DO NOT let the device put itself to sleep. DO NOT use it somewhere where the signal can drop out. Plug your device in to its charger (recommended), and then leave it to get on with things. How long it will take depends upon the server and your internet speed.

When it is done, it will say so. But, then, it generally reports this even if it failed, so you should check that the values make sense. Is the indicated time what you expected it to be? In my case, the time is a little short, it lost connection at around 97% or so. I'm happy with that, as it will just be the end of the credits that got clipped off. Better lose that than waste time downloading it all again.

FFMPEG finished.

Finishing up

Don't forget to set your sleep timeout back to its previous setting (or about 2 minutes if you don't remember what it was).

Your comments:

Please note that while I check this page every so often, I am not able to control what users write; therefore I disclaim all liability for unpleasant and/or infringing and/or defamatory material. Undesired content will be removed as soon as it is noticed. By leaving a comment, you agree not to post material that is illegal or in bad taste, and you should be aware that the time and your IP address are both recorded, should it be necessary to find out who you are. Oh, and don't bother trying to inline HTML. I'm not that stupid! ☺ ADDING COMMENTS DOES NOT WORK IF READING TRANSLATED VERSIONS.
 
You can now follow comment additions with the comment RSS feed. This is distinct from the b.log RSS feed, so you can subscribe to one or both as you wish.

Zerosquare, 1st February 2021, 00:46

While your guide is useful for people who only have access to mobile devices, I think it's easier to download those videos from a PC, then transfer them to a your mobile device:  - you get access to more convenient tools, such as YoutubeDL (despite its name, it doesn't only support YouTube)  - otherwise, you don't even need to install any plugin: you can use the browser's built-in Network inspector  - you can run FFMPEG in the background without issue  - if you want to reencode the video, it's likely faster (if your PC is reasonably recent) and it won't drain your battery

Add a comment (v0.11) [help?] . . . try the comment feed!
		Your name
		Your email	(optional)
		Validation	Are you real? Please type 13100 backwards.
		Your comment