Hand downloading over to NAS (completely external download)

[porg]

As parallel downloads this already works

MediathekView allows to define an external download app.

MediathekView > Preferences > Download and Playback > Edit sets > Your Set > Helper apps:

I created one which runs a non-interactive SSH to my NAS where I trigger a wget download, and immediately exit the SSH again, but the download continues on the NAS, like this:

Program:

/usr/bin/ssh

Arguments:

nas cd /srv/dev-disk-by-uuid-LONG-UID/Shared/Videos/ ; mkdir -p "$(dirname "**")" ; curl --continue-at - --output "**" %F 1>/dev/null 2>/dev/null &

The two special patterns ** and %f are expanded per each download invocation:

• ** gets filled with the the relative folder path and filename which you have set in the download dialog, e.g.:./Series/My Documentation Name/My Video Name.mp4

• %f gets filled with the URL for the chosen quality (low, medium, high) which you have chosen in the download dialog, e.g.https://some-content-delivery-network.com/media/a3e8-20c08f58e1c1-3c3a2d2ca7c6_2b.mp4

Explanation of the command line for the helper app

• ssh nas Established a passwordless SSH connection from my end device to my NAS.

• cd I then change into the video base directory on the NAS.
  mkdir -p “$(dirname “**”)” and I create the target directory (if it does not exist yet with the -p flag)

• curl and I start a curl instance

• --continue-at - which continues the file (if previously started), special value - means to determine the download continuation offset from the given --output file.

• --output uses my optional relative sub-directory plus the friendly name of the video that the catalog software provides.

• 1>/dev/null 2>/dev/null I redirect the output of curl to /dev/null, and therefore the non-interactive SSH connection terminates immediately after having triggered the command, whereas curl on the NAS continues to run at its own tempo, until done.

Important note:

• If you try your command first yourself in Terminal, you will need to use single quotes around the program argument which you pass to the non-interactive SSH, so ssh nas 'cd ... &'.

• But in the arguments textfield in MediathekView do not use the single quotes around the SSH program argument! Mediathek does that on your behalf. If you add single quotes there yourself, the command will just fail silently.

Serial downloads on NAS (add to download queue) - Looking for solutions

The downside of the above method is that each new download invocation causes another parallel download (running in its own curl or wget instance) which puts your downloads to a crawl as soon as you reach a certain amount of parallel downloads.

• I would prefer a solution where the download button on the end device running MediaView would trigger the NAS to add another filepath/URL pair to a download queue. And this download queue on the NAS runs in its own tempo, independent from when a download gets added.
  • wget and curl both don’t have a queue management.
  • I asked How to programmatically add URL + filepath pair to HTTP(S) download queue on NAS ? on the forum of OpenMediaVault, the distro that I use on my NAS.
  • Should I find out how, I will share this here too.

Other questions & related inquiry

• Does your built in JDownloader feature also work with a JDownloader running on another host in your LAN such as on a NAS?
• If so I would look whether JDownloader is available as a CLI version to run on a NAS.
• MediathekView NAS Edition (Web App on Docker Image) ?

[vitusson]

@porg sagte in Hand downloading over to NAS (completely external download):

As parallel downloads this already works

Serial downloads on NAS (add to download queue) - Looking for solutions

The downside of the above method is that each new download invocation causes another parallel download (running in its own curl or wget instance) which puts your downloads to a crawl as soon as you reach a certain amount of parallel downloads.

• I would prefer a solution where the download button on the end device running MediaView would trigger the NAS to add another filepath/URL pair to a download queue. And this download queue on the NAS runs in its own tempo, independent from when a download gets added.
  • wget and curl both don’t have a queue management.

Was spricht denn gegen den Einsatz von aria2 im Deamon-Mode auf dem NAS Gerät?
Das kann queue und startet nicht sofort 30 parallele Downloads.
Existieren tut das anscheinend für openmediavault https://github.com/henryouly/openmediavault-aria2 (Disclaimer: schnelles google-Ergebnis, hab selbst kein solches Gerät.)
Und eine Programmgruppe für aria2 gibt es für MV auch schon
https://forum.mediathekview.de/topic/907/downloadgeschwindigkeit-set-wget/5?_=1672208459901

[porg]

@vitusson Danke, Deine Tipps haben mich zum Ziel geführt!

Es waren noch einige Anpassungen nötig, aber nun läuft dies bei mir auf macOS mit dem MediathekView zum Stöbern und Download-Beauftragen und dem OMV NAS als aria2 server, der dann die Downloads in der Schleife vornimmt.

Anpassungen am Client (Mac)

1. MediaThekView > Einstellungen
   • Aufzeichnen und Abspielen > Set bearbeiten > Mein Set > Hilfeprogramme:
   • Programm: Muss absoluter Pfad sein zu aria2-remote.sh
   • Weder ~ (Tilde) noch $HOME werden aufgelöst!
2. aria2-remote.sh — as of commit 9558f62 from 2016-01-26:

• a) Script fails on macOS with: base64: invalid option -- w
  • Bugfix: base64 -w 0 - (--wrap) heisst am Mac --break. Dann wird base64 ausgeführt und in Folge die RPC Anfrage richtig geformt.
• b) Mit filename=$(basename "$2") kommen alle Downloads flach ins Aria Download Root Directory ( --dir). Dies gefiel mir nicht.
  • Änderte es auf filepath="$2" (ohne die basename operation) damit man von MediathekView aus im Zielfpad relative Angaben wie ./serien/tageschau/ plus dem Dateinamen Name des Videos.mp4 übergeben kann, die dann relativ zum Aria Download Root Directory fungieren, z.B. --dir=/srv/media/video, d.h. es ergibt sich dann: /srv/media/video/serien/tageschau/Name des Videos.mp4.
  • Der Download Dialog der MediathekView merkt sich die letzten Pfadangaben im Zielpfad-Dropdown Menü.
    • Somit kann man durch Verwenden des Dropdown-Menüs super bequem und konsistent in bereits genutzte Ordnerstrukturen ablegen.
• Download der verbesserten Version: aria2-remote.sh

Setup am Server (OpenMediaVault NAS)

aria2c --enable-rpc --rpc-listen-all --rpc-secret mySecret --dir=/srv/dev-disk-by-uuid-LONG-UID/Shared/Videos

• Wichtig: Ohne abschließendem Schrägstrich, damit der relative übergebene Pfad schon dazu gesetzt wird und kein Doppelschrägstrich rauskommt.

aria2-remote.sh.txt

[porg]

Schöner formatiertes Log ohne Secret:
aria2-remote.sh.txt