Methods for download request > ready to play as fast as possible

[LionelHutz]

I have a 2TB NVME drive and a 16TB HDD on a Proxmox machine and the Arr suite with Usenet to download. I want the least amount of time between downloading a TV series / movie and getting it playable on Plex and I don't want users streams to lag while transferring data over.

So I'm doing all the downloading and unpacking on the NVME so that all the writes and repairs are fast and it doesn't have to transfer over to the HDD immediately. Then at 4am I have a cron script that transfers all the contents from the NVME to the HDD. This way Plex can see it as soon as it's downloaded because it's watching both the NVME and HDD folders.

The problem is that Sonarr and Radarr don't see the TV series / movie after the move because I'm moving the content to a directory that isn't the original download location.

I've read the TrashGuides and atomic moves and I think in this case it doesn't work because of my requirements.

My new plan is to make my NVME and HDD XFS drives that are viewed as one mount point with mergerfs, then download the content to the NVME and do the same cron job to the HDD but this time it can be an atomic move because it's seen as one mount.

So my question is: does anyone have a better solution or is this the best way to do it? And have I gotten any of this wrong?

[sander]

> Then at 4am I have a cron script that transfers all the contents from the NVME to the HDD. This way Plex can see it as soon as it's downloaded because it's watching both the NVME and HDD folders.

So content becomes available after 04:00?
If so: fast downloading, but then wait until 04:00?

Why not this KISS solution:

Incomplete on the NVME
Complete on the HDD

Then content is immediatly available to *arr and Plex. No waiting until 04:00.

[LionelHutz]

So content becomes available after 04:00?
If so: fast downloading, but then wait until 04:00?

No - everything is immediately available because Plex is watching both the NVME and HDD. It's just being moved to the HDD at 04:00 so that it doesn't affect anybody at the same time that users are watching on Plex

Incomplete on the NVME
Complete on the HDD

This is my problem - transferring from NVME to HDD takes 5 minutes to complete a 15gb file (the HDD isn't a slouch either - a Seagate Exos X16). A TV show of 100gb would take 30 minutes. Which is why I've set up the cron job to transfer at 04:00, essentially delaying that part of the process until later. This way the time it takes for a user to request in Overseerr to being ready in Plex is 5 minutes (the time it takes to download, repair and unpack), and there's no potential lag for any users while it transfers because nobody is watching at 04:00

[sander]

> This is my problem - transferring from NVME to HDD takes 5 minutes to complete a 15gb file

That's 50 MB/s. That's too slow; at least I would expect ... 150 - 200 MB/s. Maybe anlayze & fix that first?