SOLVED: Write error with Windows Storage Pool network shares

Get help with all aspects of SABnzbd
Forum rules
Help us help you:
  • Are you using the latest stable version of SABnzbd? Downloads page.
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
Post Reply
omeganot
Newbie
Newbie
Posts: 1
Joined: January 19th, 2017, 1:57 pm

SOLVED: Write error with Windows Storage Pool network shares

Post by omeganot »

This kept me busy for 6 hours last night, finally coming to something that seems resolved at 3am. The learning was painful, so I thought I'd share.

Some backround: Like many other posts out there I started getting bit by the "Unpacking failed, write error or disk is full?" error. Some would finish, but most wouldn't, and recently I couldn't get a single download to finish. Unlike most others, I'm not running sabnzbd on a NAS, or Linux, or anything of the like. I've got a Windows 2012 R2 server acting as my storage and VM host. Sabnzbd, Sickbeard, Couchpotato and Plex are all running on the VM, which gives me both a little separation of duties (I really should move Plex to its own VM) as well as a way to quickly restore should I run into issues or need to move to a new server. On the server itself, I have one large storage pool, originally set up with 3 4TB hard drives. There are numerous virtual disks set up within the pool to store different media (one for TV, one for movies, one for photos, etc.). In the fall I was running low on storage, so I added a fourth 4TB hard drive. All should be well, yes? Not so much.

Within the VM I have mapped drives to the network shares for the various downloads. There's an incoming drive (for incomplete and complete files, other such things), a TV drive, and a movie drive. Sabnzbd wasn't the problem, it was configured as it has been for years. No updates seem to have altered that either. I even looked through the GitHub source to find out how the unpack and post processing is called to see where things are occurring to debug piece by piece. I had sabnzbd configured to download to the incoming drive, then it was unpacking to the TV drive but I changed that to unpack in the completed folder (turns out I didn't have things configured the way I should with the Sickbeard script). I also removed the _UNPACK folder prefix for fear that somewhere somehow I was getting file names that were too long. None of these things corrected the issue.

I checked permissions, and even granted more access to the account the VM was using to the incoming and TV folders. No change.

Unrar wasn't the problem. Running the same command I got the same error (unpack from incoming to TV, originally). Sabnzbd passed that through nicely on +Debug logs. I then tried changing the location of the unpack to the incoming drive. Same error. I then did an unpack to a different drive. No error. Hmm...

Looked into my storage pool again. Over 3.4TB of space free. All on one physical drive. Hmm... Removed some old TV shows I wasn't planning on watching again from the virtual disk that contained them, retried a download in sabnzbd. Retry, not re-download, to use the same files and settings. It worked. Hmm...

[The Answer]
Turns out, when you create virtual disks within Windows storage pools the virtual disk is by default configured to stripe the data across as many drives as there are. I had three, so it striped across three for more throughput. Good basic logic there. However, when I added a new drive it couldn't use it by default because it wanted to stripe equally across three drives, and there werern't 3 drives available with enough space to do so. Therefore, it claimed the drive was full (which was also confusing, because I configured the drive to expand virtually to 20TB and it only had 6.8TB used) any time it tried to make a write. To have used the same striping setup with that virtual disk, I would have needed to add three physical hard disks when I did my upgrade. Not exactly how I had envisioned my upgrade path.

For an explanation of how striping works in Windows storage pools, this was a great article to read - http://windowsitpro.com/windows/navigat ... -and-pools

To fix, there are two options that I can see. First, if keeping on Windows 2012 R2 (or Windows 8 I would imagine) you can create a new virtual disk with one "column" for striping. Original idea for that came from https://community.spiceworks.com/topic/ ... disk-issue in the best answer. This removes the throughput benefits of striping, but it will scale with every single physical hard disk that gets added to the pool. Then, move all files from the trouble virtual disk to the new virtual disk, and delete the old virtual disk. I'm currently in the process of doing this right now.

Second, if you are running a Windows Server 2016 instance, or using Windows 10 for your storage pool, there's a fix in a PowerShell command "Optimize-StoragePool" which will rebalance data across disks. Here's a link for more info - https://stebet.net/microsoft-finally-ad ... ge-spaces/ Note though, that the virtual disk will still have the same number of columns for striping (it can't be changed after the fact) and will likely run into this issue again, but it can be rebalanced again. I don't know the effectiveness of this long term, as I don't have my storage pool set up on those versions of Windows.

I hope this post helps people who are set up in a similar way to me. It was a long and frustrating path to the answer, so I hope I can save you some time.

User avatar
safihre
Administrator
Administrator
Posts: 4065
Joined: April 30th, 2015, 7:35 am
Location: Switzerland
Contact:

Re: SOLVED: Write error with Windows Storage Pool network sh

Post by safihre »

Thanks for sharing :)
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate

Post Reply