v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Get help with all aspects of SABnzbd
Forum rules
Help us help you:
  • Are you using the latest stable version of SABnzbd? Downloads page.
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
SpaceBass
Newbie
Newbie
Posts: 12
Joined: April 22nd, 2012, 10:05 am

v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Post by SpaceBass »

Hey folks

System:
  • FreeBSD 13.1
    Xeon 2123IT
    32gb ram
    SABnzbd v 3.5.3
Symptoms
Every few hours my queue totally freezes. The web UI is responsive and there is a number in the download speed indicator, but it never changes and downloading files never advance. If I try and restart from the web UI, I see the restart attempt in the log, but it never restarts.

In my unpacking queue there's always a file in the middle of an unpack.

So far the only thing I've managed to catch by watching the log files on the server is:

Code: Select all

DirectUnpacker still alive after download complete
It looks like there are some known bugs for this. Does anyone have any tips or fixes?
Right now I have a cron job rebooting the system every 12 hours which isn't at all sustainable or desirable
Last edited by SpaceBass on May 16th, 2022, 9:16 pm, edited 1 time in total.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Post by sander »

It looks like there are some known bugs for this.
Which?
SpaceBass
Newbie
Newbie
Posts: 12
Joined: April 22nd, 2012, 10:05 am

Re: v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Post by SpaceBass »

Ok - upon further monitoring, the last logs I see before a total lockup are the start of the unrar process.
I've tried executing the unrar command manually and it usually fails with a "no file to extract" error.
That suggests to me that unrar isn't failing gracefully or SAB is expecting a different result - is is possible I just need a different flag or something? It does seem that if I kill -9 the stuck unrar process that SAB will resume (although it is still unresponsive to kill commands or a service restart).
SpaceBass
Newbie
Newbie
Posts: 12
Joined: April 22nd, 2012, 10:05 am

Re: v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Post by SpaceBass »

sander wrote: May 16th, 2022, 4:52 pm
It looks like there are some known bugs for this.
Which?
perhaps I have used the wrong wording - may it is more accurate to say that I found similar threads and GitHub issues with a similar theme.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Post by sander »

in sabnzbd.log, what unrar version is reported? Search for a line like this:

Code: Select all

2022-05-15 18:28:27,216::INFO::[sabnzbdplus:464] UNRAR binary version 5.90
If you turn off "direct unpack" and try again, what happens?
User avatar
safihre
Administrator
Administrator
Posts: 5338
Joined: April 30th, 2015, 7:35 am
Contact:

Re: v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Post by safihre »

Can you try 3.6.0RC1?
It will show more logging that could indicate tge problem.
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
SpaceBass
Newbie
Newbie
Posts: 12
Joined: April 22nd, 2012, 10:05 am

Re: v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Post by SpaceBass »

sander wrote: May 16th, 2022, 9:29 pm in sabnzbd.log, what unrar version is reported? Search for a line like this:

Code: Select all

2022-05-15 18:28:27,216::INFO::[sabnzbdplus:464] UNRAR binary version 5.90
If you turn off "direct unpack" and try again, what happens?
the unrar version is 6.11
The log doesn't show the version but it does show the path

Code: Select all

UNRAR binary... found (/usr/local/bin/unrar)
and when I execute that binary it reports version 6.11
Last edited by SpaceBass on May 17th, 2022, 4:05 pm, edited 1 time in total.
SpaceBass
Newbie
Newbie
Posts: 12
Joined: April 22nd, 2012, 10:05 am

Re: v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Post by SpaceBass »

safihre wrote: May 17th, 2022, 1:28 am Can you try 3.6.0RC1?
It will show more logging that could indicate tge problem.
I'll investigate ... guessing there's not a FreeBSD package for it. I'll see if I can compile it from source
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Post by sander »

SpaceBass wrote: May 17th, 2022, 4:01 pm
I'll investigate ... guessing there's not a FreeBSD package for it. I'll see if I can compile it from source
Compile? SABnzbd is written in python.
SpaceBass
Newbie
Newbie
Posts: 12
Joined: April 22nd, 2012, 10:05 am

Re: v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Post by SpaceBass »

sander wrote: May 17th, 2022, 4:25 pm
SpaceBass wrote: May 17th, 2022, 4:01 pm
I'll investigate ... guessing there's not a FreeBSD package for it. I'll see if I can compile it from source
Compile? SABnzbd is written in python.
I overestimated how complicated it was. I was able to get 3.6RC1 up and running on FreeBSD without any major hiccups. I'll let it run for a few hours and keep this thread posted on results.
SpaceBass
Newbie
Newbie
Posts: 12
Joined: April 22nd, 2012, 10:05 am

Re: v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Post by SpaceBass »

I wanted to update this thread... I'm still having the freezing issue.

I've been using the various RCs from 1 and 2 and am currently on RC3.

SAB still locks up for me every few hours. The logs don't show anything meaningful.

What I do is are some unrar processes running which do not correspond to what the queue says is being DirectUnpacked.

Still looking for any tips or ideas or suggestions
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Post by sander »

So ... do you think it has to do with unrar?

If so:
* turn off direct unpack, and try a few downloads.
* if that still stalls, do a few donloads without unpack, with the drop down Post Processing :Download. And do manual unrar afterwards ... does that succeed?
* could you have disk problems? To what kind of disk are you unrarring?
SpaceBass
Newbie
Newbie
Posts: 12
Joined: April 22nd, 2012, 10:05 am

Re: v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Post by SpaceBass »

Thanks for the troubleshooting tips Sander - I disabled DirecUnpack and have had 48 hours of uptime. Seems like that's the fix for me

My disk(s) is a ZFS pool on 7200 RPM drives. The files then get moved to an EncFS mount on top of that same ZFS pool.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Re: v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Post by sander »

Nice. FWIW ... I'm not sure DirectUnpack itself is the root cause ... I think your disks play a role, or the combination of your disks , unrar and possbily DirectUnpack.

Long, long shot: directunpack takes longer (whole download duration), so maybe in the meantime your filesystem has done something else.
User avatar
safihre
Administrator
Administrator
Posts: 5338
Joined: April 30th, 2015, 7:35 am
Contact:

Re: v 3.5.3 queue freezes entirely every few hours; DirectUnpacker still alive after download complete

Post by safihre »

Could you try this:
Enable Debug logging in the Status window.
Enable Direct Unpack.
Wait for it to get stuck.
Kill unrar process(es) through command line.
Download the log and share it here or email it to me at [email protected]

Maybe we can find why it got stuck.
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
Post Reply