0.7.12 - stops on corrupted files, doesn't advance to par2

[newser]

This has happened too many times already lately, which didn't use to happen on previous versions (< 0.7.10 IIRC).

On some NZBs, there are some files which are corrupt. sabnzbd downloads them until there is about 0.38 MB or 0.74 MB left, then stops completely ... it never advances to download the PAR2 files. I attached a screenshot and also my ini file (without the [servers] section).

https://www.dropbox.com/s/bzk0c9n4m57q0 ... 04-001.png

The yellow ones are the PAR2 files. If I click on the red X next to the file that has hung, then sabnzbd discards the file entirely rather than keeping whatever it downloaded and then there isn't sufficient par2 information to repair it.

Also, looking in the temp folder, I can see the downloaded files, and also the __ADMIN__ folder which holds all the SABnzbd_article_* and SABnzbd_nzf_* chunks which total the amount of chunks downloaded ... but they never get written to a file (as corrupt). They just sit there.

The size of the __ADMIN__ folder is many times several hundreds of MB because I allowed sabnzbd to use up to 1 GB of RAM for caching as my internet connection can go up to 40 MB/s downlink (yes, those are megabytes, not megabits) which sometimes is faster than the HDD write speed. I've seen sabnzbd cache up to 500 MB on occasions.

I have tried the same NZB with Newsleecher and it downloaded all files as much as it could, saved the corrupt as files with slightly less size than 52 MB and continued to PAR2, then repaired it just fine.

Below is my ini file (without [servers]). I can also provide the nzb file in a PM if you need it for testing.

Code: Select all

__version__ = 19
[misc]
queue_complete = ""
https_port = 9090
folder_rename = 1
allow_64bit_tools = 1
password_file = ""
rss_rate = 60
https_key = server.key
cleanup_list = ,
cache_limit = 1G
ignore_unrar_dates = 0
par_option = ""
web_color = white
replace_spaces = 0
web_color2 = ""
dirscan_speed = 5
enable_filejoin = 1
create_group_folders = 0
api_key = fdb0aa9f9a8073df74f0209641507044
never_repair = 0
no_penalties = 0
bandwidth_limit = 0
dirscan_dir = ""
disable_api_key = 0
email_to = ,
rss_filenames = 0
password = ""
permissions = ""
auto_disconnect = 1
enable_date_sorting = 0
replace_dots = 0
movie_sort_extra = -cd%1
ionice = ""
enable_unzip = 1
download_free = 500M
port = 8080
email_dir = ""
pre_script = None
email_full = 0
log_dir = logs
admin_dir = admin
nzb_backup_dir = ""
tv_sort_countries = 1
date_categories = tv,
ssl_type = v23
sfv_check = 0
date_sort_string = ""
schedlines = ,
host = localhost
ignore_samples = 2
enable_tsjoin = 1
auto_browser = 1
pause_on_post_processing = 1
enable_unrar = 1
email_rss = 0
email_account = ""
email_server = ""
allow_streaming = 0
config_lock = 0
api_warnings = 1
tv_sort_string = ""
username = ""
unpack_check = 1
download_dir = v:\down\nzb\temp
size_limit = ""
enable_par_cleanup = 1
safe_postproc = 1
cache_dir = cache
complete_dir = v:\down\nzb
replace_illegal = 1
par2_multicore = 1
warned_old_queue = 0
language = en
check_new_rel = 1
enable_https = 0
no_dupes = 0
top_only = 0
quick_check = 1
movie_sort_string = ""
email_from = ""
movie_extra_folder = 0
email_endjob = 0
tv_categories = ,
pause_on_pwrar = 2
login_realm = SABnzbd
send_group = 0
refresh_rate = 0
movie_categories = movies,
script_dir = ""
email_pwd = ""
ignore_wrong_unrar = 0
nice = ""
enable_movie_sorting = 0
https_cert = server.cert
web_dir2 = ""
fail_on_crc = 1
nzb_key = 772f13020d4d37196e4926aaac98fe9f
auto_sort = 0
ampm = 1
web_dir = smpl
queue_complete_pers = 0
enable_tv_sorting = 0
folder_max_length = 128
win_menu = 1
movie_rename_limit = 100M
pre_check = 0
osx_speed = 1
quota_period = m
quota_day = ""
use_pickle = 0
quota_size = ""
osx_menu = 1
wait_ext_drive = 5
req_completion_rate = 100.2
fsys_type = 0
allow_incomplete_nzb = 0
uniconfig = 1
randomize_server_ip = 0
quota_resume = 0
no_ipv6 = 0
nomedia_marker = ""
max_art_opt = 0
max_art_tries = 3
start_paused = 0
https_chain = ""
history_limit = 50
show_sysload = 2
keep_awake = 1
rss_odd_titles = nzbindex.nl/, nzbindex.com/, nzbclub.com/
overwrite_files = 0
prio_sort_list = ,
news_items = 1
empty_postproc = 0
local_range = ""
ipv6_servers = 1
web_watchdog = 0
warn_empty_nzb = 1
wait_for_dfolder = 0
fail_hopeless = 1
[logging]
log_new = 0
max_log_size = 5242880
log_level = 0
log_backups = 5
enable_cherrypy_logging = 0
[newzbin]
username = ""
url = ""
bookmark_rate = 60
bookmarks = 0
password = ""
unbookmark = 1
[nzbmatrix]
username = ""
apikey = ""
del_bookmark = 1
[growl]
growl_enable = 1
ntfosd_enable = 0
growl_password = ""
growl_server = ""
notify_classes = startup, download, pp, complete, other
ncenter_enable = 0
[categories]
[[*]]
priority = 0
pp = 3
name = *
script = None
newzbin = ""
dir = ""
[nzbxxx]
username = ""
apikey = ""

[shypike]

Multiple servers?

[newser]

Multiple servers indeed. Do note that I wasn't experiencing this issue with older releases, and I had encountered corrupted files back then too - it was reaching the par2 files and then repairing just fine.

Let me know if you need more info.

[shypike]

The usual cause of this phenomenon is that one of your servers times out
when it doesn't have a certain article.
Check your retries (Config->Switches) and make sure that "Only for optional" below it isn't on.
Also don't overdo it with timeouts in Config->Servers. E.g. 120 seconds and 3 retries means 6 minutes of waiting per such missing article.
Astraweb involved?

[newser]

Indeed Astraweb is the server. Actually, due to some limitations my ISP imposes, I am tunnelling several parallel connections to the same astraweb server (I have a block account with them and they allow multiple source IPs). In sabnzbd they appear as different servers.

Regarding the 6 minutes timeout: I left the thing running for over 6h and it's still stuck at the same point.

Timeout for all servers is 120 seconds. Max retries is 3. "Only for optional servers" is NOT checked (as seen in the ini above).

[newser]

p.s. Note that Newsleecher had no problem with the exact same nzb, same server config (multiple servers as tunnels to Astraweb), and same corrupt files. It doesn't seem like a server issue.

[shypike]

A server can still misbehave, while SABnzbd may not be able to handle such behaviour very well.
Can you email an example NZB to [email protected] ?
Please add the URL of this post and which servers you are using.

BTW: you should be able to see which server is tried last, by looking at Status->Connections.

[newser]

Is any info emailed to [email protected] published on the web or can I assume it is kept private?

[shypike]

Of course it is kept confidential.

[wimpie1]

I have the same problem, went back to 0.7.11 and the problem went away.

To me it seems that is a par problem. The par process was still running after 12+ hours (normally takes 30 to 60 minutes), and consuming 10 to 20 % of the CPU. I tried classic par and multipar, both the same problem.

I had no problems with 0.7.12 when the post doesn't need repair.

Don't think it has anything to do with a spicific nzb, I have 1 server and 5 blokaccounts. SABnzb kept downloading other nzb's, but never fetched the needed extra par blocks. In effect groing the history list while it hangs on the first "need repair" post. Windows 7 computer.


Wim

[shypike]

I have the same problem, went back to 0.7.11 and the problem went away.

There are no changes in 0.7.12 (relative to 0.7.11) that deal with par2 files.
I'm not saying there isn't a problem, but such a problem would be in both versions.

[wimpie1]

I have the same problem, went back to 0.7.11 and the problem went away.

There are no changes in 0.7.12 (relative to 0.7.11) that deal with par2 files.
I'm not saying there isn't a problem, but such a problem would be in both versions.

I'll see what I can do. Almost all files that I download are 20+ GB and my connection is only 4 Mbit. I will try to get something reproduceable (with a smaller file).

[shypike]

To me it seems that is a par problem. The par process was still running after 12+ hours (normally takes 30 to 60 minutes), and consuming 10 to 20 % of the CPU. I tried classic par and multipar, both the same problem.

If par2 hangs or takes excessive time, there nothing we can do about it.
Also the par2 tool in 0.7.11 and 12 is the same.

[Begoodman]

Same proble here. OSX 10.8.4. Multiple servers. Sab 7.11 worked fine. After installing 7.12, par2 slows and eventually hangs whole system, reverting to 7.11 didn't help. Par2 parameters (-t- and -t0) throw errors and its not possible to turn off par2 multicore at all. My SWAG is that a change in config file, 7.11 vs. 7.12, causes the whole conundrum.

[shypike]

reverting to 7.11 didn't help.[./quote]
Because 0.7.12 doesn't cause the problem.

Par2 parameters (-t- and -t0) throw errors

These parameters are not supported by the par2 programs included with SABnzbd.
Did you have another version installed?

and its not possible to turn off par2 multicore at all.

That was never an option for OSX, only for Windows.