Duplicate detection

Questions and bug reports for Beta releases should be posted here.
Forum rules
Help us help you:
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
Post Reply
Zeikzeil
Newbie
Newbie
Posts: 6
Joined: August 16th, 2010, 6:32 am

Duplicate detection

Post by Zeikzeil »

It appears SABnzbd 0.8.0Alpha3 does not handle duplicate nzb files the way it should. I've just tested this by copying a .nzb from my .sabnzbd/nzb directory to my import directory and SABnzbd simply started downloading. The .nzb file in .sabnzd/nzb was overwritten. I've rechecked my configuration and it's set to pause duplicate nzb files.

EDIT: Right after posting it occurred to me to test with a .nzb file that was actually handled/downloaded by 0.8.0Alpha3. I copied one over to my import directory and it paused as it should. That means only old .nzb files are not detected correctly. I suppose this report can be safely disregarded.

User avatar
shypike
Administrator
Administrator
Posts: 21039
Joined: January 18th, 2008, 12:49 pm

Re: Duplicate detection

Post by shypike »

SABnzbd 0.8.0 no longer uses the backup folder to check duplicates.
Instead it relies on the combined md5 ssignature of the combined article-id's
For series it also keeps track of season and episode numbers.
0.8.0 does not use a backup folder used by 0.7.x for checking; that bit isn't listed in the readme file.

Zeikzeil
Newbie
Newbie
Posts: 6
Joined: August 16th, 2010, 6:32 am

Re: Duplicate detection

Post by Zeikzeil »

I suspected as much when I saw "Detect identical NZB files (based on NZB content)" in the config. I didn't remember that being there in 0.7.x. Thanks for clearing things up.

wodix
Newbie
Newbie
Posts: 25
Joined: May 14th, 2009, 1:56 am

Re: Duplicate detection

Post by wodix »

I have at the moment ~40000 nzb files in my backup folder for a daily used duplication checking. Is there any work a round to transfer and save this duplication information to the 0.8 version of sabnzbd ?

I tried allready to reimport and delete this nzb in to the queue, but that was not working, only after a full download i get a working duplucation checking with 0.8.

At the moment i switched my sabnzbd installation back to 0.7.x. I hope to find any solution to save my history of allready downloaded nzb.

User avatar
shypike
Administrator
Administrator
Posts: 21039
Joined: January 18th, 2008, 12:49 pm

Re: Duplicate detection

Post by shypike »

SABnzbd will not import existing duplicate info.
There is one, very convoluted, way to do it.
1. Move the content of the backup to another location
2. Pause SABnzbd
3. Enter all backups into the queue
4. Set post-processing to "Download"
5. Enable Config->Special->empty_postproc
6. Disable Config->Special->safe_postproc
6. Use multi-ops to set all jobs to "Stop"

BTW: I'll look at the option to keep using the backup folder for duplicate checks.

ErikBrown
Release Testers
Release Testers
Posts: 135
Joined: December 20th, 2009, 1:25 am
Location: Home

Re: Duplicate detection

Post by ErikBrown »

BTW: I'll look at the option to keep using the backup folder for duplicate checks.
Hi shypike,

Would it not be a much nicer option to use the database for duplicate detection as planned but provide a utility to import the MD5 signatures of all the existing NZB files in the backup folder into the new duplicate database after upgrading to 0.8?

Erik

User avatar
shypike
Administrator
Administrator
Posts: 21039
Joined: January 18th, 2008, 12:49 pm

Re: Duplicate detection

Post by shypike »

No, because you would end up with 4000 ghost entries in the database, which need special treatment.
And a lot of people don't even use duplicate detection.

wodix
Newbie
Newbie
Posts: 25
Joined: May 14th, 2009, 1:56 am

Re: Duplicate detection

Post by wodix »

Thank you very much for your technical support !
shypike wrote:SABnzbd will not import existing duplicate info.
There is one, very convoluted, way to do it.
1. Move the content of the backup to another location
2. Pause SABnzbd
3. Enter all backups into the queue
4. Set post-processing to "Download"
5. Enable Config->Special->empty_postproc
6. Disable Config->Special->safe_postproc
7. Use multi-ops to set all jobs to "Stop"
I could do step 1-5 with the 0.8beta1 version but i can not find the option "safe_postproc" at the config\Special page or directly in the .ini file. There is no description under http://wiki.sabnzbd.org/configure-special-0-8 for this setting. So please give me a litte hint how step 6 is working. What is the next step after 6, is it first resume the queue ? and do you meen at step 7 with "stop" = delete all queue entrys ? and btw. what is the filename of the new duplication database in 0.8 ?
shypike wrote:BTW: I'll look at the option to keep using the backup folder for duplicate checks.
If this could be possible, i will be very happy about this (duplication checking with my full history is very important in my case !

User avatar
shypike
Administrator
Administrator
Posts: 21039
Joined: January 18th, 2008, 12:49 pm

Re: Duplicate detection

Post by shypike »

Can you wait until Beta2?
That will have the option to keep using the backup folder.

wodix
Newbie
Newbie
Posts: 25
Joined: May 14th, 2009, 1:56 am

Re: Duplicate detection

Post by wodix »

shypike wrote:Can you wait until Beta2?
That will have the option to keep using the backup folder.
Greate Info ! sure i can wait, at the moment i still use version 0.7 "in production ;) " but i look forward to test 0.8 beta2.

thank you again.

Post Reply