Skip on name collision during extract?

Get help with all aspects of SABnzbd
Forum rules
Help us help you:
  • Are you using the latest stable version of SABnzbd? Downloads page.
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
Post Reply
AnonyMouse
Newbie
Newbie
Posts: 27
Joined: December 9th, 2017, 10:45 am

Skip on name collision during extract?

Post by AnonyMouse »

Hello,

I am pretty new still to sabnzbd, but I've been liking it a lot. One thing I seem to have happen to me fairly often is duplicates in my output folder, with a ".1" tagged on to the end. Is sabnzbd doing this, or is this the unpacker? More to the point, I'd rather have a skip/abort on collision. Is that possible to set? If not, I was thinking I could just do an immediate cleanup/removal with a post-processing script, unless someone has a better way to handle that.

Thank you!
User avatar
safihre
Administrator
Administrator
Posts: 5366
Joined: April 30th, 2015, 7:35 am
Contact:

Re: Skip on name collision during extract?

Post by safihre »

Do you mean folders, or files that have the .1?
Did you download files with the same name?
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
AnonyMouse
Newbie
Newbie
Posts: 27
Joined: December 9th, 2017, 10:45 am

Re: Skip on name collision during extract?

Post by AnonyMouse »

safihre wrote: December 9th, 2017, 12:07 pm Do you mean folders, or files that have the .1?
Did you download files with the same name?
Folders. I accidentally download things I downloaded earlier without realizing it. It's not a super big deal to clean up, but it would save me the effort of having to clean it up of I could just prevent it altogether. Something in the process is detecting the folder name collision and auto appending .1 .2 etc (I've never gotten to .3 but presumably it's possible). Rather then prevent the collision by renaming, I'd rather just have it dump the download.
User avatar
safihre
Administrator
Administrator
Posts: 5366
Joined: April 30th, 2015, 7:35 am
Contact:

Re: Skip on name collision during extract?

Post by safihre »

Well we do have Duplicate detection in Config Switches, have you tried that?
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
AnonyMouse
Newbie
Newbie
Posts: 27
Joined: December 9th, 2017, 10:45 am

Re: Skip on name collision during extract?

Post by AnonyMouse »

safihre wrote: December 9th, 2017, 5:30 pm Well we do have Duplicate detection in Config Switches, have you tried that?
Well, I have that turned on as far as I know. It's actually not clear to me why these aren't always caught. My understanding was that the dup detection is primarily based on nzb duplication? I haven't ever really cleared...well hmm...I see my .nzb backup value is empty. So maybe it hasn't been retaining nzb's like I thought. So just so I'm certain, the nzb based duplication is based on the nzb already existing here? Or is there some sort of hash stored elsewhere? Because even just having an nzb wouldn't be enough since I usually don't give names to nzb's when I pull them down, so I wouldn't run into nzb name collisions all that often.

So, checking my settings, I do have Detect Duplicate Downloads and Detect duplicate episodes in series checked. The series one does catch stuff every so often, but flat out dupes seem to get by, since I get those ".1" folders. Clearly _something_ knows that the target extraction folder already exists, presumably the unpacker process when deciding to move the final extraction to the target folder. I guess I'll start with the nzb's being retained or not and go from there.

Thank you
User avatar
safihre
Administrator
Administrator
Posts: 5366
Joined: April 30th, 2015, 7:35 am
Contact:

Re: Skip on name collision during extract?

Post by safihre »

The duplicate works on the hash that's saved in the history database. So jobs with identical filenames but different NZB content wouldn't be caught by it.
However, the series duplicate checked should catch these.

It could also be a problem with Direct Unpack, which I know for some users sometimes generates these extra folders. Unfortunately I was never able to locate the problem.
But in your case you say it's really 2 NZBs that you added with same name, right?
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
AnonyMouse
Newbie
Newbie
Posts: 27
Joined: December 9th, 2017, 10:45 am

Re: Skip on name collision during extract?

Post by AnonyMouse »

safihre wrote: December 10th, 2017, 3:10 am The duplicate works on the hash that's saved in the history database. So jobs with identical filenames but different NZB content wouldn't be caught by it.
However, the series duplicate checked should catch these.
Ok, that fits with what I'm seeing. I believe series duplicate does catch these mostly.
safihre wrote: December 10th, 2017, 3:10 am It could also be a problem with Direct Unpack, which I know for some users sometimes generates these extra folders. Unfortunately I was never able to locate the problem.
But in your case you say it's really 2 NZBs that you added with same name, right?
Well, I'm not sure it's a problem happening in my case. What I think it is, is me not realizing I had already grabbed something, and re-grabbing it. It's identically named, identical content. The other possibility (and this is perhaps the trickier one) is it's a repost, but I would expect the hash would catch that, so there must be something different.. Same failing on my part of not realizing I already grabbed it previously, and re-getting it. Resulting folder would otherwise be identical, so I get the ".1" response. That's the bit I was trying to prevent. I'm willing to take the risk of it being identically named but different content, and instead of getting a appended folder name on collision, I'd rather it just abort the operation and not unpack, and treat it as a duplicate.

It must be happening at the unpack, the more I think about it. It's clearly getting passed the hash check. When nzb's are moved to the archive folder, if there is a collision there, is the existing one overwritten, or is similar name collision behavior employed to prevent losing a file? I will try to pay closer attention to the nzb's when I see a .1 and try to discern characteristics that might be allowing it to be missed by the dupe check.

Thanks!
User avatar
safihre
Administrator
Administrator
Posts: 5366
Joined: April 30th, 2015, 7:35 am
Contact:

Re: Skip on name collision during extract?

Post by safihre »

Could you maybe enable Debug logging in the Status Window, then after it happens again send me the log (click Show logging in the same window) at [email protected]?
Also include the job-name that you found was identical.
You use 2.3.1 right?
If you like our support, check our special newsserver deal or donate at: https://sabnzbd.org/donate
AnonyMouse
Newbie
Newbie
Posts: 27
Joined: December 9th, 2017, 10:45 am

Re: Skip on name collision during extract?

Post by AnonyMouse »

Sure, I'm happy to try to catch it with logging. It's just that it depends on my to notice it happened. I don't always notice it right away.

On the version, yes, it reports as 2.3.1 [f1695ec]
Post Reply