RSS pulling duplicates

[Astyanax]

Hi all:

I've been using sabnzbd for a few months now with nzbmatrix as my RSS/API feed. No problems until a week or so ago, when it started pulling duplicates for every TV show download. I have made no setting adjustments; it just pulls down two of everything. Here is my setup:

Version: 0.5.2
OS: Windows XP SP3
Install-type: Windows installer
Firewall Software: None
Are you using IPV6? I don't know, probably not.
Is the issue reproducible? Yes, every time.


Any ideas?

Thanks!

[shypike]

Only when titles match 100%, they will be detected as duplicates.
Often posts differ in details.
In 0.6.0 there will be some improvements in this.

[Astyanax]

Thanks for the info. Unfortunately, these seem to be identical posts within the feed itself. Even the post URLs match perfectly, and EVERY post is being grabbed twice...

Within the feed, I see every listing in there twice. Like so:

    http://url-to-item00001
    http://url-to-item00001    Show name one
    http://url-to-item00002
    http://url-to-item00002    Show name two
    http://url-to-item00003
    http://url-to-item00003    Show name three
    http://url-to-item00004
    http://url-to-item00004    Show name four

First, a URL, then a line break. Then the SAME URL (and ID) and the title of the show, and then a line break. And then the next item.

When I PREVIEW the RSS feed in sabnzbd, the items are listed only once each. But when I download them, it downloads twice. So it means the Preview is catching the duplicates, but the downloader is not...

Thanks!

[shypike]

Please PM the RSS URL to me.

[Astyanax]

The RSS URL has been sent.

Interestingly enough, I think I've figured out what caused it, and uncovered a small bug. I realized that this started happening AFTER I started asking sabnzbd to execute a custom script (batch file) on queue finish. The moment I turned that feature off, I stopped getting duplicate downloads from the RSS feed. For now, I'll have to run my script manually, but I hope that helps!

Thank you!!!

[Astyanax]

Just wanted to ask if you had any thoughts on this? Are you able to duplicate this as a bug when an "On queue finish" script is run?

Thanks!

[shypike]

I haven't had time to look into this.
It is however extremely unlikely that there is a connection between duplicates and an end-of-queue script.
What does your script do?

[Astyanax]

It does two things:

1. Execute "theRenamer" program via command line which renames and moves the files to another drive.

2. execut a .vbs file which deletes empty folders from the destination parent folder.

So basically it's a batch file that runs those two things.

When I select it to execute upon queue completion, I get duplicate downloads of everything. When I tell the drop-down not to run anything, I get singles.

Weird!

[shypike]

One thing you can try is run a dummy end-of-queue script.
If the same behavior occurs, it's likely to be a SAbzndb issue.
If not, it must be something your script does.

[Astyanax]

Thanks for that suggestion. I ran a dummy batch file that does nothing but echo "hello world". Guess what? The duplication is not happening.

My best guess is that my script is moving the file to a new folder too soon, and sabnzbd is thinking the file is not there and attempting to re-download.

So I'm guessing my best bet is to put a delay in the batch file or a delay in the execution somehow. I will experiment with that next.

Thank you!

[shypike]

The best thing to do is to launch a new script from the user script with "start".
Have the new script begin with a delay.
That way, the first script will have returned control to SABNzbd before the second one starts.

It's still weird, because the end-of-queue script is run when all activities within SABnzbd are completed.

[Astyanax]

Did exactly what you said:

1. Created a new script called "postProcess.bat".
2. This script launches my "renameFiles.bat" using the "start" command.
3. renameFiles.bat waits 10 seconds.
4. renameFiles.bat launches "theRenamer.exe" and "deleteEmptyFolders.vbs" using the "call" command.
5. renameFiles.bat then exits.

Unfortunately, I got a duplicate download again. Odd. I'll try upping the delay to 60 seconds and let you know.

Thanks!

[Astyanax]

Nope, the 60-second delay didn't help. Still duplicates. For now, I can run a third-part script execution program that does the same thing outside of sabnzbd.

My best guess at this time is that sabnzbd doesn't like an end-of-queue script that moves the files somewhere else.

Thanks!