Renaming is Failing on Linux

Report & discuss bugs found in SABnzbd
Forum rules
Help us help you:
  • Are you using the latest stable version of SABnzbd? Downloads page.
  • Tell us what system you run SABnzbd on.
  • Adhere to the forum rules.
  • Do you experience problems during downloading?
    Check your connection in Status and Interface settings window.
    Use Test Server in Config > Servers.
    We will probably ask you to do a test using only basic settings.
  • Do you experience problems during repair or unpacking?
    Enable +Debug logging in the Status and Interface settings window and share the relevant parts of the log here using [ code ] sections.
Post Reply
2devnull
Newbie
Newbie
Posts: 6
Joined: March 19th, 2015, 11:02 am

Renaming is Failing on Linux

Post by 2devnull »

Hi,
Is there a way to fix this?

Ubuntu 14.04 with SABnzbd 0.7.20.

Thank you.

Code: Select all

2015-03-19 04:07:17,956::ERROR::[postproc:408] Error renaming "/folders/complete/_UNPACK_All.In.1080p.MP4-KTRC" to "/folders/complete/All.In.1080p.MP4-KTRC"
2015-03-19 04:07:17,957::INFO::[postproc:409] Traceback: 
Traceback (most recent call last):
  File "/usr/share/sabnzbdplus/sabnzbd/postproc.py", line 406, in process_job
    newfiles = rename_and_collapse_folder(tmp_workdir_complete, workdir_complete, newfiles)
  File "/usr/share/sabnzbdplus/sabnzbd/postproc.py", line 802, in rename_and_collapse_folder
    renamer(oldpath, newpath)
  File "/usr/share/sabnzbdplus/sabnzbd/misc.py", line 1289, in renamer
    os.rename(old, new)
OSError: [Errno 18] Invalid cross-device link


2devnull
Newbie
Newbie
Posts: 6
Joined: March 19th, 2015, 11:02 am

Switch from os.rename to shutil.move to support cross-fs ren

Post by 2devnull »

Perhaps this implementation may fix it?

https://rbcommons.com/s/twitter/r/1157/

My file system is AUFS on XFS.
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: Renaming is Failing on Linux

Post by shypike »

How did you setup your folders?
It seems that SABnzbd needs to move files across symlinks.
That is a rather strange setup.
2devnull
Newbie
Newbie
Posts: 6
Joined: March 19th, 2015, 11:02 am

Re: Renaming is Failing on Linux

Post by 2devnull »

Hi shypike,
I'm not sure I understand the question, isn't it just trying to rename the folder within the same parent folder?

i.e.:
/folders/complete/_UNPACK_All.In.1080p.MP4-KTRC to /folders/complete/All.In.1080p.MP4-KTRC

Here is some more references to similar issues:
https://github.com/NZKoz/ocdtv/commit/e ... d73c959ac9
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: Renaming is Failing on Linux

Post by shypike »

From what I've tested, this should not give problems.
However, it seems that the behavior is different across platforms, or maybe across Python versions.
I'll look at it again.
2devnull
Newbie
Newbie
Posts: 6
Joined: March 19th, 2015, 11:02 am

Re: Renaming is Failing on Linux

Post by 2devnull »

Is there a workaround in the meantime, i.e. I thought I read somewhere there was a switch to disable the _UNPACK_, _ERROR_ etc.?
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: Renaming is Failing on Linux

Post by shypike »

Accept a folder per download instead of moving files from their temp folder to another folder.
The __unpack__ etc renaming isn't the issue.
2devnull
Newbie
Newbie
Posts: 6
Joined: March 19th, 2015, 11:02 am

Re: Renaming is Failing on Linux

Post by 2devnull »

Does that mean set the values for:

"Temporary Download Folder" and "Completed Download Folder"

to be the same?
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: Renaming is Failing on Linux

Post by shypike »

No, not at all.
Normally SABnzbd will create an __unpack__JOB folder in the "completed download folder".
Afterwards it will rename it to JOB.
However there are several options (like in Sorting) to have SABnzbd move the files in JOB to the folder below.
It's the latter situation that seems to cause the "cross-link" error.
2devnull
Newbie
Newbie
Posts: 6
Joined: March 19th, 2015, 11:02 am

Re: Renaming is Failing on Linux

Post by 2devnull »

I lost you at "folder below". Which is that if JOB is under "completed download folder" and JOB is trying to be renamed, why would files need to be moved?
User avatar
shypike
Administrator
Administrator
Posts: 19774
Joined: January 18th, 2008, 12:49 pm

Re: Renaming is Failing on Linux

Post by shypike »

My folder/tv show/bla-season4ep3.mkv ==> My folder/bla-season4ep3.mkv
This is done with the os.rename function, but apparently some circumstances
make it necessary to use shutil.move instead.
It should not be necessary because the file doesn't cross file systems,
but it looks like it can be a problem for os.rename
I'll change that in the next release and then we'll see if it really helps.
User avatar
sander
Release Testers
Release Testers
Posts: 8811
Joined: January 22nd, 2008, 2:22 pm

Linux: OSError: [Errno 18] Invalid cross-device link

Post by sander »

Ah, interesting: between a Windows NTFS and a Linux ext4 partition, I'm able to reproduce the error with os.rename/renames(), and no problem with shutil.move()

Code: Select all

sander@superstreamer:~$ python
Python 2.7.9 (default, Mar  9 2015, 23:17:55) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> old="/media/sander/Windows/ProgramData/blabla"
>>> new="/home/sander/blabla"

>>> import os
>>> os.rename(old, new)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 18] Invalid cross-device link

>>> os.renames(old, new)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/os.py", line 199, in renames
    rename(old, new)
OSError: [Errno 18] Invalid cross-device link
>>> 

>>> import shutil
>>> shutil.move(old, new)
>>> 
Proof that it worked:

Code: Select all

sander@superstreamer:~$ ll  /media/sander/Windows/ProgramData/blabla
ls: cannot access /media/sander/Windows/ProgramData/blabla: No such file or directory
sander@superstreamer:~$ ll /home/sander/blabla 
-rwxrwxrwx 1 sander sander 0 mrt 22 06:33 /home/sander/blabla*

So shutil.move() indeed solves this problem.
Post Reply