Please - remove accents ...

[Renfield]

Hi, I'm a french gy, running 0.4.7 on my QNap NAS (Ubuntu, I think)

When I'm downloading files which contains accents, unrar process fail.

please, remove accents when retrieving files...

é=>e
è=>e
ê=>e
î=>i
ô=>o
â=>a
à=>a

my FTP server don't like them too so I must go and renome files manually in order to handle them remotely.

[shypike]

He, we worked our butts off to implement proper accented character support,
to stop our French (and other non-English) users from complaining.
And now you ask us to remove it?
Cannot you get a decent ftp server instead  ?
An alternative is to write a script to convert these characters.

[Renfield]

I agree that I could write a script.

But the jobs which contains some accents are _FAILED_, so I think you could do something.

files are unrared but .rar files are still there (DELETE option has been specifies in sabnzbd)

my FTP server does not like accents, but it is not your problem. I will write a script to rename these files on the fly.

[shypike]

Failure of downloads is usually caused by something else.
There are two incompatible versions of par2 at large.
The old one uses Latin-1 for accented characters, the new one uses Unicode.
If the poster and the downloader use different versions there is a
possibility that a par2 repair will not work. We cannot solve this problem.
Right now we run the new version for jobs that have no accented char filenames.
Otherwise we run the old one.
This is for OSX and Windows.

For Unix the situation is more complicated, because encoding on Unix file systems
is very messy.
We use the par2 that is installed on the platform (by the user!), so here we have
no influence on how par2 interprets accented characters.
If it fails often, you have have the wrong par2 installed.

Personally I use the Perl script convmv to normalize filenames.
I run it (with a wrapper script for the parameters) as an end-of-job script.
(http://www.j3e.de/linux/convmv/).
It comes as a package for most Linux distros.

[Renfield]

I wrote a script, which I run manually

_FAILED state does not rely on par2.

I got, in log:

2009-03-26 03:50:14,145::WARNING::[newsunpack] Missing expected file: Die Hard 1 - Piège De Cristal.avi => unrar error?

instead of "Piège"

[shypike]

Can you email the NZB file to [email protected] ?

[doctor madness]

Hi,

I'm a french user (oh no, another frog eater !)
Please don't drop the accents !!! It's a very bad idea from Renfield !

My problem with accented character is simple and easy to reproduce.
When a RAR file contains file(s) with accents in name, i got the following error :
At least one file failed to be unpacked, skipping

So, I think so the problem in SABnzbd is to compare filename is RAR file (using rar l RAR_FILE ?) and the uncompressed file(s) failed.
I've a nzb that could be use to reproduce the problem. I updated my SABnzbd today to 0.4.8, but nothing changed.
May I send the nzb to you ?

[shypike]

What SABnzbd does is this:
- Create a list of files that will be unpacked
- Run the unpack tool (unrar or unzip)
- Catch the console output of the tool
- Check if all files actually have been created

Of course this will fail when the unpack tool and SABnzbd have different ideas
about character set translation.
So far I have never seen problems like this on Linux, because Linux filesystems
are agnostic and just write whatever filename you have, without translating any character.

I'll check your NZB file tonight.

It looks like your system may actually do character translation.
I have done this in past too, but found that it was incompatible with Samba and Windows clients.
Given that this is a can of worms, maybe the actual cheking of file existince
is too paranoid and we would better be checking for error messages from the unpack tool.

[shypike]

The thing downloads fine on my system, however I have "neutral" language settings.
Meaning that on my system every filename is as it is created by unrar.
RAR archives contain Unicode filenames.
The unrar program honours the LC_CTYPE setting and translates.
SABnzbd ignores LC_CTYPE and uses untranslated names.
That's the reason the check fails.

The prospect of implementing LC_CTYPE support isn't very nice.
Especially because Python sucks in this area.
I will need to discuss this issue with the team.

PS: a quick fix might be to resort to just counting the number of files
and not checking (at least for non-Windows and non-OSX).