[Synology NAS] Fixed non-ASCII chars and more...(2013/10/02)

Come up with a useful post-processing script? Share it here!
starsys
Newbie
Newbie
Posts: 33
Joined: July 26th, 2011, 4:17 am

Re: Fixed foreign accents on Synology NAS running SABnzbd

Post by starsys » May 29th, 2012, 4:12 am

Hello LapinFou.

Peux tu me dire quelles sont les différences entre la version SAbnzbd de Superzebulon et Synocommunity ?
Perso tu conseilles laquelle ?
Je devrais recevoir mon NAS de retour de SAV cette semaine.
Merci.

User avatar
LapinFou
Jr. Member
Jr. Member
Posts: 82
Joined: April 23rd, 2012, 7:35 am
Location: France, Caen

Re: Fixed foreign accents on Synology NAS running SABnzbd

Post by LapinFou » May 29th, 2012, 10:34 am

Il n'a pas vraiment de différence, mis à part que le Package SynoCommunity inclus mon script depuis la version 0.6.15-2.

PS: Quand tout sera rentré dans l'ordre pour ton NAS, merci de poster un message pour dire si tout marche bien ;)
Synology Model: DS-411+
Firmware Version: DSM 5.2-5592
HDD Model: 4xSeagate ST2000DL003 2To - SHR

How to fix foreign accents issue with SABnzbd
Corriger Les Problème D'accents Sous Sabnzbd

starsys
Newbie
Newbie
Posts: 33
Joined: July 26th, 2011, 4:17 am

Re: Fixed foreign accents on Synology NAS running SABnzbd

Post by starsys » June 3rd, 2012, 4:05 am

Hello. J'ai récupéré mon NAS. tout semble marcher parfaitement.
Je vais chez mes parents cet après midi, je regarde aussi de leur coté mais y'a pas de raisons que ca marche pas. Merci encore.

User avatar
LapinFou
Jr. Member
Jr. Member
Posts: 82
Joined: April 23rd, 2012, 7:35 am
Location: France, Caen

Re: Fixed foreign accents on Synology NAS running SABnzbd

Post by LapinFou » June 3rd, 2012, 5:09 am

Merci pour le feedback.
Heureux de savoir que le renommage fonctionne bien ainsi que, je suppose, l'indexation auto dans le DLNA.
O0
Synology Model: DS-411+
Firmware Version: DSM 5.2-5592
HDD Model: 4xSeagate ST2000DL003 2To - SHR

How to fix foreign accents issue with SABnzbd
Corriger Les Problème D'accents Sous Sabnzbd

arto65
Newbie
Newbie
Posts: 35
Joined: July 26th, 2010, 3:05 am

Re: Fixed foreign accents on Synology NAS running SABnzbd

Post by arto65 » June 4th, 2012, 4:52 pm

Thanks a lot for that script!
I see that it would also be possible to enable full UTF8 support on the synology, but it's troublesome and has to be done after each dsm update. Your solution is easy and survives updates!
Your script is awesome, thanks again!

User avatar
LapinFou
Jr. Member
Jr. Member
Posts: 82
Joined: April 23rd, 2012, 7:35 am
Location: France, Caen

Re: Fixed foreign accents on Synology NAS running SABnzbd

Post by LapinFou » June 5th, 2012, 7:03 am

You're very welcome. I'm glad to see an happy user !!
:)

You're right this issue is coming from the fact that the default system file under root is NOT UTF-8. Soon, I'll try to contact Synology to see if they can add the locale CLI for a future DSM release. And you're also right the benefit of this script is to not be deleted after a DSM upgrade.
Synology Model: DS-411+
Firmware Version: DSM 5.2-5592
HDD Model: 4xSeagate ST2000DL003 2To - SHR

How to fix foreign accents issue with SABnzbd
Corriger Les Problème D'accents Sous Sabnzbd

starsys
Newbie
Newbie
Posts: 33
Joined: July 26th, 2011, 4:17 am

Re: Fixed foreign accents on Syno NAS running SABnzbd (06/05

Post by starsys » June 15th, 2012, 7:58 am

Testé aussi chez mes parents : aucun soucis.

Au top !

callten061
Newbie
Newbie
Posts: 6
Joined: July 21st, 2012, 6:09 am

Re: [Synology NAS] Fixed foreign accents (06/05)

Post by callten061 » July 22nd, 2012, 12:40 am

that it is working thanks!!!

User avatar
LapinFou
Jr. Member
Jr. Member
Posts: 82
Joined: April 23rd, 2012, 7:35 am
Location: France, Caen

Re: [Synology NAS] Fixed foreign accents (06/05)

Post by LapinFou » July 22nd, 2012, 5:06 am

You are welcome !!
;D
Synology Model: DS-411+
Firmware Version: DSM 5.2-5592
HDD Model: 4xSeagate ST2000DL003 2To - SHR

How to fix foreign accents issue with SABnzbd
Corriger Les Problème D'accents Sous Sabnzbd

User avatar
LapinFou
Jr. Member
Jr. Member
Posts: 82
Joined: April 23rd, 2012, 7:35 am
Location: France, Caen

Re: [Synology NAS] Fixed foreign accents (07/25/2012)

Post by LapinFou » July 25th, 2012, 10:56 am

Edit 25th of July: added some snapshots
Synology Model: DS-411+
Firmware Version: DSM 5.2-5592
HDD Model: 4xSeagate ST2000DL003 2To - SHR

How to fix foreign accents issue with SABnzbd
Corriger Les Problème D'accents Sous Sabnzbd

User avatar
LapinFou
Jr. Member
Jr. Member
Posts: 82
Joined: April 23rd, 2012, 7:35 am
Location: France, Caen

Re: [Synology NAS] Fixed foreign accents (2012/09/16)

Post by LapinFou » September 16th, 2012, 12:26 pm

[Edit 16th of September: added 7z unpack capability + move to a destination folder + DLNA indexer]
[Also proposed a modified sabToSickBeard.py version in order to unpack .7z archive.]
Synology Model: DS-411+
Firmware Version: DSM 5.2-5592
HDD Model: 4xSeagate ST2000DL003 2To - SHR

How to fix foreign accents issue with SABnzbd
Corriger Les Problème D'accents Sous Sabnzbd

Raik85
Newbie
Newbie
Posts: 2
Joined: October 26th, 2012, 4:16 pm

Re: [Synology NAS] Fixed foreign accents (2012/09/16)

Post by Raik85 » October 26th, 2012, 4:19 pm

Sorry, but i don't get it and it makes me crazy ;/

Where in Line 54 can i enable DLNA indexing with synoindex?

I see some Guys have the same Question in Frensh, but i can't read it - shame on me.

My CharTranslator.py:

Code: Select all

#!/usr/local/sabnzbd/env/bin/python -OO
# -*- coding: iso-8859-15 -*-
#
# If a file has been archieved under an ISO-8859 environment and unarchived
# under an UTF8 environment, then you will get an encoding format problem.
# The file will not be readable through SAMBA.
#
# Renaming script for SABnzbd runnning under Synology NAS.
# By default the NZB software is running under UTF-8 encoding format
# in order to correctly handle the french accents (éèàç...) a SABnzbd
# post-script must be run in order to convert the file encoding.
#
# To fix this problem, you must convert the encoding format
# to the UTF8 (default Synology encoding)
# The script is trying to detect if the original file/directory are coming
# from a RAR archive. In this case the unrar command on your Syno system will
# unpack in CP437 format (DOS).
# NB: in all cases, files will be readable through samba, even if the detection
# failed. But converted characters will not be good, ex: Î? instead é
# 
# Remark: I guess it should work for any other encoding style. Just replace
# ISO-8859-15 (Western Europe) by the one coresponding to your country:  
# http://en.wikipedia.org/wiki/Character_encoding#Common_character_encodings
# 
# Done by LapinFou
#   date   | version |     comment
#--------------------------------------
# 12-04-22 |   1.0   | Initial version
# 12-04-22 |   1.1   | Change encoding to ISO-8859-15
#                    | Added CP437 special characters (0x80-0x9A)
# 12-04-24 |   1.2   | Mixed encoding is now supported
#                    | UTF-8 encoding format detected
# 12-04-24 |   1.3   | Fixed typo line 57 (test must be 0xA0, not 0xA1)
#

# get library modules
import sys, os

########################
# ----- Functions ---- #
########################

# Special character hex range:
# CP437: 0x80-0x9A (fortunately not used in ISO-8859-15)
# UTF-8: 1st hex code 0xC2-0xC3 followed by a 2nd hex code 0xA1-0xFF
# ISO-8859-15: 0xA0-0xFF
# The function will detect if fileDirName contains a special character
# If there is special character, detects if it is a UTF-8, CP437 or ISO-8859-15 encoding
def renameFunc(fullPath, fileDirName):
	encodingDetected = False
	# parsing all files/directories in odrer to detect if CP437 is used
	for Idx in range(len(fileDirName)):
		# /!\ detection is done 2char by 2char for UTF-8 special character
		if (len(fileDirName) != 1) & (Idx < (len(fileDirName) - 1)):
			# Detect UTF-8
			if ((fileDirName[Idx] == '\xC2') | (fileDirName[Idx] == '\xC3')) & ((fileDirName[Idx+1] >= '\xA0') & (fileDirName[Idx+1] <= '\xFF')):
				print os.path.join(fullPath, fileDirName) + " -> UTF-8 detected: Nothing to be done"
				encodingDetected = True
				break;
			# Detect CP437
			elif (fileDirName[Idx] >= '\x80') & (fileDirName[Idx] <= '\x9A'):
				utf8Name = fileDirName.decode('cp437')
				utf8Name = utf8Name.encode('utf-8')
				os.rename(os.path.join(fullPath, fileDirName), os.path.join(fullPath, utf8Name))
				print os.path.join(fullPath, utf8Name) + " -> CP437 detected: Renamed"
				encodingDetected = True
				break;
			# Detect ISO-8859-15
			elif (fileDirName[Idx] >= '\xA0') & (fileDirName[Idx] <= '\xFF'):
				utf8Name = fileDirName.decode('iso-8859-15')
				utf8Name = utf8Name.encode('utf-8')
				os.rename(os.path.join(fullPath, fileDirName), os.path.join(fullPath, utf8Name))
				print os.path.join(fullPath, utf8Name) + " -> ISO-8859-15 detected: Renamed"
				encodingDetected = True
				break;
		else:
			# Detect CP437
			if (fileDirName[Idx] >= '\x80') & (fileDirName[Idx] <= '\x9A'):
				utf8Name = fileDirName.decode('cp437')
				utf8Name = utf8Name.encode('utf-8')
				os.rename(os.path.join(fullPath, fileDirName), os.path.join(fullPath, utf8Name))
				print os.path.join(fullPath, utf8Name) + " -> CP437 detected: Renamed"
				encodingDetected = True
				break;
			# Detect ISO-8859-15
			elif (fileDirName[Idx] >= '\xA0') & (fileDirName[Idx] <= '\xFF'):
				utf8Name = fileDirName.decode('iso-8859-15')
				utf8Name = utf8Name.encode('utf-8')
				os.rename(os.path.join(fullPath, fileDirName), os.path.join(fullPath, utf8Name))
				print os.path.join(fullPath, utf8Name) + " -> ISO-8859-15 detected: Renamed"
				encodingDetected = True
				break;
	if (encodingDetected == False):
		print os.path.join(fullPath, fileDirName) + " -> No special characters detected: Nothing to be done"
	
	return

########################
# --- Main Program --- #
########################

# get argument: directory of the SABnzbd job
os.chdir(sys.argv[1])

# display directory of the SABnzbd job
currentFolder = os.getcwd()
print "Current folder is" + currentFolder

# rename SABnzbd job directory (coming from SABnzbd: never in CP437 format)
print "Renaming folders to UTF-8..."
renameFunc('', currentFolder)

# process each sub-folders starting from the deepest level
for dirname, dirnames, filenames in os.walk('.', topdown=False):
	for subdirname in dirnames:
		renameFunc(dirname, subdirname)
print "Folder renaming Done !"
print ""

# process each file recursively
print "Renaming files to UTF-8..."
for dirname, dirnames, filenames in os.walk('.'):
	for filename in filenames:
		renameFunc(dirname, filename)
print "Files renaming Done !"
print ""
print "Character encoding translation done!"

User avatar
LapinFou
Jr. Member
Jr. Member
Posts: 82
Joined: April 23rd, 2012, 7:35 am
Location: France, Caen

Re: [Synology NAS] Fixed foreign accents (2012/09/16)

Post by LapinFou » October 28th, 2012, 10:07 am

Hi Raik85,

Your problem is: you don't have the latest version of the script ! ::)
Please copy/paste from the first post the version 1.4.
Please be careful:
1- Don't used SELECT ALL button of this forum, otherwise a tab will be added in front of each line, and the Python script will not bet able to be launched.
2- Used an advanced editor (such as NotePad++), to ensure EOL (End OF Line) are in Unix format and not in Windows format. Otherwise, you will get an error like '-ash command can not be found'
3- Be sure the first line match where your Python command is installed.

Keep me in touch!
Last edited by LapinFou on October 30th, 2012, 4:30 pm, edited 1 time in total.
Synology Model: DS-411+
Firmware Version: DSM 5.2-5592
HDD Model: 4xSeagate ST2000DL003 2To - SHR

How to fix foreign accents issue with SABnzbd
Corriger Les Problème D'accents Sous Sabnzbd

Raik85
Newbie
Newbie
Posts: 2
Joined: October 26th, 2012, 4:16 pm

Re: [Synology NAS] Fixed foreign accents (2012/09/16)

Post by Raik85 » October 30th, 2012, 3:37 pm

Hervorragend!

I thank you, now it works ;-)

User avatar
LapinFou
Jr. Member
Jr. Member
Posts: 82
Joined: April 23rd, 2012, 7:35 am
Location: France, Caen

Re: [Synology NAS] Fixed foreign accents (2012/09/16)

Post by LapinFou » October 30th, 2012, 4:31 pm

You're welcome!
Synology Model: DS-411+
Firmware Version: DSM 5.2-5592
HDD Model: 4xSeagate ST2000DL003 2To - SHR

How to fix foreign accents issue with SABnzbd
Corriger Les Problème D'accents Sous Sabnzbd

Post Reply