Problem with accented characters

[fouellet]

Hi,

I installed the latest French pack to my Enterprise Network environment on Linux, but in Firefox all the accented characters are translated into strange characters… Through the "view source" we can see that in the rendered HTML the characters have been translated to two &#nnn;
values which have nothing to do with the original characters.

Any option we need to set in the config files to change the encoding?
I tried various values in Firefox's "languages" menu but nothing seems to work.

Thanks.

[pierre]

Hi François,

I downloaded french translation files on my computer (Windows XP)
and copied these (using WINSCP) into the right folder :

/opt/alfresco/tomcat/webapps/alfresco/WEB-INF/classes/alfresco/messages‍

- I had the same problem so i thought it was a char - encoding problem
- I had to open translation files using WINSCP and add # then remove # to save it again on the server
- Everything is back to normal

[img]http://img511.imageshack.us/img511/9510/screenie0cn.png[/img]

[fshipley]

Hi,

The French language files are encoded in Unicode ( UTF-8 ). What encoding are you using in Firefox ?

The files also have Windows line delimiters ( CRLF ).

The only # characters in the language properties files are used for comments (# as the first character on a line). Pierre, are those the # characters you had to change or did your editor replace the accentuated characters with &#nnn; in the properties file.

The properties files in the language pack have also been converted with the native2ascii converter (http://java.sun.com/j2se/1.4.2/docs/tooldocs/windows/native2ascii.html). Once converted, the accentuated characters are strings like \u00e0 for à etc.

Any extra info you have would be appreciated so I can understand what is happening.

Thanks for your help,
Frank

[pierre]

I think the bug happened because I have copied files from Windows to Linux.

A problem that often bites people working with different platforms, such as a PC running Windows and a web server running Linux, is the different character codes used to terminate lines in text files.

Windows, and DOS before it, uses a pair of CR and LF characters to terminate lines. UNIX (Including Linux and FreeBSD) uses an LF character only. The Apple Macintosh, finally, uses a CR character only. In other words: a complete mess.

Problems arise when transferring text files between different operating systems and using software that is not smart enough to detect the line break style used by a file. E.g. if you open a UNIX file in Microsoft Notepad, it will display the text as if the file contained no line breaks at all. If you open a Windows file in a UNIX editor like "joe" or "vi", you will see a control character (the CR) at the end of each line. Older versions of Perl on Linux would refuse to run any script that used Windows line breaks, aborting with an unhelpful error message.

source : http://www.editpadpro.com/tricklinebreak.html

I just added an # at the beginning of the files to know which ones are good and those who aren't.

Example:

before

# Action service externalised display strings

compare_property_value_evaluator.invalid_operation=L''opération {0} ne peut pas être appliquée sur une propriété de type {1}.

after

#
# Action service externalised display strings

compare_property_value_evaluator.invalid_operation=L''opération {0} ne peut pas être appliquée sur une propriété de type {1}.

Hope you've understood now,

Best regards,

[fshipley]

Hi Pierre,

Did you get that file from Sourceforge because your example line should already have been converted with native2ascii and look as follows :

# Action service externalised display strings

compare_property_value_evaluator.invalid_operation=L''op\u00e9ration {0} ne peut pas \u00eatre appliqu\u00e9e sur une propri\u00e9t\u00e9 de type {1}.

Thanks,
Frank

[pierre]

I didn't get the files from sourceforge, i downloaded files from

The source files encoded in UTF-8 can be found here :
https://svn.neodoc.biz/alfresco/trunk/i18n/fr_FR/
source : http://www.alfresco.org/forums/viewtopic.php?t=150

I had problems with UTF8 because of this in my config file :
/etc/sysconfig/i18n
LANG="fr_BE"
SUPPORTED="fr_BE:fr_BE:fr"

Had to rename my translation files from action-config_fr_FR.properties
to action-config_fr_BE.properties to make it work and now it's working quite well ! 

[fshipley]

Hi Pierre,

Yes, so that's probably what the problem is. We just need confirmation from François (fouellet).

The French language source files on https://svn.neodoc.biz/alfresco/trunk/i18n/fr_FR/ should not be used directly with your Alfresco installation. They are the source files we use for updating the French translation. These source files need to be converted with native2ascii before they are used in Alfresco.

The French language pack available for download on sourceforge http://sourceforge.net/project/showfiles.php?group_id=143373&package_id=162536 should be used for installing the French language files. The language pack  files have already been converted with native2ascii.

Hope there isn't any more confusion.

Best regards,
Frank

[fouellet]

:cry: Thank you all for the goods tips, but still, it does not seem to work!

The files were not copied from Windoze to Linux, and when I look at them with vim or OpenOffice the accented characters look good, there are no extraneous CR/LF anywhere…

When I start Gnome with the fr_FR.UTF-8 locale all the X menus come in French and even the main Alfresco login page shows text in French.

But when getting to the main navigation screen with Firefox characters are not rendered properly. For example, the "é" gets translated to "é" (literally the ampersand followed by the #NNN; code).
So it looks like a Firefox issue?

Why would the french characters get translated to two characters :?:

In Firefox I set the language to be fr_fr with UTF-8. Would Firefox use a different locale translation?

:roll: Puzzled…

Oh, one more thing:

In the web-client-config.xml file  the
<language locale="fr_FR">Français</language>
line gets the word "Français" to show properly in the login drop list!
Why would the ç show up normally here and not on the other pages?

Thanks.

[pierre]

Thanks for the info Frank

I don't think Firefox is the problem… I had the same problem with Firefox even if I changed View -> "Unicode" or "Occidental" it was showing é characters in both cases.  We are willing to help you but we need more informations on your config.

1. About your translation files : sourceforge or svn.neodoc.biz ?
2. Could you tell us what's your linux distribution ?
3. Could you show us the content of your /etc/sysconfig/i18n file ?
4. alfresco.log ok ?

Best regards,

[fouellet]

Solved!
Using native2ascii to convert all files got everything sorted out.
Still cannot understand why the "Français" shows up properly in the drop list though…

BTW I found a few typos in the French package, I will submit corrections, and will probably put together a fr_CA package for French-canadian.

Thanks all. This product and forum rock!  :wink: