Charset & encoding through CIFS / Webdav / FTP...

[jgoffinet]

Hi,

I've made some tests through several protocols. Here is the config I used:

- server: linux (debian) / mysql / tomcat bundle (1.0.0 release) - no special config

- client: WinXP Pro SP2 with :
   * Windows Explorer to access CIFS share
   * Windows Explorer with network places to access Webdav
   * Windows Explorer with network places to access FTP
   * DavExplorer to access webdav
   * FileZilla to access FTP

Here are the results of my investigations:

1. I had no success in connecting to FTP with FileZilla. Here is the log on the client side (no log on the server side but a java.net.ConnectException: Connection timed out):

– beginning of log
PORT 192,168,1,113,6,173
Réponse :   200 Port OK
Commande :   LIST
Réponse :   150 File status okay, about to open data connection
Erreur :   Inactivité détectée !
Erreur :   N'a pas pu récupérer la liste du répertoire
– end of log

(the 2 last lines mean: "inactivity detected" and "couldn't retrieve directory list").

2. Everything worked fine otherwise… until I tried to create spaces with accents in their names. I created the followings:
- "cifs-é" with Win Explorer
- "webdav-é" with network place
- "ftp-é" with network place
- "davexplo-é" with dav explorer
- "web-é" with web client
and looked at the result in the different clients.

3. CIFS, Web client, DavExplorer: for spaces created with CIFS, web client and DavExplorer, there seems to be no problem: space names are shown correctly, and it is possible to browse into them.

4. FTP (through network place): the folder "ftp-é" is shown as "ftp-?" in every client, even the one which was used to create it. It is impossible to browse into it through Network place. It is browsable only via web client and dav explorer (even if it is displayed as ftp-?). cifs-é, web-é, etc. are also shown as cifs-?, web-?, etc and are not browsable through FTP network place.

5. Webdav (through network place): the folder "webdav-é" is shown as webdav-é in every client (network place, cifs share, web client, dav explorer),but it is possible to browse into it. Other folders are shown correctly (cifs-é, web-é, etc.) but trying to access them leads to an error (folder unavailable).

Did anyone else encounter such troubles with accentuated files ?

Jean

[lgr]

The UTF-8 support in FTP fixes problems with accents/umlauts etc. in 1.4 community - but it depends on the FTP client enabling it.

Ok.

Actually, i tested with 3 ftp clients, i was not sure of utf8 support of two of them (ftp through windows explorer network places and windows command prompt ftp client, both winxp sp2 FR), but the third one, filezilla 2.2.29 under windows, has different behaviours while forcing or not utf-8 enabling. And these 2 behaviours don't create correctly the accented filenames.

Can you tell me which ftp client to use, to have a correct utf-8 support ?

Laurent.

[lgr]

Just another information, i've checked with smartftp 2.0.1, and it didn't work either.

but something pointed out : when issuing FEAT command, only "SIZE" is answered, none "UTF8" or anything else.

Is there something to configure so that Alfresco answers UTF8 when asking for ftp features ?

Laurent.

[lgr]

Kevin,

Do you have any more information about UTF8 support ?

I don't see any feat UTF8 in the repository source :
in java/org/alfresco/filesys/ftp/FTPSrvSession.java :

   protected final void procFeatures(FTPRequest req) throws IOException
    {
        // Check if the user is logged in

        if (isLoggedOn() == false)
        {
            sendFTPResponse(500, "");
            return;
        }

        // Send back the list of features supported by this FTP server

        sendFTPResponse( 211, "Features");
        sendFTPResponse( "SIZE");
        sendFTPResponse( 211, "End");
    }
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Did i miss something or http://issues.alfresco.com/browse/AR-628 does not match the released source ? (included in alfresco-sdk-1.4.0)

Laurent.

[paulhh]

Hi Laurent

That issue was fixed after the Community release was made, in the extended QA phase for the Enterprise release.  Like another fixes that were made in this phase, they will get folded into the HEAD code before the next release.  We've just merged in the WCM code, so these other fixes will arrive over the next few weeks.

Cheers
Paul.

[lgr]

Ok, all is clear now

Thanks Paul,

Laurent.

[tarkadu]

Please look at Webdav as well - I cannot make a connection using Windows Explorer and Webdav to folders with German Umlauts in their names - for all other folders it's all right.

If it is a settings problem please inform me!

Thanks,

Tarkadu

[lgr]

Hi Laurent

That issue was fixed after the Community release was made, in the extended QA phase for the Enterprise release.  Like another fixes that were made in this phase, they will get folded into the HEAD code before the next release.  We've just merged in the WCM code, so these other fixes will arrive over the next few weeks.

Hi Paul,

I think the issue is not really closed.
Uploading of accented files works well, that is fixed.
But working with accented folders does not seem to work well.
When i try to upload a file which is located in a folder which name is accented, my favorite filezilla dialogs with the FTP server :

OPTS UTF8 ON
200 OPTS UTF8 ON
PWD
257 "/"
CWD /Alfresco/Clients/Sch\303\251ma/20060922/
250 Requested file action OK
PWD
257 "/Alfresco/Clients/Sch?ma/20060922/"
‍‍‍‍‍‍‍‍‍

When uploading the file in the accented folder, my also favorite ftp under Windows explorer also says (the accent is missing) :

550 Invalid Path /Alfresco/Clients/Schma/20060922/‍

Laurent.

[lgr]

oops, i forgot to tell that i didn't find any jira issue on this subject.

Laurent.

[alexandra]

I have the same issues for folder names with Swedish characters åäö and ÅÄÖ. I have tried several different ftp clients and the same result. ISO-8859-1 removes the characters and with UTF-8 I only get questionmarks…

I think support for international characters is crucial for a platform like this. Especially since language localization is built-in for the GUI.

Any hints of this issue is fixed in Alfresco 2.0?

[gary_spencer]

Hi,

There is a <charSet> configuration tag for the FTP server to change the character set used for UTF-8 conversions rather than using the JVM default.

When running the server on a Mac with Windows FTP clients I had to use a setting of <charet>Cp1252</charSet> to get the correct conversions.

Cheers

Gary