character encoding problem with processTemplate?

[alrice]

All, I am getting very productive with the webscripts environment. It's great. But i ran into an urgent and puzzing character encoding issue. Any suggestions greatly appreciated- thanks in advance. We have lots of foreign names in this application, so it's important to be able to support accented characters at least, and preferably in utf-8 unicode.

So i've been writing all my html forms in dreamweaver doctype xhtml, and

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />‍

The html forms have this type and charset specified also:

<form action="${url.serviceContext}/apply/postdoc08/edit?guest=true" 
        method="post"
        enctype="multipart/form-data"
      charset="utf-8">‍‍‍‍

I have a freemarker template applicant.xml.ftl that starts off

<?xml version="1.0" encoding="UTF-8"?>‍

So everything should be utf8 right? Indeed if i edit my source xml document with extended characters,  and save as UTF8 in TextEdit or Oxygen.app, then all my html pages render the text just fine. Those freemarker templates are xhtml, utf-8, and are loading the xml file thusly:

<#assign dom=document.xmlNodeModel>‍

So that all works, but the problems arises when submitting the html form. The POST is received by a .js file, which applies the template and saves the result out to the repository. Here is where i think the character encoding is getting lost:

var tScriptSpace = companyhome.childByNamePath( gPathToScriptSpace  );
var tXMLTemplateNode = tScriptSpace.childByNamePath( gXMLTemplate );
var tArgs = formFieldsToArgsArray();
var tNewApplicantXMLStr = tApplicantXMLNode.processTemplate( tXMLTemplateNode, tArgs);
tApplicantXMLNode.content = tNewApplicantXMLStr;
‍‍‍‍‍‍

My best guess is the call to processTemplate() is returning the wrong character encoding somehow? Not sure. Where have I gone wrong or is it a problem with processTemplate or something i have overlooked?

[kevinr]

Tomcat expands the WAR into a folder called 'alfresco' in the webapps folder. Inside you will find WEB-INF/lib/alfresco-webclient.jar. You can expand that JAR file (it is just a ZIP archive) using any ZIP tool, overrwite the class file that i provide (keeping the folder structure intact!) and then repack up the jar file archive and overwrite it in the WEB-INF/lib folder.

Once that is done (and you've restarted the server of course) you can try the Upload example or your example and it should work ok with upload UTF-8 characters.

Obviously don't do this on your production server etc. etc.

Kevin

[alrice]

Thanks that makes sense Kevin. Alright, I tested it out but am not seeing any different behavior. I added a new comment to the JIRA issue.

[kevinr]

My investigations have found a further issue when webscripts using the /service path do not have the character encoding set correctly on the request. This is why the fix isn't working for you, I am using the /wcservice path (using the webclientauthenticator) which is correctly setting the encoding on the request.

Please see my further comments on the jira bug report.

I will provide a new fix for this shortly.

Kevin

[kevinr]

Please find an updated patch here  (contains 3 updated .class files for the alfresco-web-client.jar). It should work correctly with /service based webscripts.

Thanks,

Kevin

[kevinr]

A further issue was found (with the patch itself doh), updated files available on the JIRA issue.

Thanks,

Kevin