Hyland Connect

damien77 · ‎10-09-2008

Hi,

I'm experiencing some problems in getting Freemarker renditions to parse web form character encodings properly. I'm generating content in a variety of languages and alphabets, and they should all be encoded as UTF-8. Unfortunately, my FTL renditions do not encode text properly (but XSL renditions work correctly).

I've written a simple test form to illustrate the problem.

My test schema is:

<?xml version="1.0" encoding="UTF-8"?>

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
           xmlns:alf="http://www.alfresco.org"
           xmlns:tag="http://www.taglab.com/test"
           targetNamespace="http://www.taglab.com/test"
           elementFormDefault="qualified">

  <!– Type declarations –>
  <xs:complexType name="testType">
    <xs:sequence>
      <xs:element name="text" type="xs:string" minOccurs="1" maxOccurs="1">
        <xs:annotation>
          <xs:appinfo>
            <!– WYSIWYG editor –>
            <alf:appearance>full</alf:appearance>
          </xs:appinfo>
        </xs:annotation>
      </xs:element>
    </xs:sequence>
  </xs:complexType>

  <!– Root element –>
  <xs:element name="test" type="tag:testType"/>
</xs:schema>‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

And I've written two test renditions, one in FTL and one in XSL.

XSL rendition:

<?xml version="1.0" encoding="UTF-8"?>

<xsl:stylesheet version="1.0"
    xmlns:xhtml="http://www.w3.org/1999/xhtml"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:tag="http://www.taglab.com/test"
    xmlns:fn="http://www.w3.org/2005/02/xpath-functions"
    exclude-result-prefixes="xhtml">
  <xsl:output method="html" encoding="UTF-8" indent="yes"
              doctype-public="-//W3C//DTD XHTML 1.0 Transitional//EN"
              doctype-system="http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'/>

  <xsl:preserve-space elements="*"/>
  
  <xsl:template match="/">
    <html>
      <head>
      </head>
      <body>
        <!– disable-output-escaping preserves HTML tags in CDATA –>
        <xsl:value-of select="/tag:test/tag:text" disable-output-escaping="yes"/>
      </body>
    </html>
  </xsl:template>
</xsl:stylesheet>‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

FTL rendition:

<#ftl encoding="UTF-8" ns_prefixes={"D":"http://www.taglab.com/test"}>

<html>
  <head>
    <META http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    ${test.text}
  </body>
</html>‍‍‍‍‍‍‍‍‍‍

So, if I now enter some data into the form, for example:

English български 简体中文繁體中文 Česky Dansk Suomi Deutsch Ελληνικά Italiano Norsk Polski Português Русский Español Svenska Türkçe

The generated XML is correct:

<?xml version="1.0" encoding="UTF-8"?>
<tag:test xmlns:alf="http://www.alfresco.org" xmlns:chiba="http://chiba.sourceforge.net/xforms" xmlns:ev="http://www.w3.org/2001/xml-events" xmlns:tag="http://www.taglab.com/test" xmlns:xf="http://www.w3.org/2002/xforms" xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <tag:text>English български 简体中文 繁體中文 Česky Dansk Suomi Deutsch Ελληνικά Italiano Norsk Polski Português Русский Español Svenska Türkçe</tag:text>
</tag:test>‍‍‍‍

The XSL rendition is correct:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns:tag="http://www.taglab.com/test" xmlns:fn="http://www.w3.org/2005/02/xpath-functions">
<head>
<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>English български 简体中文 繁體中文 Česky Dansk Suomi Deutsch Ελληνικά Italiano Norsk Polski Português Русский Español Svenska Türkçe</body>
</html>
‍‍‍‍‍‍‍‍

But the FTL has failed to encode the text properly:

<html>
  <head>
    <META http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    English ????????? ???? ???? ?esky Dansk Suomi Deutsch ???????? Italiano Norsk Polski Portugu?s ??????? Espa?ol Svenska T?rk?e
  </body>
</html>
‍‍‍‍‍‍‍‍‍

Any ideas why the FTL is ignoring the encoding directive? Or have I missed something? I've seen this discussed on the forums WRT to Freemarker scripts used in web scripts, but the solution didn't seem relevant to web forms.

The Alfresco server will ultimately be hosted by a third party, so I'd prefer to not modify configuration files on the server. I'm using Alfresco 2.2E.

Cheers,
Damien.

damien77 · ‎10-09-2008

I've found that this problem can be fixed by overriding the file.encoding property when starting up alfresco. (But as I've already stated, this is not my ideal solution)

This solution is also mentioned in https://issues.alfresco.com/jira/browse/AR-1041 and http://forums.alfresco.com/en/viewtopic.php?p=12070#12070

(Modifying the default encoding property for the freeMarkerProcessor bean does not work, at least in the case of WCM forms.)

In any event, I do not think that it is desirable to set the FTL encoding globally. Given that the XSLT and direct creation of HTML content both work 'out-of-the-box', I am inclined to think that the FTL encoding is at fault. Shouldn't the output 'inherit' the encoding specified in the FTL form?

Hyland Connect

Freemarker UTF encoding