Hyland

resplin · ‎06-05-2015

The official documentation is at: http://docs.alfresco.com

{{AVMWarning}}
AVM
File System Receiver

One of the 'deployment servers' for Alfresco. (The other being the ASR.)

The FSR is a small stand alone server that receives updates from an Alfresco repository running Web Content Management and publishes them to a flat file system. The published flat files will then typically be published by a web server such as Apache for static content or an application server such as Tomcat, Jboss or IIS for dynamic content.

From Alfresco 3.2 onwards The WCM Deployment Engine and the File System Deployment Target replace the FSR.

Installation

To install the FSR either build it from the source code with the alfresco ant build system
or install the alfresco deployment project downloaded from either source forge or the enterprise release website.

Building the FSR

First obtain the source code from Alfresco SVN

The ant build system assembles the FSR in the 'deployment/build' folder. The target to use is 'build-deployment'

Deployment of the FSR will simply consist of copying the files to the target location. (for example on windows c:/Alfresco/Deployment or on linux /opt/alfresco/deployment)

Downloading the FSR

Simply unzip the contents of the alfresco 'deployment' package into the target location.

Or use the 'deployment' installer for Windows and Linux which will install the FSR and guide you through the configuration.

Configuration

Configuration of the FSR is via a 'deployment.properties' file and the 'application-context.xml' file which are located in the FSR installation directory (for example on windows c:/Alfresco/Deployment or on linux /opt/alfresco/deployment).

deployment.properties


dep.datadir : where the FSR should store its temporary data
dep.logdir : where the FSR should store its log data
dep.metadatadir : where the FSR should save metadata
dep.rmi.port : default 44100.   The port number of the rmi registry
dep.rmi.service.port : default 44101.   The port number for the deployment service

application-context.xml

Here is the sample application-context.xml file.

  <bean id='configuration' class='org.alfresco.deployment.config.Configuration'
         init-method='init'>
       <property name='dataDirectory'>
           <value>${dep.datadir}</value>
       </property>
       <property name='logDirectory'>
           <value>${dep.logdir}</value>
       </property>
       <property name='metaDataDirectory'>
           <value>${dep.metadatadir}</value>
       </property>
       <property name='targetData'>
           <map>
            <entry key='default'>
                <map>
                 <entry key='root'><value>target</value></entry>
                 <entry key='user'><value>admin</value></entry>
                 <entry key='password'><value>admin</value></entry>
                </map>
            </entry>
            </map>
       </property>
   </bean>

root : where the FSR should place the deployed files, in this case './target'
user : used to restrict access to the FSR, in this case 'admin'
password : used to restrict access to the FSR, in this case 'admin'

error On Overwrite

Please note that this is a version 3 specific feature

If there is a mismatch between the metadata and the target filesystem there may be errors during deployment. For example if a file is deleted by hand to the target filesystem.

You can configure what the FSR should do if it attempts to delete a file that does not exist or create a file that is already there.

You can set the errorOnOverwrite parameter.

If true will throw an error, the default behaviour is to raise a warning and carry on.

Alfresco (sending side) configuration

number of send threads

This is a feature of version 3.0 and Labs 3C

To assist with cases where files are being deployed over a network with high latency the deployment client is multi-threaded and sends several files at once. By default 5 sends are done in parallel.

To change the number of sends in parallel change the numberOfSendingThreads property.

Example reducing the number of threads to 4.


  <bean id='deploymentService' class='org.alfresco.repo.deploy.DeploymentServiceImpl' lazy-init='default' autowire='default' dependency-check='default'>
    <property name='avmService'>
      <ref bean='indexingAVMService' /> 
    </property>
    <property name='numberOfSendingThreads'>
      <value>4</value> 
    </property>
    <property name='deploymentReceiverTransportAdapters'>
     <map>
      <entry key='default'>
       <ref bean='rmiFSRAdapter' /> 
      </entry>
     </map>
    </property>
  </bean>

number of deployments in parallel

Deployment is controlled through the Action Service which controls how many deployments happen in parallel. If you need to deploy to many servers then you may need to increase the number of deployments in parallel. But if you run out of processing power or memory then you may need to reduce this setting.

Version 3 of alfresco has its own pool of deployment threads, older versions of Alfresco share the same thread pool for all actions.

Version 3 configuration - extract from action-services-context.xml showing configuration of the deployment pool.


<bean id='deploymentAsyncThreadPool' class='org.alfresco.util.ThreadPoolExecutorFactoryBean' lazy-init='default' autowire='default' dependency-check='default'>
- <property name='poolName'>
  <value>deploymentAsyncAction</value> 
  </property>
- <property name='corePoolSize'>
  <value>2</value> 
  </property>
- <property name='maximumPoolSize'>
  <value>3</value> 
  </property>
</bean>

RMI Networking Notes

RMI is a simple to use java networking protocol.

At the cost of a little performance you can also run the FSR on a single tcp port,
just make dep.rmi.port and dep.rmi.service.port the same number.

You can also run the rmi registry on another machine.

To run any remote application over a network requires the correct permissions, in particular unix (and its variants) will require 'root' permissions to run on ports less than 1024. And if there is a firewall then it needs to allow remote connection to the ports used by the FSR.

Multi NIC Receivers

When deploying to a deployment server running on a multi-NIC system it may be necessary to bind the RMI registry to a particular ip address. To do this you must add the following to the java command in deploy_start.sh/.bat:

-Djava.rmi.server.hostname=x.x.x.x

Where x.x.x.x is the IP address assigned to the NIC you want to bind to.

Spring Documentation for more information.

Thanks to Rakesh Kalra at Care.com

Deployment on Linux

When deploying on Linux some distros have an a firewall active. The firewall needs to be configured to allow Alfresco to communicate with the FSR.

Need to set the /etc/hosts file correctly.

Need to set the java.rmi.server.hostname as #Multi NIC Receivers

Need to set permissions such that the FSR can read and write to the filesystem.

FSR metadata

The FSR has a simple repository of metadata to track which files, directories and versions are on the target file system. This data is maintained automatically by the FSR. It reduces the amount of files that need to be deployed.

In order to force a full redeploy you can safely delete the contents of the metadata directory, but please ensure that a deployment is not already in progress!

The location of the metadata is configured through the ${dep.metadatadir} property.

Version 3 FSRs have functionality to validate that the metadata and the target deployment are consistent. Validation occurs after the FSR starts up and after the FSR detects an inconsistency. For example trying to delete a file that does not exist or overwriting a file that was outside the control of the FSR.

The FSR can either issue an error upon detecting a problem or automatically fix the problem. There is an autoFix parameter in the definition of the Target which controls whether the FSR will attempt to fix the metadata itself or just issue a warning.

  <property name='targetData'>
     <map>
        <entry key='default'>
           <map>
            <entry key='root'><value>target</value></entry>
            <entry key='user'><value>buffy</value></entry>
            <entry key='password'><value>slayer</value></entry>
               <entry key='autoFix'><value>true</value></entry>
           </map>
        </entry>
     </map>
   </property>

FSR log directory

The log directory is used to store log files which track the status of a deployment as it is in progress.
You may delete these files to save disk space, but make sure you do not attempt to delete the log file of a deployment that is in progress.

FSR data directory

The FSR uses a temporary directory to store content prior to a deployment being comitted.
During the commit phase of a deployment content is copied from this temporary directory to the destination filesystem.
This filesystem should only contain data during a deployment.

Starting / Stopping the FSR

The FSR is implemented as a set of java libraries and is inherently multi-platform.
To simplify the command line, Bourne shell scripts are provided for UNIX and Windows batch files are provided for Windows.

Please note that the deploy_stop was released in the 2.1.6 service pack, it is not part of 2.1.0.

run the deloy_start script to start the file system receiver.

run the deploy_stop script to stop the file system receiver.

On Windows

If you have used the deployment installer to install the FSR then there will be Start Menu actions available to start or stop the FSR from the Start Menu. These are simply calling the deploy_start.bat and deploy_stop.bat scripts.

You can also call the deploy_start and deploy_stop scripts from a command prompt.

It is also possible to install the FSR as a Windows service which can automatically start when Windows starts.

Extending the FSR

postCommit callback

This feature is available in 2.1.0 and upwards.

There is a postCommit callback available which is called after each successful deployment. For example some users of alfresco use ftp to transfer files or send emails when deployment is complete.

If exceptions are thrown by the callbacks then the exceptions will be logged. But it is too late to do anything else.

There is a sample adapter that will call an operating system command, such as a batch script, or the callback can call a custom java class.

The callback(s) are configured in the 'target' properties of the deployment configuration. They are called in sequence.

Diagram to show the FSDeploymentRunnable class

FSDeployment runnable class.jpg

Users need to implement the 'FSDeploymentRunnable' interface.

Two implementations are provided.

SampleRunnable : an example java class that demonstrates how to use this feature
ProgramRunnable : execute a operating system command after deployment

prepare callback

This feature is avilable in Alfresco 3.1.0/3D and upwards.

The prepare callbacks (which are also implementations of the same FSDeploymentRunnable interface used in postCommit callbacks) are called by the FSR immediately after the complete set of deployment files have been transferred to the FSR's temporary storage areas.

If exceptions are thrown by the callbacks then the deployment will fail and be rolled back. The exception will be propagated to the system calling the FSR and the exception should be logged on both FSR and calling system.

The callbacks occur within the deployment transaction so should minimise the time they take to run since there are resources waiting for deployment to complete.

Payload transformation

This is a feature of version 3.0 & Labs 3C

The data streams out of Alfresco and into the FSR can be transformed by content transformation.

To support use/cases such as

file compression for slow networks.
encryption for sensitive data.
character code page conversion.
transformation of content as it is deployed.

Users wishing to build their own transformers will implement the interfaces org.alfresco.deployment.DeploymentTransportInputFilter and org.alfresco.deployment.DeploymentTransportOutputFilter

Three sample transformers are provided in the package org.alfresco.deployment.transformation. You can use the existing three transformers as a model for writing your own payload transformers.

CompressionTransformer : compresses the outgoing stream and decompresses the incomming stream.
SampleEncryptionTransformer : encrypts the outgoing stream and decrypts the incomming stream.
TestCompressionTransformer : does nothing but enables testing.

On the Alfresco Server, content transformers are defined in the spring configuration file 'deployment-service-context.xml'
On the File System Receiver, transformers are defined in spring configuration file 'application-context.xml'.

The content transformers may have initialisation parameters.

Here is an extract from deployment-service-context.xml showing the configuration of two transformers. The first transformer has two initialisation properties.


     <bean id='deploymentEncryptor' class='org.alfresco.deployment.transformers.SampleEncryptionTransformer'>
        <property name='password'>
          <value>Alfresco</value>
        </property>
        <property name='cipherName'>
          <value>PBEWithMD5AndDES</value>
        </property>
     </bean>  
     
     <bean id='deploymentCompressor' class='org.alfresco.deployment.transformers.CompressionTransformer'>
     </bean>

Once the content transformers have been defined in 'deployment-service-content.xml' or 'application-context.xml' then then need to be plugged into the transport adapter.

Transport Adapters

This is a new feature in available in version 3.0 & Labs 3C

The Alfresco deployment service supports the configuration of multiple transport adapters to enable connection to remote file system receivers using different network protocols.

Each transport adapter provide an implementation of the Transport protocol and is provided with an optional list of payload transformers. Transport adapters implement the interface DeploymentReceiverTransportAdapter which defines a single method to obtain an instance of DeploymentReceiverTransport.

Transport adapters are configured on the Alfresco Server in the spring configuration file 'deployment-service-context.xml'.
In the FSR transformers are configured in the 'application-context.xml' file. An instance of Alfresco server may support many different transports but each FSR only exposes a single transport.

By default only the 'default' transport is defined which will allow communication with existing versions of alfresco.

Here is an extract from deployment-service-context.xml showing the configuration of two transformer adapters.



     <bean id='rmiFSRAdapter' class='org.alfresco.deployment.impl.client.DeploymentReceiverTransportAdapterRMI'>
     </bean>
     
     <bean id='encryptedRMIFSRAdapter' class='org.alfresco.deployment.impl.client.DeploymentReceiverTransportAdapterRMI'>
          <property name='transformers'>
            <list>
             <ref bean='deploymentEncryptor'></ref>
            </list>
        </property>
     </bean>

The final piece of configuration is in the configuration of the deployment server, which is passed a list of transport adapters.

The name 'default' is reserved to mean the communication method used by older versions of Alfresco. You are free to name your own adapters.

Here is an extract from deployment-service-context.xml showing the configuration of two transformer adapters.

    <bean id='deploymentService' class='org.alfresco.repo.deploy.DeploymentServiceImpl'>
       <property name='avmService'>
           <ref bean='indexingAVMService'/>
       </property>
       
       <property name='deploymentReceiverTransportAdapters'>
           <map>
              <entry key='default'>
                <ref bean='rmiFSRAdapter'></ref>
              </entry>
              <entry key='encrypted RMI'>
                <ref bean='encryptedRMIFSRAdapter'></ref>
              </entry>
           </map>
       </property>
    </bean>

Here is an extract from application-context.xml showing the configuration of the transport adapter to support the encrypted RMI defined above.

    
<bean id='deploymentReceiverService' class='org.alfresco.deployment.impl.server.DeploymentReceiverServiceImpl' 
          init-method='init'>
        <property name='configuration'>
            <ref bean='configuration'/>
        </property>
        <property name='transformers'>
          <list>
            <ref bean='deploymentEncryptor'/>
          </list>
        </property>
    </bean>

Once the transport adapters have been defined then the 'Edit Web Project Settings' UI action will enable the transport adapter to be selected.

Transport Name.GIF

Running the FSR over protocols other than RMI

RMI is a simple to use Java network protocol however there are one or two potential problems which is why some users are interested in using other network protocols.

interoperability issues:although rare there can be problems communicating between different implementations of Java.
encryption: the RMI protocol is not encrypted so sensitive data can be read by the devious.
corporate standards: organisations may have standards for which communication protocols are allowed or they may have infrastructure that requires messages of a certain type, for example a http load balancer.

Tunnelling RMI over HTTP and HTTPS

One of the features of the Sun implementation of RMI is that if the RMI ports are blocked by a firewall then the transport automatically attempts to connect via HTTP on port 80. And a Java web server can then process this HTTP traffic and forward it to the RMI server.

Article on tunnelling with Tomcat

Before using tunnelling please consider the security implications. For an externally facing web site you will probably want to install another web server and block access to it from the outside with a firewall.

Replacing RMI

This is a feature is available in version 3.0 & Labs 3C

Spring provides a number of 'adapters' for different network protocols. Alfresco uses Spring and the FSR is coded such that the protocol can be switched by defining a transport adapter that will connect the deployment service to your network protocol. You can also go a step further and define your own protocol, either by extending spring or building your own transport. You can use the existing class DeploymentReceiverTransportAdapterSpringHTTP as a model to get started.

Spring remoting reference document

Running the FSR over HTTP

HTTP is a well known protocol that is supported via a wide variety of web servers.

To run an FSR over http requires two configuration changes.

a) In the Alfresco server configuration define a transport adapter that uses spring http.

b) On the File System Receiver expose the FSR transport interface via spring http

Here is an extract from deployment-service-context.xml showing the configuration of the spring http adapter.


     <bean id='springHttpFSRAdapter' class='org.alfresco.deployment.impl.client.DeploymentReceiverTransportAdapterSpringHTTP'>
     </bean>

The final piece of configuration on the Alfresco server is to pass this adapter into the deployment service.


     <bean id='deploymentService' class='org.alfresco.repo.deploy.DeploymentServiceImpl'>
        <property name='avmService'>
            <ref bean='indexingAVMService'/>
        </property>
        
        <property name='deploymentReceiverTransportAdapters'>
            <map>
               <entry key='default'>
                 <ref bean='rmiFSRAdapter'></ref>
               </entry>
               
               <entry key='spring HTTP'>
                 <ref bean='springHttpFSRAdapter'></ref>
               </entry>
            </map>
        </property>
     </bean>

To expose the FSR over http requires a web server which will receive the http messages and delegate the content to the FSR.

The following example illustrates how to package the FSR as a web archive (WAR) to be served by a Java servlet engine such as Tomcat.


AlfrescoFSR.war
    WEB-INF
        deployment-config.xml
        deployment-servlet.xml
        web.xml
  
    classes
        commons-logging.properties
        deployment.properties
        log4j.properties
        
    lib
        alfresco-core.jar
        alfresco-deployment.jar
        commons-logging-1.0.4.jar
        jug-lgpl-2.0.0.jar
        spring-2.0.2.jar

Here are the contents of the web.xml file.


<web-app>
   <display-name>Alfresco FSR</display-name>
    
   <description>AlfrescoFSR in a HTTP Server</description>
   
   <distributable/>

    <servlet>
      <servlet-name>deployment</servlet-name>
      <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
      <load-on-startup>1</load-on-startup>
   </servlet>

   <servlet-mapping>
      <servlet-name>deployment</servlet-name>
      <url-pattern>/deployment</url-pattern>
   </servlet-mapping>

   <listener>
      <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
   </listener>

   <context-param>
      <param-name>contextConfigLocation</param-name>
      <param-value>/WEB-INF/deployment-config.xml</param-value>
   </context-param>
       
</web-app>

Here are the contents of the deployment-servlet.xml file.


<beans xmlns='http://www.springframework.org/schema/beans' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xmlns:p='http://www.springframework.org/schema/p' xmlns:util='http://www.springframework.org/schema/util' xsi:schemaLocation='http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-2.5.xsd'>
  <bean id='deploymentReceiverTransportHTTP' class='org.springframework.remoting.httpinvoker.HttpInvokerServiceExporter' p:service-ref='deploymentReceiverTransport'>
    <property name='serviceInterface'>
       <value>org.alfresco.deployment.DeploymentReceiverTransport</value> 
    </property>
  </bean>
  <bean id='urlMapping' class='org.springframework.web.servlet.handler.SimpleUrlHandlerMapping'>
    <property name='mappings'>
      <props>
        <prop key='/deployment'>deploymentReceiverTransportHTTP</prop> 
      </props>
    </property>
  </bean>
</beans>

Here are the contents of the deployment-config.xml file.


<beans xmlns='http://www.springframework.org/schema/beans'
       xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
       xmlns:p='http://www.springframework.org/schema/p'
       xmlns:util='http://www.springframework.org/schema/util'
       xsi:schemaLocation='http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
                           http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-2.5.xsd'>


    <bean id='properties' 
          class='org.springframework.beans.factory.config.PropertyPlaceholderConfigurer'>
        <property name='ignoreUnresolvablePlaceholders'>
            <value>true</value>
        </property>
        <property name='locations'>
            <list>
                <value>classpath:deployment.properties</value>
            </list>
        </property>
    </bean>  
   
   <bean id='configuration' class='org.alfresco.deployment.config.Configuration'
          init-method='init'>
        <property name='dataDirectory'>
            <value>${dep.datadir}</value>
        </property>
        <property name='logDirectory'>
            <value>${dep.logdir}</value>
        </property>
        <property name='metaDataDirectory'>
            <value>${dep.metadatadir}</value>
        </property>
        <property name='targetData'>
            <map>
             <entry key='default'>
                 <map>
                  <entry key='root'><value>c:/FSR/www</value></entry>
                  <entry key='user'><value>admin</value></entry>
                  <entry key='password'><value>admin</value></entry>
                  </map>
             </entry>
      </map>
        </property>
    </bean>
    
    <bean id='deploymentReceiverService' 
     class='org.alfresco.deployment.impl.server.DeploymentReceiverServiceImpl' 
        init-method='init'>
        <property name='configuration'>
            <ref bean='configuration'/>
        </property>
    </bean>
    
    <bean id='deploymentReceiverTransport' 
     class='org.alfresco.deployment.impl.server.DeploymentReceiverTransportImpl' >
        <property name='deploymentReceiverService'>
            <ref bean='deploymentReceiverService'/>
        </property>
    </bean>
    
</beans>

Implementing your own protocol

It's possible to build and impement your own protocol. This will involve providing an implementation of the org.alfresco.deployment.Transport interface and then defining the spring configuration to configure the transport adapters for your new protocol.

Alfresco FSR uses two interfaces. org.alfresco.deployment.DeploymentReceiverService is the service interface used by the repository to deploy files and org.alfresco.deployment.Transport which provides extra methods for dealing with streaming large files across a network.

Fsr transport server side cld.jpg

Debugging the FSR

The FSR can be successfully run in a the Eclipse debugger or can be debugged as a remote process.

Debug as a Java application within Eclipse

Here are the settings to create a 'debug option'
Type: Java Application

Main Tab:

Project : Deployment
Main class : org.alfresco.deployment.Main

Arguments Tab:

Program Arguments: application-context.xml

Classpath Tab:

Classpath : add the deployment project.

Source Tab:

Source : add the deployment project.

Debugging the FSR as a remote process

To debug as a remote process debugging needs to be enabled in the command line which starts the FSR. Then a Java debugger can connect to this remote JVM.

The following Java command line arguments open a debug port 4001 for debugging.


set DEBUG=-Xdebug -Xrunjdwp:transport=dt_socket,address=4001,server=y,suspend=n
set JAVA_OPTS=%DEBUG%

Then in Eclipse, set up a debugging project to debug this remote JVM.

Version History and compatibility

Versions 3.1.X, 3.0.X, 2.9.X, 2.2.X are fully compatible with regard to the FSR. A version 3 Alfresco will be able to use a version 2.2 FSR and vice versa. Content transformations will require a version 3 client and server.

Question: are versions 2.1 and 2.2 compatible, will they interoperate ?

3.2.0

The WCM Deployment Engine replaces the FSR. And the File System Deployment Target deploys to the filesystem.

3.1.0

Added prepare callback.

3.0.0

Added pluggable transport protocols.
Added content transformation
Changed the way that metadata is initialised. The FSR now no longer assumes exclusive control of the deployment target.
Added metadata validation so metadata is validated on FSR start up.
Performance improvement

2.2.1

Adds display group to the deployment screens. And select all functionality to the deployment screens.
Default configuration changed to hard-code RMI service ports rather than using random ports.

2.2.0

Changes to the web project settings for deployment configuration

2.1.6

Adds the deploy_stop script to cleanly shut down a FSR

2.1.3

Performance improvement

2.1.0

First release
Addition of the FSDeploymentRunnable hook. (which becomes known later as postCommit)