Clustering: does it work in 3.2?

[ofrxnz]

I have been configuring alfresco 3.2 and have been unable to get any cache/index clustering at all to work.  (file replication works)

Has anyone gotten clustering to work on 3.2 and if so any pointers?

i renamed ehcache-custome.xml.sample.cluster to ehcache-custom.xml

i set the alfresco.cluster.name=TestCluster and index.recovery.mode=AUTO flags in the global.properties file

set this flag in the log4j file for debuging
log4j.logger.org.alfresco.enterprise.repo.cache.jgroups=INFO

and i had nothing new in the logs about clustering show up about another node. When i turned several other flags to debug, i kept receiving errors about the localhost not listening on some port in the 30000s. 

i have also verified the ports are open in teh firewall 4446 and 7800 both tcp and udp

since then i have also tried variations on the following flags in the global.properties file

alfresco.jgroups.defaultProtocol=TCP
alfresco.tcp.start_port=7800
alfresco.tcp.initial_hosts=10.8.7.21[7800],10.8.7.22[7800], localhost[7800]

alfresco.udp.mcast_addr=230.0.0.1
alfresco.udp.mcast_port=4446
alfresco.udp.ip_tt=2

thanks for any ideas

Adam

[eleaese]

Hi Adam,

Same here. I followed steps from Jeff Potts, at http://www.optaros.com/blogs/alfresco-31-clustering-easier-with-jgroups and Alfresco Wiki , except for the typo on renaming the file ehcluster-config.xml (it should be ehcache-custom.xml). It didnt work as expected. JGroups is not loaded. I think there's a difference in context files between Enterprise 3.1 and Community 3.2 that avoids JGroups configuration.

Would you kind sharing with me the log4j properties you enable for debugging.

Regards,

Luis

[pondermuse]

Hi,

I am currently trying to setup two Alfresco 3.3 community servers (running on Win XP) into a cluster by following instructions given in URLs such as http://wiki.alfresco.com/wiki/Configuring_JGroups_and_Alfresco_Clusters.

So far I've seen no signs or logs of anything actually being clustered and I am therefore wanting to ask whether the online guides on clustering Alfresco apply to the community edition of the software. Or is clustering only possible with enterprise versions of Alfresco?

Thanks in advance,
PM.

[eleaese]

Hi Pondermuse,

Clustering is possible with Alfresco Community (I actually made it work with the directions from http://wiki.alfresco.com/wiki/Cluster_Configuration_V2.1.3_and_Later).
Regarding JGroups, the following log4j categories may work: http://wiki.alfresco.com/wiki/Cluster_Configuration_V2.1.3_and_Later#Tracking_issues.

Hope those directions help you.

Luis

[pondermuse]

Hi Luis,

Thanks for the quick response. I have followed the directions in http://wiki.alfresco.com/wiki/Cluster_Configuration_V2.1.3_and_Later but without luck.

I describe below what I have done in my attempts to get clustering working…

Setup: 2x Win XP (SP3) servers each with default Alfresco 3.3 community installed under C:\Alfresco. (each server runs its own mysql server as bundled within Alfresco)

First, I edited C:\Alfresco\tomcat\shared\classes\alfresco-global.properties and included the following:

index.recovery.mode=FULL (I've also tried AUTO here…)
alfresco.cluster.name=ABCCluster
alfresco.jgroups.defaultProtocol=TCP
alfresco.tcp.initial_hosts=192.168.1.6[7800],192.168.1.9[7800]

Second, I created a network shared folder C:\alf_shared in 192.168.1.6 to use as a shared content store between the two servers. (This folder is then mapped as a network drive in 192.168.1.9 as Z:\)

Third, on 192.168.1.6 I edited C:\Alfresco\tomcat\shared\classes\alfresco\extension\replicating-content-services-context.xml as follows:

- in <bean id="localDriveContentStore" …> I've edited value with <value>C:\Alfresco\alf_data\contentstore</value>
- in <bean id="networkContentStore" …> I've edited value with <value>C:\alf_shared</value>

Fourth, on 192.168.1.9 I edited C:\Alfresco\tomcat\shared\classes\alfresco\extension\replicating-content-services-context.xml as follows:

- in <bean id="localDriveContentStore" …> I've edited value with <value>C:\Alfresco\alf_data\contentstore</value>
- in <bean id="networkContentStore" …> I've edited value with <value>Z:\</value>

Fifth, I copied C:\Alfresco\tomcat\shared\classes\alfresco\extension\ehcache-custom.xml.sample to the same directory as ehcache-custom.xml.

Sixth, I edited C:\Alfresco\tomcat\webapps\alfresco\WEB-INF\classes\log4j.properties as follows:

log4j.logger.net.sf.ehcache.distribution=DEBUG
log4j.logger.org.alfresco.repo.node.index.IndexTransactionTracker=DEBUG
log4j.logger.org.alfresco.repo.node.index.AVMRemoteSnapshotTracker=DEBUG
log4j.logger.org.alfresco.repo.jgroups=debug
log4j.logger.org.alfresco.enterprise.repo.cache.jgroups=debug

This is all I've done so far…

When I startup the Alfresco servers the console log shows ehcache and node.index. logs but no jgroups logs whatsoever…

Also, the shared content store location C:\alf_shared (Z:\ on the remote server's mapped network drive) remains empty and no data is copied into it…

Thanks again for any help on this.

Regards,
PM.

[eleaese]

Hi Pondermuse,

I'm sharing my notes with you regarding Alf Clustering:

• Servers hostnames should be resolved by DNS/hosts file. Edit C:\windows\system32\drivers\etc\hosts, describe IPs and names for both servers.


• Confirm with testname3.java


• Start one instance to bootstrap the repository, then start the remaining Alfresco server.


• Check table alf_server. It should contain the IP addresses of the cluster members.

About the steps you follow, I recommend you start with a single repository (C: on one server, Z: mapped to the same folder, on the other), to avoid replication. Delete  replicating-content-services-context.xml.
Alfresco #1 dir.root is C:\alfresco\alf_data, Alfresco #2 is 3 is Z:\alfresco\alf_data. Then, each server should have its own directories for indexes. The directives dir.indexes, dir.indexes.backup, dir.indexes.lock control the location. Put them in each alfresco-global.properties.

===============testname3.java============ 
import java.net.InetAddress; 
import java.net.UnknownHostException; 

class testname3 
{ 
    public static void main(String[] args) throws UnknownHostException 
    { 
        String m_srvName = InetAddress.getLocalHost().getHostAddress(); 
        System.out.println(m_srvName); 
    } 
} 
javac testname3.java
java testname3 
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Hope it helps …

[eleaese]

Pondermuse,

Just a quick note. Paths for java apps should use forward slash (/) or escaping the backslash (\\) in Windows. 

Regards, Luis

[pondermuse]

Hi Luis,

I have followed your tips to enable clustering by first getting the two servers sharing the same content store but when I start up the secondary server, I am getting:

INFO The Alfresco root data directory ('dir.root') is: Z:\Alfresco\alf_data
ERROR CONTENT INTEGRITY ERROR: System content not found in content store.

From your suggestions, I've made the master server's C: drive shared and mapped on the secondary server as Z:. On the secondary server, I can happily browse to the master server's content store through Z:\Alfresco\alf_data.

I have also made sure that the servers are now reachable via fqdn too.

What I am still not not too sure about is what the dir.indexes, dir.indexes.backup and dir.indexes.lock values you mentioned are. As a guess, I've created a folder for each: C:\alf_index, C:\alf_index_backup and C:\alf_index_lock (I've created these on both servers. i.e. locally on each). So, my C:\Alfresco\tomcat\shared\classes\alfresco-global.properties files now contain the following sections:

On primary master server -
dir.root=C:\\Alfresco\\alf_data
dir.indexes=C:\\alf_index
dir.indexes.backup=C:\\alf_index_backup
dir.indexes.lock=C:\\alf_index_lock

On secondary slave server -
dir.root=Z:\\Alfresco\\alf_data
dir.indexes=C:\\alf_index
dir.indexes.backup=C:\\alf_index_backup
dir.indexes.lock=C:\\alf_index_lock

Are these dir.indexes assumptions ok or am I way off the mark here?

Thanks again for your help,
PM.

[eleaese]

Hi Pondermuse,

You need to rebuild the indexes on both nodes. As the documentation about clustering states, each Alfresco node should have its own index files.
Edit alfresco-global.properties and include the following line (replace or comment the current value if it's different):

index.recovery.mode=FULL‍

Then, restart each node an watch alfresco.log messages showing the indexes are rebuilt.

Also check the table alf_server on the Alfresco's database. It should contain the IP addresses of the cluster members. If not, please include them both.

Regards,

Luis

[pondermuse]

Hi Luis,

The 'CONTENT INTEGRITY ERROR: System content not found in content store' error log still shows up when starting up the secondary Alfresco server.

I had a look at the alf_server table on each server's db and they had each only their own ip_address entry listed. So I inserted the other server's ip_address on each server's alf_server table. Re-started both Alfresco servers (master first) but secondary server still shows the error on startup…

The files C:\Alfresco\tomcat\shared\classes\alfresco-global.properties on each server contain the line index.recovery.mode=FULL and both server's logs show their indexes being re-built (i.e. INFO [FullIndexRecoveryComponent] Index recovery completed and INFO [AVMFullIndexRecoveryComponent] Finished rebuilding indexes for AVM stores).

Points to note:
- The current setup marks the master server's content store as being shared. The master server's content store and master server's db contained data content prior to the content store being shared with the secondary server. Even though both server's are now marked to share the same content store, each server still runs its own mysql db. Is this ok? Could the problem maybe lie in the two server's dbs being out of sync?

Thanks again,
PM.