FSR deployment of large site failure

[brannonk]

Hi Everyone,

I have been testing the community version 2.9B and am very impressed. I have several sites that I want to manage with the WCM component ( these are pre-existing sites). I was able to import and set up deployment on all of them sucessfully except for one, our main site.
It might be the site size ( about 8-9G worth of files ).
The error I get after it copies what seems to be all the files to the temp data directory is " org.alfresco.deployment.impl.DeploymentException: Failure during prepare phase; rolled back."
which seems to be caused by:
"Caused by: org.alfresco.deployment.impl.DeploymentException: Could not create metadata directory C:\Alfresco\Deployment\depmeta\default.md\\ROOT\testsite\home\roads\images
at org.alfresco.deployment.impl.server.Target.update(Target.java:320)
at org.alfresco.deployment.impl.server.Deployment.update(Deployment.java:297)
at org.alfresco.deployment.impl.server.DeploymentReceiverServiceImpl.commit(DeploymentReceiverServiceImpl.java:195)
… 64 more"

I thought it was a permissions problem ( this is hosted on a Win2003 server ). The permissions on the directories are the same for all the deployment( I have all the deployments going to the same FSR - just running multiple ports/multiple instances ). The base directory is also already created when the FSR is started, so it is not missing.
I tried increasing the memory the runtime could use in the fsr startup bat file - still no change

Is there something else I should be looking at or?

[kvc]

Yes, this was an issue identified back in December via a support case.  It is in both 2.1 Community and 2.9 Community.

The issue is related to the size of your deployment.  The challenge:  Hibernate is deficient in its cache management, never flushing certain Java object types.  It grows without
bounds until it dies.  This is what you are hitting.

We've hit this Hibernate cache problem is nearly every other area of the product dealing with large transaction (bulk import, submit, you name it).   To resolve, we've had to
get deep in the guts of Hibernate to effectively manage Hibernate's cache for itself.

Now, the issue was raised via a Support case in December and fixed roughly a week after that.  That fix is in our 2.1.2 Enterprise release, our upcoming 2.1.3 Enterprise release, and
our recently introduced 2.2.0 Enterprise release (all for customers on subscription).  This fix upon release of 2.2.0 was synced with our public SVN, meaning all members of the
Community can get it today.  Our next official build of main development branch (the Community codeline) will not be until June.   I recommend either waiting until then or getting
access to our enterprise maintenance train.

Now, other good news:  the fix also includes a MASSIVE performance boost for large deployments.  Deployment times now scale linearly with the number of assets; we weren't quite
linear in 2.1 / 2.9.  What this means:  for small deployments, you won't see much difference, but for deployments of this size, it could very well cut times by half or more.  We like
this fix!

Also, if you are doing big FSR deployments, I highly encourage you to read the WCM Deployment wiki page:

http://wiki.alfresco.com/wiki/Deployment

There's a *host* of great stuff we added to 2.2 and plan on rolling into the next Community build.  Do take a look and let us know what you think!


Kevin

[brannonk]

Thank you for the update!
I tried the nightly builds 2.9C_dev and the issue was still there, so I build the war from the svn head and deployed it in the standalone tomcat install.
This looks like it fixed it, but I ran into another stop that I am resolving now.
The FSR did not seem to like foreign language characters in the folder name. ( french character in an english windows system )
The deployment errored out with:
Could not create metadata directory
C:\Alfresco\Deployment\depmeta\default.md\\ROOT\testsite\home\roads\images\fran?ios

The "?" taking the place of the foreign character.

Not a major show stopper for us.

we are looking at the enterprise, just wanting to test it fully before hand.
I am really like the new features; they are something we have been waiting for. ( auto deploy , exclusions, inclusions )