https://connect.hyland.com/t5/nuxeo-forum/massive-1000-000-docs-import-of-documents-without-acls-with/m-p/319368#M6369 <P><STRONG>Environment</STRONG></P> <UL> <LI>OS: Windows server 2008R2 service pack 1</LI> <LI>Java : JDK 1.7 64 bits</LI> <LI>Nuxeo server: heap memory = 999 Mo</LI> <LI>Nuxeo base <UL> <LI>contents : 100 000 documents</LI> <LI>search full index disabled</LI> <LI>document created without ACLs</LI> </UL> </LI> </UL> <P><STRONG>Scenario</STRONG></P> <P>a) After creation of 20 000 more documents</P> <UL> <LI><P>there is '2014-02-07 15:36:59,241 WARN [org.nuxeo.ecm.core.event.tx.PostCommitSynchronousRunner] PostCommitListeners are too slow'</P> </LI> <LI><P>the speed of the document creation is 12 100 docs/hour while it was 25 000 at the beginning.</P> </LI> </UL> <P>b) Then after creation of 40 000 more documents</P> <UL> <LI>the speed of the document creation is 3500 docs/hour</LI> <LI>there is a "java heap space out of memory"</LI> </UL> <P><STRONG>Questions</STRONG></P> <OL> <LI><P>How to avoid the message 'PostCommitListeners are too slow' and keep a speed at least 20 000 docs/hour to import 1 000 000 docs in a reasonable time ?</P> </LI> <LI><P>How to improve the speed of the document creation knowing that our software can supply Nuxeo with 500 000 docs/hour ?</P> </LI> <LI><P>How to avoid "java heap space out of memory" ?</P> </LI> </OL> <P>Why Nuxeo uses so much memory while</P> <UL> <LI>we only create documents one by one</LI> <LI>and search if document exists before creation ?</LI> </UL> Tue, 18 Feb 2014 14:19:57 GMT jbouche_ 2014-02-18T14:19:57Z https://connect.hyland.com/t5/nuxeo-forum/massive-1000-000-docs-import-of-documents-without-acls-with/m-p/319368#M6369 <P><STRONG>Environment</STRONG></P> <UL> <LI>OS: Windows server 2008R2 service pack 1</LI> <LI>Java : JDK 1.7 64 bits</LI> <LI>Nuxeo server: heap memory = 999 Mo</LI> <LI>Nuxeo base <UL> <LI>contents : 100 000 documents</LI> <LI>search full index disabled</LI> <LI>document created without ACLs</LI> </UL> </LI> </UL> <P><STRONG>Scenario</STRONG></P> <P>a) After creation of 20 000 more documents</P> <UL> <LI><P>there is '2014-02-07 15:36:59,241 WARN [org.nuxeo.ecm.core.event.tx.PostCommitSynchronousRunner] PostCommitListeners are too slow'</P> </LI> <LI><P>the speed of the document creation is 12 100 docs/hour while it was 25 000 at the beginning.</P> </LI> </UL> <P>b) Then after creation of 40 000 more documents</P> <UL> <LI>the speed of the document creation is 3500 docs/hour</LI> <LI>there is a "java heap space out of memory"</LI> </UL> <P><STRONG>Questions</STRONG></P> <OL> <LI><P>How to avoid the message 'PostCommitListeners are too slow' and keep a speed at least 20 000 docs/hour to import 1 000 000 docs in a reasonable time ?</P> </LI> <LI><P>How to improve the speed of the document creation knowing that our software can supply Nuxeo with 500 000 docs/hour ?</P> </LI> <LI><P>How to avoid "java heap space out of memory" ?</P> </LI> </OL> <P>Why Nuxeo uses so much memory while</P> <UL> <LI>we only create documents one by one</LI> <LI>and search if document exists before creation ?</LI> </UL> Tue, 18 Feb 2014 14:19:57 GMT https://connect.hyland.com/t5/nuxeo-forum/massive-1000-000-docs-import-of-documents-without-acls-with/m-p/319368#M6369 jbouche_ 2014-02-18T14:19:57Z https://connect.hyland.com/t5/nuxeo-forum/massive-1000-000-docs-import-of-documents-without-acls-with/m-p/319369#M6370 <P>You might want to take a look at <A href="http://doc.nuxeo.com/display/public/NXDOC/Nuxeo+Bulk+Document+Importer">Nuxeo's bulk importer</A> to achieve your desired result. Or write your own custom importer for even more control over the import process, transactions, etc.</P> Fri, 21 Feb 2014 22:14:29 GMT https://connect.hyland.com/t5/nuxeo-forum/massive-1000-000-docs-import-of-documents-without-acls-with/m-p/319369#M6370 bruce_Grant 2014-02-21T22:14:29Z