https://connect.hyland.com/t5/nuxeo-forum/are-instructions-scripts-available-for-running-the-funkload/m-p/328098#M15099 <P>Hi,</P> <P>Yes import was done using the nuxeo-platform-importer with file of 100k, you can find some more information on this report <A href="http://public.dev.nuxeo.com/%7Eben/bench-10m-tomcat/">Navigation on large-scale storage</A>.</P> <P>The database was tuned for doing <A href="http://doc.nuxeo.com/display/ADMINDOC/Configuring+PostgreSQL#ConfiguringPostgreSQL-Massimportspecifictuning">mass import</A> and the import was done around 50 docs/s.</P> <P>Using SQL injection you can do that much faster ~2000 docs/s, you can find some example of python script in my <A href="https://gist.github.com/bdelbosc">github gist</A>.</P> <P>Also all those numbers refer to old hardware (2006) and software, I bet that with PG 9.2 using SSD drive you can get at least one order of magnitude faster.</P> Fri, 28 Jun 2013 11:05:14 GMT ben_ 2013-06-28T11:05:14Z https://connect.hyland.com/t5/nuxeo-forum/are-instructions-scripts-available-for-running-the-funkload/m-p/328097#M15098 <P>Are instructions/scripts available for running <A href="http://public.dev.nuxeo.com/%7Eben/bench-navigation-3m/">Nuxeo Navigation on large-scale storage Benchmark</A>? How were the Funkload scripts configured? Also, was nuxeo-platform-importer used to load the data or was it loaded via SQL scripts? If nuxeo-platform-importer was used, how was it configured?</P> Tue, 25 Jun 2013 20:14:38 GMT https://connect.hyland.com/t5/nuxeo-forum/are-instructions-scripts-available-for-running-the-funkload/m-p/328097#M15098 rg1_ 2013-06-25T20:14:38Z https://connect.hyland.com/t5/nuxeo-forum/are-instructions-scripts-available-for-running-the-funkload/m-p/328098#M15099 <P>Hi,</P> <P>Yes import was done using the nuxeo-platform-importer with file of 100k, you can find some more information on this report <A href="http://public.dev.nuxeo.com/%7Eben/bench-10m-tomcat/">Navigation on large-scale storage</A>.</P> <P>The database was tuned for doing <A href="http://doc.nuxeo.com/display/ADMINDOC/Configuring+PostgreSQL#ConfiguringPostgreSQL-Massimportspecifictuning">mass import</A> and the import was done around 50 docs/s.</P> <P>Using SQL injection you can do that much faster ~2000 docs/s, you can find some example of python script in my <A href="https://gist.github.com/bdelbosc">github gist</A>.</P> <P>Also all those numbers refer to old hardware (2006) and software, I bet that with PG 9.2 using SSD drive you can get at least one order of magnitude faster.</P> Fri, 28 Jun 2013 11:05:14 GMT https://connect.hyland.com/t5/nuxeo-forum/are-instructions-scripts-available-for-running-the-funkload/m-p/328098#M15099 ben_ 2013-06-28T11:05:14Z https://connect.hyland.com/t5/nuxeo-forum/are-instructions-scripts-available-for-running-the-funkload/m-p/328099#M15100 <P>Specifically, how was the nuxeo-platform-importer configured/driven to accomplish the mass importation of 3m/10m documents?</P> Tue, 02 Jul 2013 11:55:52 GMT https://connect.hyland.com/t5/nuxeo-forum/are-instructions-scripts-available-for-running-the-funkload/m-p/328099#M15100 rg1_ 2013-07-02T11:55:52Z