topic Re: Search must performed 2 times to get the complete list in Alfresco Archive

Search must performed 2 times to get the complete list

wmay — Tue, 13 Sep 2011 07:17:01 GMT

I have the problem that i often have to search 2 times to get the complete result list - the first time when i perform the search - only a quick search using a single word which should bring back a longer result list e.g. 200 document hits.When i search the first time i get only a single document as

Re: Search must performed 2 times to get the complete list

mrogers — Tue, 13 Sep 2011 09:08:44 GMT

Yes there is a timeout for the query, it sounds like the first run is finding so much information and ubsequent queries find more.

How are you calling search ?

Re: Search must performed 2 times to get the complete list

wmay — Tue, 13 Sep 2011 09:48:25 GMT

My tests shows that this is a problem which comes up with the normal alfresco explorer and also with share. I used the quick search - all items - e.f. i search for "atom" it takes 20 or 30 sec. to get a result - this result is a single document, then when i perform the same search again i get "all" results - 56 pages in the alfresco explorer - there are for sure more documents with as his but i think there exists a limit and not all documents comes back. Same happens with Share search. The result of the second search comes back within less than a second. From this time on when the first time all results are found - it is fast and with each search using the criteria like before brings back "all" documents - so it seesm that this result or search is now cached. When i do this with an other word - the same happens again.

I have 250.000 documents, all PDF, the information i searched is not in the document content text but only in the filename of the documents. Tested with 3.4d, My-SQL, MS-Windows 2008 Server, 6-8GB RAM, 2-4 cores

Why this can happen - this confuses each user that one search performed 2 times brings back one time one result document and then 56 pages ? without a warning or any other information that there was a time-out or something like this.

Strange is also that when i immediately press search again the complete result comes up correct. Why this can not be with the first try ?

first search result: http://www.may.co.at/webrr/respfile.php?file=MAYComp/Current/RO/first_search.png
second search result: http://www.may.co.at/webrr/respfile.php?file=MAYComp/Current/RO/second_search.png

Re: Search must performed 2 times to get the complete list

mrogers — Tue, 13 Sep 2011 12:40:34 GMT

These are the two properties to change.

system.acl.maxPermissionCheckTimeMillis=10000
system.acl.maxPermissionChecks=1000

Re: Search must performed 2 times to get the complete list

wmay — Tue, 13 Sep 2011 14:11:40 GMT

Are this the correct parameters which should be so so that it works or in which direction they should changed ?
Are these the default settings ?
Any recommendation what would be the optimal values for these parameters - from / to ?

Re: Search must performed 2 times to get the complete list

wmay — Tue, 13 Sep 2011 14:28:55 GMT

In which file i can find them ?
Where is this file located ?

Thank you for your help

Re: Search must performed 2 times to get the complete list

mrogers — Tue, 13 Sep 2011 15:35:47 GMT

Put
system.acl.maxPermissionCheckTimeMillis=30000
in your alfresco-global.properties file.

Restart alfresco.

See if that does the trick.

Re: Search must performed 2 times to get the complete list

wmay — Tue, 13 Sep 2011 15:47:24 GMT

ok, i did it - first search took ~28 sek to get back a result - but the same happens - only one document found - then immediatley after a second search and the full result with 56 pages comes up.

Re: Search must performed 2 times to get the complete list

mrogers — Tue, 13 Sep 2011 15:51:17 GMT

try to increase the maxPermissionChecks and see if that does the trick.

Re: Search must performed 2 times to get the complete list

wmay — Tue, 13 Sep 2011 15:58:28 GMT

i changed it to 40000 and 10000 but allways the same result - no change - after 30 sek. the first result and then when performing a second search all.

Re: Search must performed 2 times to get the complete list

wmay — Tue, 13 Sep 2011 16:13:14 GMT

I retested it with the share search function - "advanced search" - and i searched only for "atom" in the name field.

Same like Alf. Explorer - after 30 sek. a result but only one document - then i did the search again and the next search with "all" documents = 250 documents = 5 pages came up after 15 sec. this is the only difference to the search via Alf Explorer where this 2nd and complete result comes immediately.

Re: Search must performed 2 times to get the complete list

mrogers — Tue, 13 Sep 2011 16:33:12 GMT

What version of 3.2 are you using?

Re: Search must performed 2 times to get the complete list

wmay — Tue, 13 Sep 2011 16:37:32 GMT

CE 3.4 (d3370)

Re: Search must performed 2 times to get the complete list

wmay — Thu, 22 Sep 2011 06:53:12 GMT

Hallo,

We "played around" with this parameter but we did now have the impression that this parameter has an real effect to solve this problem - is ther someone who is able to reproduce this and is this a know bug / problem or is there any chance to solve this problem by some settings / paramters ?

This effect can be reproduced with all the different search functions - node browser / share / Alf explorer…

thank you for your help.

Re: Search must performed 2 times to get the complete list

afaust — Thu, 22 Sep 2011 08:14:37 GMT

Hello,

just a quick question: What is your setting for the following parameter?


system.readpermissions.bulkfetchsize
‍‍‍

This defaults to 1000. It controls the amount of metadata loaded while checking for permissions and is the main contributor to the duration of the permission check. The paramaters Mike has listed control how lenient the permission check is in regards to duration - this parameter directly affects how long the check can take in the wort cases. This parameter is especially important for paged requests and result sets where the user has access to only a limited fraction of the content items. As Alfresco pre-loads content based on the low-level Lucene search, it may load to much content into memory, i.e. content that is not contained in the requested page or the user does not have permissions for anyway.

In my experience, in a properly managed content repository with large volumes of content items and properly differentiated permissions, the observed performance is closer to a worst case than to the best case scenario. The worst case is something like this: you request a result page of 25 items and have the parameter set to the default of 1000 - if the 25 items you actually have access to are located/sorted at the 1st, 1001st, 2001st … position, you may end up loading up to 25000 content items in memory just for the permission check alone.

The problem with permission checking is that there is not guarantee of a certain performance for an arbitrary search - nor is there a narrow corridor performance measurements fall into. This is why I always set the parameter to 0 in installations I set up / manage. This of course is detrimental to retrieval of large page sizes as content is only loaded into memory on demand - but provides a guaranteed linear scalability and near unnoticable variance for common page sizes in user oriented searches.

Regards

Re: Search must performed 2 times to get the complete list

wmay — Thu, 22 Sep 2011 13:05:57 GMT

Ok, thank you very much - this is the solution so that we do not have to perform the search 2 times to get the complete list.

Re: Search must performed 2 times to get the complete list

dhartford — Tue, 29 Nov 2011 16:20:09 GMT

confirmed I ran into a similar issue, thank you for posting this information and your own confirmation.

problem statement: First search for a common value took a long time but only returned one result. The second search for the same value was quick but returned many (the expected) results.

resolution in alfresco-global.properties to have the first search always return all/expected results. Problem and resolution experienced/resolved on Alfresco 3.4.d:

system.acl.maxPermissionCheckTimeMillis=30000
system.acl.maxPermissionChecks=1000
system.readpermissions.bulkfetchsize=0

‍‍‍‍‍

Re: Search must performed 2 times to get the complete list

andy — Fri, 13 Jan 2012 09:57:56 GMT

Hi

These issues are resolved in Alfresco 4.0 using SOLR.

Andy