import a lot of files

[mliwski]

Hi,
We are testing alfresco to store scanned images.
To import them into the repository, we generate an ACP file and import it into a folder.

If the ACP file contains more than 7000 files, we have a few problems:
1- some of the files are imported but lose the metadata and the name of file is the UUID. this happened aleatory.
2- the page counter, every time we enter to the folder, shows a different number of pages.
3- the performance of a folder that contain a lot of files is really poor.

Please let us know how we should configurate Alfresco to work with big folders.

THANKSSSS!!

[kevinr]

What version of Alfresco are you using? Hopefully the performance of large folders is a little better in 2.1.

>2- the page counter, every time we enter to the folder, shows a different number of pages.
It's a bit strange yes but it's by design. The idea is to limit the load on the server by having a maximum time limit to wait for the call to get the folder contents to come back from the server, if it takes too long the first N files are shown. Due to various caching effects, the number of files returned tends to grow as you visit the same folder again and again. This can be tuned or disabled via config (see repository.properties)
#
# Properties to limit resources spent on individual searches
#
# The maximum time spent pruning results
system.acl.maxPermissionCheckTimeMillis=10000
# The maximum number of results to perform permission checks against
system.acl.maxPermissionChecks=1000

>3- the performance of a folder that contain a lot of files is really poor.
This may be true, but the web-client UI is not the only way you can deal with the objects. Realistically is a user actually going to step through 100's of pages to find what they want? No - they are going to use Search or categories etc. to find things. You can use templates or custom views to display more sensibly constrained lists of files and present them to your users. Using a deeper folder structure will also help (i.e. less files per folder, but more folders).

Thanks,

Kevin

[mliwski]

Kevin,
Thanks for your quick reply!!

We agree in fact that we have to make more spaces or create a custom view to manage a lot of files.

But we still with the problem when we import an ACP with more than 1000 files some of the files are imported but lose the metadata and the name of file is the UUID (8f085af4-2bfa-11dc-89e8-a73a2f1ea948). this happened aleatory.

There are not any error log about this, and we can see the file, so the unzip procces works fine… the problem, we think, is in the metadata assign.

We are using Alfresco from SVN HEAD, and we alredy check the ACP file and it´s ok.

Best regards
Matias

[carlosvilaboa]

Hi, I am getting the same results when performing a copy of a very structured folder. My folder is not very big (the acp is about 1 megabyte) but it's metadata "intensive".

Sometimes the copy of the folder finishs ok, but sometimes the copy is done missing metadata and for some nodes what I see is the uuid of the node instead of the name. I am using Alfresco 2.0.3

Any help would be appreciated!!