Poor performance during document deletion

[fschnell]

A colleague of mine (who also wrote this article) did some trials on our out-of-the-box Alfresco 2.0 community with MySQL on OpenSuse to see how long it would take to copy and paste a complex file structure from drive to drive. The file structure chosen consisted of 296 files within 29 folders, a total of 24.3 MB.

The transfer was done between
  😧 - a local drive on my desktop machine
  X: - a conventional shared drive on the local area network
  V: - an Alfresco/CIFS 'virtual drive' on the local area network

Most transfers were repeated four times and averaged, in order to smooth out the peaks and troughs of network performance. Each transfer was done by copy and paste to make sure that data was really being transferred, even where it wasn't necessary.

   from D:   from X:   from V:
to D:   3 s   7 s   25 s
to X:   16 s   16 s   41 s
to V:   116 s   134 s   147 s
delete   3 s   3 s   1505 s
‍‍‍‍‍‍

Transfers to and from the V: drive were slower than with a conventional file system but this is generally acceptable, considering that these are transfers involving a Document Management system and the number of files involved. However, the time taken to delete from the V: drive, over 25 minutes(!), was totally unacceptable. Windows would continually report how much time was left but generally underestimated by a factor of about four. Most of the time, the system didn't seem to be doing anything at all and I think that that most users would conclude that the system had hung.

He repeated the same transfers but using a much simpler file structure of only 6 large files in 1 folder, but with a similar overall size of 24.0 MB. This was to illustrate that the most important factor in transferring file structures is the number of files, rather than the overall file size.

   from D:   from X:   from V:
to D:   <1 s   3 s   4 s
to X:   4 s   6 s   6 s
to V:   8 s   7 s   12 s
delete   <1 s   <1 s   32 s
‍‍‍‍‍‍

This time the performance of the V: drive is much closer to that of a conventional drive and we don't think that most users would notice a difference most of the time. The exception again is deletion, which is still unacceptably long at about 5 seconds per file.

A possible work-around for the deletion problem might be to cut and paste, or drag-and-drop file structures to a dedicated 'deletion' folder on the V: drive. The files can then be deleted from there in the background. This isn't a pretty solution and not one that you can expect normal users to want to use.

Question: Why does deletion take so long? Deleting the file in the filestore and its reference in the database should be fast.

Tx
Frank

[derek]

Hi,

Files and folders are, by default, archived.  So the delete is not only moving the nodes, but is ensuring that all association to and from the nodes are archived as well.  This is an expensive process, but can be switched off in the model.

If it is off, then the process still has to walk the tree to delete the nodes.  The reason why we don't do this in the background is that it is possible that the delete actually violates some integrity constraint and should therefore be disallowed.  Each node in the tree has to be evaluated to against the model constraints.

There are solutions to these problems, ofcourse.  We could preserve associations to deleted nodes and avoid displaying them in the clients, for instance.  This would allow us to just move and archive the top-level node.  Complete deletion can do something similar.

Regards

[derek]

Could you repeat the same tests using the latest code on from our SVN?  With such a long-running transaction, you'll need the support to keep the memory sizes in check.

Regards