cancel
Showing results for 
Search instead for 
Did you mean: 

lazy binaries from cloud providers (s3) are read prematurely

Lorenzol_
Champ on-the-rise
Champ on-the-rise

we're using the cloud plugin to read blobs from s3

we noticed a loads of queries for metadata toward our s3 bucket, for example if we ask the document model for one attachment and load the files schema the cloud provider goes and read the metadata for all attached files

This is forced by the BinaryBlobProvider at method readBlob (line 73) where it asks for the LazyBinary length, making it less lazy than it could be.

We tried reading attachments bot from a document model and from the document.get operation, with the same result.

is there any other operation that fetches a Blob without going through the readBlob calling getLength()?

3 REPLIES 3

Florent_Guillau
World-Class Innovator
World-Class Innovator

Hi,

What Nuxeo version are you using? There have been fixes very recently to improve the way with deal with lazy binaries and the fetching of the length metadata in some cases (NXP-18369).

To answer your last question, at this time the getLength() is always called, but usually this hits the local cache of S3 files so doesn't need to use the network as often as one could fear. There could still be improvements though, tell me if you still see the issue with a version of Nuxeo marked as fixed in the above ticket.

yeah we noticed that the local caches were using up space and recently expanded the server disks for that, we then moved to having the clients fetch the binaries themselves. I'm on 7.3, will try 10 but I'll also try changing the binary provider. It has a reference to the binary so instead of caching the length there I'll just change it to get the length from the binary only when asked.

The preferred way is for you to send us pull requests via github. For patches that add non-trivial functionality and aren't just small fixes we'll ask you to sign a small [contributor agreement](https

Getting started

Find what you came for

We want to make your experience in Hyland Connect as valuable as possible, so we put together some helpful links.