Aspect (or property) with Geometry, Geo-Spatial Data type.

[steve_tekell]

Is it feasible to add geo-spatial information into Alfresco (or any JCR)?
I would like to associate content items with spatial data - points, polygons, etc.  Something stored in a spatial (geometry) typed column, so that it can be properly indexed and queried.  There would then be some UI (google maps, ESRI+Dojo) to input geometry for an item and query for items spatially as well.

Currently a custom model supports types like decimal, date, text, etc, but nothing for geo-spatial data.

How would you add a spatial data type to a custom model in Alfresco?
What plans does Alfresco have to support geo-spatial information?

At this point it doesn't look feasible to do this within Alfresco or any JCR.

BTW There is Hibernate Spatial to add Hibernate user types for geometry data type.  However, MySQL InnoDB (Alfresco's default) doesn't support spatial indexes.  PostgreSQL (w/PostGIS) and Oracle are better choices for this.

[stevereiner]

Although not the more general geo-spatial data type support you mentioned,  there is some geo-location support in the Calais Integration Alfresco server extension (and auto geo-tagging/search UI in FlexSpaces including a google map).  If geo-location info is available from Calais, it will be stored in an aspect on the semantic tags (categories) that can be automatically associated with docs based on content. 

http://forums.alfresco.com/en/viewtopic.php?f=32&t=15766

http://integratedsemantics.org/2008/12/08/calais-integration-for-alfresco-geo-tagging-flexspaces-par...

[steve_tekell]

It looks like you are storing lat/lon as decimals (an existing data type) and there's no way to store anything other than a point.

So am I correct to think it's not feasible to add a new datatype with a corresponding sql type into Alfresco (or any JCR)?

Geometry types in the database is a requirement for me.  I need more than point data, and I need to be able to efficiently query for something like 'all publications within or intersecting a given polygon between given dates with certain authors'.  So while I see how I could build the UI creating/viewing geo-spatial data, there's no way extend the model in Alfresco to store it properly.

Although not the more general geo-spatial data type support you mentioned,  there is some geo-location support in the Calais Integration Alfresco server extension (and auto geo-tagging/search UI in FlexSpaces including a google map).  If geo-location info is available from Calais, it will be stored in an aspect on the semantic tags (categories) that can be automatically associated with docs based on content. 

http://forums.alfresco.com/en/viewtopic.php?f=32&t=15766

http://integratedsemantics.org/2008/12/08/calais-integration-for-alfresco-geo-tagging-flexspaces-par...

[mrogers]

To answer your question yes it is feasible to add geo-spatial types.    However as you have no doubt concluded the current version of Alfresco does not support these types and there would be issues with database support and the query engine. 

This would make an interesting project for an extension to Alfresco.   

How many people would make use geo-spatial searching if Alfresco supported it?

[steve_tekell]

I am little confused by your reply.

When I say "feasible" I didn't mean can Alfresco be changed to support geo-spatial types in a few years with Alfresco 4 or 5.
I mean can I, through existing extensions points in Alfresco 3, add support for sql geometry types.
For that matter, can I add support for any new data types?

Can this really be accomplished just as an extension?
I don't want to manage a fork or Alfresco to do this.

As to how many people will want to query spatial information in Alfresco, I dunno, but the examples I am familiar with are in science and government.  Here there is a very significant unfulfilled need.  The question is not whether or not will we will do this, but how - with or without Alfresco.

Now, forget querying for a moment, just having the ability to store spatial data in standard sql geometry types like any other properties in a aspect would be a good start.

To answer your question yes it is feasible to add geo-spatial types.    However as you have no doubt concluded the current version of Alfresco does not support these types and there would be issues with database support and the query engine. 

This would make an interesting project for an extension to Alfresco.   

How many people would make use geo-spatial searching if Alfresco supported it?

[mbailen]

Hi,

Has anybody made any movement on this topic (integrating PostGIS geo-spatial data with Alfresco)?

If no other solution is available, we are considering storing the geo-spatial data in a table outside of Alfresco with ID's pointing back to the documents.  I was hoping for something more elegant.

Thanks, Mark

[bnordgren]

Also interested in geospatial aspects/properties.

In my organization (US Forest Service research), some people are just starting to ask for it, essentially implementing it in their legacy custom built document indexer (it's not really a content management system).  However, I suspect its one of those things that would catch on like wildfire if it was available, and it seems to be something that no one else offers.  Querying by the area referenced in the document: not possible if you're not the author or if you happen to use two different keywords for the same location.

+1 on the hibernate spatial, and retaining database independence.

[bnordgren]

Note: I just located a Lucene spatial contrib too.  After poking around a bit in Alfresco's development website, it seems that if one wants to search for documents using any parameter, Lucene must be the thing doing the searching, right?

The downside of the Lucene extension is that it appears to be based on GeoHashing, which is essentially an algorithm for converting lat/lon points into a text representation.  It cannot handle anything other than points.  Further, I'm not clear on whether the hash can be directly used to determine proximity (e.g., does the document refer to a point which is within "x" distance of my search point P)?  I'd think you'd have to convert ALL of the hashes back to coordinates, then do the calculation, which would seem to defeat the purpose of using a hash in the first place.  Also, the spatial contrib seems to implement their own distance calculation and map projection algorithms instead of using a library for that.

Spatially aware databases usually maintain their own spatial indices for use with queries.  If the metadata is stored in PostGIS, I see no advantage in using a Lucene spatial index. 

So I guess the questions relevant to this forum are:

• What is the component which needs to understand spatial objects if Alfresco is to implement spatial queries (Lucene?  Hibernate?  both?)


• Can we just let the spatial database do the spatial query and somehow merge it with the results of the Lucene search for the other properties?


• Being unaware of how Lucene works, am I to understand that it copies the properties out of the RDBMS containing the metadata, then creates indices and executes searches against its own copy?  What guarantees that Lucene is in sync with the RDBMS?


• Are there any recommendations as to how someone interested in this topic might proceed in the most expeditious manner possible, avoiding common new-developer pitfalls? (Note: no promises)

[mbailen]

Glad to see some more interest in this topic.  I am dealing with similar issues and if we stick with Alfresco, we will need to implement a solution by summer of 2010.  Until it becomes a priority for the Alfresco team, I don't see them adding spatial types to their architecture (whether it uses Lucene or not).

We have white-boarded a solution, but will likely not attempt it until the spring.  Our initial plan was to avoid mucking about with Alfresco code.  We intended on using an aspect (string) to hold the WKT of all geo-spatial entries in the DMS.  These entries (document ID's and WKT) would then be dumped into an external PostGIS table (daily? to avoiding syncing issues).  And then our searches would become two-phased.  First narrow down the search through Alfresco (by keywords or other criteria) and then pass the document id's and WKT's to an external java/hibernate class for the second phase.  The second phase would be the geo-spatial search and it would hit the daily-generated table (and likely be pretty expensive).  The plan seemed simple enough on the white board, eh?

One issue we immediately spotted was the scenario where a huge number of documents are returned by Alfresco (perhaps zero non-spatial search terms were provided).  Running every Alfresco Document ID through its own personal hibernate query seemed just silly.  One optimization we were considering was to make Alfresco a little more geo-savvy by using the WKT to compute quadrants (a misnomer used in our mapping group…  perhaps *sections* is a better word?).  These quadrants could be a singular aspect, computed on ingestion.  Using these quadrants, it may be possible to narrow down search results by a 100 or 1000, etc., depending on the schema.  It adds some overhead, but may be worth the time.  Also, it may depend a lot on your data.

Anyway, the solution above seems spectacularly bad when compared to a straight PostGIS query, but we haven't though of an easier way to accomplish this without altering the Alfresco code base.  Let me know how you progress.  Or if you have better ideas.  We likely won't know more until we are elbow-deep into it. . .

[bnordgren]

The more I think about this, the more important a proper declaration of "scope" becomes to me.  Geospatial operations, while not rocket science, are very tedious and easy to implement incorrectly.  There is a steep learning curve for geospatial newbies, which only gets steeper when the various geospatial metadata standards start coming into play. 

Take a look at GeoNetwork, particularly SpatialFilter and its children.  If I read this correctly, this heirerchy should serve as an adapter layer between geospatial filters and the Lucene search engine.  Geospatial filters are defined in the GeoTools library, which in turn leverages the topology algorithms implemented in the Java Topology Suite (JTS).  JTS is, of course, the Java "original" of the C++ port more commonly known as the Geometry Engine Open Source (GEOS)  library, which is in turn the code which PostGIS uses to do the heavy lifting.

If we want integrated, unified searches of geospatial and nongeospatial characteristics within Alfresco quickly, I'd suggest carving out the family of adapter classes listed above into a standalone Lucene module.  Of course, you still have to brew up a geospatial UI for searches and possibly result display.

Another, less invasive, alternative is to delegate all searches or maybe just geospatial searches to GeoNetwork.  Under this scenario, Alfresco exposes all of its geospatial metadata via one of the mutually supported mechanisms (i.e., WebDAV) to GeoNetwork.  GeoNetwork could then perform searches on Alfresco content.  The site could be configured to allow users direct access to the GeoNetwork web UI, or it could hide the UI and expose search results via Alfresco's interface to OpenSearch.  Within Alfresco, "geospatial metadata", in the XML format comprehensible to GeoNetwork, could be expressed as its own document type in the content model.  A child association could be formed between "something in the repository" and the new geospatial metadata type.  Alfresco's main task would then be to aggregate all the geospatial metadata it manages, ensure that "links to the actual content" are correct, inject whatever non-geospatial metadata it can from the associated object, and provide access to the file.  Alfresco can search everything that's not geospatial, and GeoNetwork can possibly search everything.  We can come up with an XML metadata template for a lat/lon point and/or a box.  Maybe an importer based on GDAL which converts shapefiles to the XML format for more complex shapes.

Is there any possibility of using a portal to hide the fact that two different web apps are being used?  I don't know much about portals.

The downside is that GeoNetwork appears to be written in French and I'm a spoiled English speaker.  Installation guides and such are English friendly, but I'm not sure I want to look too closely at the code.

Does this sound rational? I'm not an Alfresco user yet, so I don't have a good feel for what it can and can't do.  My perceptions of ECM in general and Alfresco in particular may suffer from the sales pitch where it is quite capable of washing my car, feeding the dogs, and building a spaceship all in one.  Think its realistic or pie-in-the-sky?