Overview
One of the most critical and time consuming process in relation to the Alfresco Content Connector for Salesforce module is migrating all the existing Notes & Attachments content from Salesforce instance to the On-Premise Alfresco Content Services (ACS) instance. This requires lot of planning and thought as to how this use case can be implemented and how the content migration process can be completed successfully. When the end users start using the connector widget in Salesforce they will be able to upload new documents and make changes to existing documents etc within Salesforce but the actual content is stored in an ACS instance. Some of the core components/steps involved in this process are
- Number of documents/attachments that need to be migrated to Alfresco
- Salesforce API limits available per 24 hour period. Use SOQL query or ACC API functions to retrieve the content. We need to make sure the available API calls are not fully used by the migration activity.
- Naming conventions - some of the special characters used in SFDC's filename may not be supported in Alfresco, so some consideration must be given to manipulate the filenames in Alfresco when the documents are saved.
- Perform a full content migration activity in a Salesforce DEV/UAT environment that has a complete copy of production data. This will assist in determining issues such as naming conventions, unsupported documents/document types etc.
- Custom Alfresco WebScripts/Scheduled Actions to import all content from SFDC to Alfresco
- Make sure there are enough system resources in the Alfresco server to hold all the content in the repository. This includes disk capacity, heap/memory allocated to JVM, CPU etc.
We will look in to all these core components in more detail later in this blog post.
Why Content Migration?
Why would we want to migrate all the existing content from SFDC to Alfresco? of course there are many reasons why this migration activity is required/mandatory for all the businesses/end users who will use this module.
- End users need to be able to access/find the legacy or existing documents and contracts in the same way as they normally do.
- Moving content over to ACS instance will save plenty of money for the businesses since the Salesforce storage costs are really expensive.
- Easier access to all documents/notes using the ACC search feature available in the widget.
- Content can be accessed in both Salesforce and Alfresco depending on the end user's preference. Some custom Smart Folders can be configured in ACS to get a holistic view of the contents that the end user require. For example, end users would want to only see the content related to their region. In which case a Smart folder can be configured to query for a Salesforce object based on a particular metadata property value, such as sobjectAccountRegion, sobjectAccount:Regional_Team etc.
Steps Involved
- Determine the total number of Notes & Attachments that must be migrated/imported to ACS - This can be determined by using Salesforce Developer Workbench. Use a SOQL query to get a list of all Notes/Attachments of a specific Salesforce object. For example Accounts, Opportunities etc.
You may also use the RESTExplorer feature in the workbench to execute this query.
Some sample SOQL queries are
To retrieve attachments of all Opportunity objects in SFDC - Select Name, Id, ParentId from Attachment where ParentId IN (Select Id from Opportunity)To retrieve Notes of all Account objects in SFDC - Select Id,(Select Title, Id, ParentId, Body from Notes) from Account
2. Develop Alfresco Webscripts/Aysychronous action to retrieve and migrate all content from SFDC to Alfresco - It is probably a good idea to develop an Alfresco Asynchronous action (Java based) as opposed to a Webscript to perform the migration process. This is to ensure the actual migration runs in the background and there are no timeouts or anything as such we may see using the normal webscipts. Based on the amount of content it take can take few hours for the action to complete successfully. We will use the ACC module's available API's to make a connection to Salesforce and then use the relevant functions to retrieve the content. The code snippet to make a connection and execute a SOQL query are below. To get the content of an Attachment object in SFDC use the getBlob method. It should be something like below,
apiResult = connection.getApi().sObjectsOperations(getInstanceUrl()).getBlob("Attachment", documentId, "Body");
Once the connection and query are established you can then look to save the file in Alfresco based on the end-user/business needs. The code snippet to save the document in alfresco is below.
3. “Warm up” of Alfresco folder structures without human intervention - One of the prerequsite to the content migration process is pre-creating all the SFDC objects folder structure within ACS and make sure the folders are mapped to the appropriate SFDC object. This can be achieved by exporting all the SFDC objects details in to CSV file. the CSV file must contain information such as SFDC Record Id, AccountId, Opportunity Id, Region etc. The SFDC Record ID is unique for each and every Salesforce object and the Content Connector widget identifies the mapping between ACS and SFDC using this ID. Before executing the content migration code, we would need to make sure all the objects exist in ACS first. Once CSV file is generated, then develop a custom ACS web script to process the CSV file line by line and create the appropriate folder structures and assign metadata values accordingly. Once ready execute the web script to auto create all folder structures in ACS. A simple code snippet is below.
4. Trigger the Migration code to import all content from SFDC to ACS - Once all the folder hierarchy for the required SFDC objects are setup in ACS, you may now execute the Asynchronous action developed in Step 2. To execute the Java Asynchronous action you may create a simple web script which executes this action. A simple code snippet is below.
var document = companyhome.childByNamePath("trigger.txt"); var sfdcAction = actions.create("get-sfdcopps-attach"); sfdcAction.executeAsynchronously(document); model.result=document;You may choose to execute this in multiple transactions instead of a single transaction. If there are tens and thousands of documents that need to be imported then the best approach is to have multiple transaction at the migration code level.
Once the migration process is complete, the documents must be located in the relevant folder hierarchy within ACS and its associated object page in SFDC.
5. Validate the content - Once the document import process is complete, make sure to test & validate that all the documents are imported to the appropriate folder hierarchy in ACS and also it is accessible within the Salesforce Content Connector widget. Key things to check are the thumbnails, preview, file names, file content etc.
Hope all this information helps in your next Alfresco Content Services content migration activity.
