Hyland Connect

simon · ‎04-10-2006

Hi Alfresco,

We are trying to enable people (and group) synchronisation with LDAP. It's not working as expected, but you have probably guessed that by now. First we woul like to enable people synchronisation, group support will follow later but lets keep it as simple as possible for now.

The configuration is based on the documentation we found in the WIKI.

1. The Alfresco system is not available when all the configuration references for group synchro are commented out. There are no errors in the log and the even the login page is not available. This problem is solved when I uncomment the group configuration parameters, strange…

2. Changed the ldap-authentication-context.xml file (the other sections in this file are left alone) like this:

<bean id="ldapPeopleExportSource" class="org.alfresco.repo.security.authentication.ldap.LDAPPersonExportSource">
        <property name="personQuery">
            <value>(objectclass=Person)</value>
        </property>
        <property name="searchBase">
            <value>dc=company,dc=be</value>
        </property>
        <property name="userIdAttributeName">
            <value>cn</value>
        </property>
         <property name="LDAPInitialDirContextFactory">
            <ref bean="ldapInitialDirContextFactory" />
        </property>
        <property name="personService">
            <ref bean="personService"></ref>
        </property>
        <property name="namespaceService">
            <ref bean="namespaceService" />
        </property>
        <property name="defaultHomeFolder">
            <value>/app:company_home</value>
        </property>
        <property name="attributeMapping">
            <map>
                <entry key="cm:userName">
                    <value>cn</value>
                </entry>
                <entry key="cm:firstName">
                    <value>givenName</value>
                </entry>
                <entry key="cm:lastName">
                    <value>sn</value>
                </entry>
                <entry key="cm:email">
                    <value>mail</value>
                </entry>
                <entry key="cm:organizationId">
                    <value>cn</value>
                </entry>
            </map>
        </property>
    </bean>‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

3. Uncommented the relevant part in the scheduled-jobs-context.xml. Remember the "leave the group support for now":

<ref bean="ldapPeopleTrigger" />‍

4. Reboot the server and wait (startDelay parameter does its job).

5. Job starts running and outputs the following error:

15:42:30,371 ERROR [org.quartz.core.JobRunShell] Job DEFAULT.ldapPeopleJobDetail threw an unhandled Exception:
org.alfresco.repo.importer.ExportSourceImporterException: Failed to import
        at org.alfresco.repo.importer.ExportSourceImporter.doImport(ExportSourceImporter.java:165)
        at org.alfresco.repo.importer.ImporterJob.execute(ImporterJob.java:36)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:191)
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:516)
Caused by: java.lang.NullPointerException
        at org.alfresco.repo.security.authentication.ldap.LDAPPersonExportSource.generateExport(Unknown Source)
        at org.alfresco.repo.importer.ExportSourceImporter.doImport(ExportSourceImporter.java:149)
        … 3 more
15:42:30,371 ERROR [org.quartz.core.ErrorLogger] Job (DEFAULT.ldapPeopleJobDetail threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: org.alfresco.repo.importer.ExportSourceImporterException: Failed to import]
        at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:516)
* Nested Exception (Underlying Cause) —————
org.alfresco.repo.importer.ExportSourceImporterException: Failed to import
        at org.alfresco.repo.importer.ExportSourceImporter.doImport(ExportSourceImporter.java:165)
        at org.alfresco.repo.importer.ImporterJob.execute(ImporterJob.java:36)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:191)
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:516)
Caused by: java.lang.NullPointerException
        at org.alfresco.repo.security.authentication.ldap.LDAPPersonExportSource.generateExport(Unknown Source)
        at org.alfresco.repo.importer.ExportSourceImporter.doImport(ExportSourceImporter.java:149)
        … 3 more‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

We traced the connections between the Alfresco server and Active Directory with tcpdump and could see "some" activity so there is "a" connection. Don't know what's transfered and if this is relevant or not.

Could the NullPointerException have anything to do with these settings in the ldap-authentication-context.xml file?

<property name="storeRef">
<value>${alfresco_user_store.store}</value>
</property>
<property name="path">
<value>/${alfresco_user_store.system_container.childname}/${alfresco_user_store.authorities_container.childname}</value>
</property>
‍‍‍‍‍‍‍

Anyone experienced the same problems? We use the Enterprise 1.2 version on Linux.

Thanks!

andy · ‎04-12-2006

Hi Simon

There should be no issue pullling users and groups out of AD.
In effect a group is the same as in OpenLDAP - an object with a repeating attribute that contains the DN to other groups or users.

The userIdAttributeName in group import should match that used in the import of People. So that is fine, except you have two sorts of people. It does not matter that users and groups use the same attribute for the primary key, so long as all users and groups have their primary key.

It is possible there is trouble with groups defined across the two ldap servers. The group and people types will be different in each. If referral is working for queries you may have got into trouble. You need a common supertype, but I have not looked to see what that would be. To get going I would suggest groups in the two ldap servers should be self contained, containing only groups and users from the same ldap server. Define two jobs to import groups and match these with the teo jobs to import people. I had not thought of your scenario where there is potential cross membership of groups and people…

AD uses "user" as opposed to person. I can not recall if a user is a person. The sAMAccountName on user could be used as the uid in AD.

What are people going to type in at the login screen?
CN of inetOrgPerson/user or Full DN or (sAMAccountName/uid)

Is your error exactly the same?
If so, you would seem to be having some error importing people again - someone who does not have the attribute identified by the userIdAttributeName key. May be you can exclude such entries in the query.

You may be finding groups that do not have a cn attribute set?

Hope one of the above helps!

Regards

Andy

simon · ‎04-13-2006

It is possible there is trouble with groups defined across the two LDAP servers.

Indeed, this could be an issue but there is no connection to the second LDAP for the moment (we'll probably need cross membership of groups later on). We would like to fix group support with AD first. I removed the configuration settings to our OpenLDAP to keep it "simple". The AD groups contain "AD only" people so there shouldn't be any cross LDAP conflicts. Some of these groups contain people that aren't/shouldn't be imported in Alfresco, could this cause an error?

AD uses "user" as opposed to person.

All our users are Person objects, even in AD so this shouldn't be the problem, should it?

What are people going to type in at the login screen?

AD people will use the CN in the login screen but I don't see how this is related to the group import?

Is your error exactly the same?

Yes, the error is exactly the same and no, importing people again is not a problem, this still works. The error (Job DEFAULT.ldapGroupJobDetail threw an unhandled Exception) is quite general so it could be something else.

You may be finding groups that do not have a CN attribute set?

I'm afraid not, all our groups have a CN attribute set so again… this should not cause the error.

We tried to locate the problem yesterday and it could have something to do with the size limit of LDAP requests. Our request limit is 1000 for AD. We checked the people import and indeed, the last imported person is number 1000 (like half the entries that should be imported). We are now trying to solve this restriction to import the other people as well. This restriction doesn't throw an error, maybe you could add this as an extra test (seems important and one of those hard to find problems).

The same restriction applies for our group import but no single group is imported (as opposed to the people import). Some of our groups contain all our people (so again > 1000) and we have > 3000 groups at the moment. Could this have something to do with the "I don't import a single group but import the first 1000 people entries" problem?

For what it's worth: thanks again Andy!

andy · ‎04-13-2006

Hi Simon

OK.

Can you send/post the stack trace for the group import error?
Is anything reported with debug on?

The limit should just mean you get an incomplete set. I had the same issue with OpenLDAP which limits to 500 by default. There may also be time limits on queries. If it takes too long it may give up and report nothing. I would check this. This could explain no groups.

Check the queries/results/timings using an LDAP client (e.g softerra)

Person will be fine.

There is nothing else I can think of ….

Regards

Andy

simon · ‎04-13-2006

This is the alfresco.log for a Alfresco startup with people and group synchronisation against AD (no multiple LDAP's , or multiple jobs) with the logging for org.alfresco.repo.security.authentication.ldap set to DEBUG:

1:04:35,068 ERROR [org.alfresco.repo.content.transform.magick.AbstractImageMagickContentTransformer] ImageMagickContentTransformer not available: Failed to execute command: imconvert /opt/alfresco/tomcat/temp/Alfresco/ImageMagickContentTransformer_init_source_52346.gif  /opt/alfresco/tomcat/temp/Alfresco/ImageMagickContentTransformer_init_target_52347.png
11:04:36,268 INFO  [org.alfresco.repo.admin.patch.PatchExecuter] Checking for patches to apply …
11:04:36,318 INFO  [org.alfresco.repo.admin.patch.PatchExecuter] No patches were required.
11:04:36,338 ERROR [org.alfresco.smb.protocol] Failed to get local domain/workgroup name, using default of WORKGROUP
11:04:36,338 ERROR [org.alfresco.smb.protocol] (This may be due to firewall settings or incorrect <broadcast> setting)
11:04:36,348 ERROR [org.alfresco.smb.protocol] File server configuration error, Wrong authentication setup for alfresco authenticator
org.alfresco.error.AlfrescoRuntimeException: Wrong authentication setup for alfresco authenticator
   at org.alfresco.filesys.server.config.ServerConfiguration.processSecurityConfig(ServerConfiguration.java:1570)
   at org.alfresco.filesys.server.config.ServerConfiguration.init(ServerConfiguration.java:492)
   at org.alfresco.filesys.server.config.ServerConfiguration.onApplicationEvent(ServerConfiguration.java:423)
   at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:45)
   at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:225)
   at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:323)
   at org.springframework.web.context.support.AbstractRefreshableWebApplicationContext.refresh(AbstractRefreshableWebApplicationContext.java:134)
   at org.springframework.web.context.ContextLoader.createWebApplicationContext(ContextLoader.java:246)
   at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:184)
   at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:49)
   at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3692)
   at org.apache.catalina.core.StandardContext.start(StandardContext.java:4127)
   at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:759)
   at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:739)
   at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:524)
   at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:804)
   at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:693)
   at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:472)
   at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1118)
   at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:310)
   at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
   at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1020)
   at org.apache.catalina.core.StandardHost.start(StandardHost.java:718)
   at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1012)
   at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:442)
   at org.apache.catalina.core.StandardService.start(StandardService.java:450)
   at org.apache.catalina.core.StandardServer.start(StandardServer.java:680)
   at org.apache.catalina.startup.Catalina.start(Catalina.java:536)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
   at java.lang.reflect.Method.invoke(Method.java:585)
   at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:275)
   at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
11:04:36,378 ERROR [org.alfresco.ftp.protocol] FTP Socket error
java.net.BindException: Address already in use
   at java.net.PlainSocketImpl.socketBind(Native Method)
   at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
   at java.net.ServerSocket.bind(ServerSocket.java:319)
   at java.net.ServerSocket.<init>(ServerSocket.java:185)
   at java.net.ServerSocket.<init>(ServerSocket.java:141)
   at org.alfresco.filesys.ftp.FTPNetworkServer.run(FTPNetworkServer.java:377)
   at java.lang.Thread.run(Thread.java:595)
11:04:36,378 INFO  [org.alfresco.service.descriptor.DescriptorService] Alfresco JVM - v1.5.0_06-b05; maximum heap size 455.125MB
11:04:36,378 WARN  [org.alfresco.service.descriptor.DescriptorService] Alfresco JVM - WARNING - maximum heap size 455.125MB is less than recommended 512MB
11:04:36,388 INFO  [org.alfresco.service.descriptor.DescriptorService] Alfresco started (Enterprise Network): Current version 1.2.0 schema 6 - Installed version 1.1.2 schema 0
11:04:46,129 DEBUG [org.alfresco.repo.security.authentication.ldap.LDAPPersonExportSource] Adding user for accountA‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

… importing the 998 other accounts …‍

11:05:12,241 DEBUG [org.alfresco.repo.security.authentication.ldap.LDAPPersonExportSource] Adding user for accountZ
11:05:25,881 ERROR [org.quartz.core.JobRunShell] Job DEFAULT.ldapGroupJobDetail threw an unhandled Exception: 
org.alfresco.repo.importer.ExportSourceImporterException: Failed to import
   at org.alfresco.repo.importer.ExportSourceImporter.doImport(ExportSourceImporter.java:165)
   at org.alfresco.repo.importer.ImporterJob.execute(ImporterJob.java:36)
   at org.quartz.core.JobRunShell.run(JobRunShell.java:191)
   at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:516)
Caused by: java.lang.NullPointerException
   at org.alfresco.repo.security.authentication.ldap.LDAPGroupExportSource.?(Unknown Source)
   at org.alfresco.repo.security.authentication.ldap.LDAPGroupExportSource.generateExport(Unknown Source)
   at org.alfresco.repo.importer.ExportSourceImporter.doImport(ExportSourceImporter.java:149)
   … 3 more
11:05:25,881 ERROR [org.quartz.core.ErrorLogger] Job (DEFAULT.ldapGroupJobDetail threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: org.alfresco.repo.importer.ExportSourceImporterException: Failed to import]
   at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
   at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:516)
* Nested Exception (Underlying Cause) —————
org.alfresco.repo.importer.ExportSourceImporterException: Failed to import
   at org.alfresco.repo.importer.ExportSourceImporter.doImport(ExportSourceImporter.java:165)
   at org.alfresco.repo.importer.ImporterJob.execute(ImporterJob.java:36)
   at org.quartz.core.JobRunShell.run(JobRunShell.java:191)
   at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:516)
Caused by: java.lang.NullPointerException
   at org.alfresco.repo.security.authentication.ldap.LDAPGroupExportSource.?(Unknown Source)
   at org.alfresco.repo.security.authentication.ldap.LDAPGroupExportSource.generateExport(Unknown Source)
   at org.alfresco.repo.importer.ExportSourceImporter.doImport(ExportSourceImporter.java:149)
   … 3 more‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

andy · ‎04-13-2006

Hi Simon

If you do not see the output from


            if (s_logger.isDebugEnabled())
            {
                s_logger.debug("Found " + lookup.size());
            }
‍‍‍‍‍‍

The only possibilities I can see are:
1) Some property/bean is not wired in (but this looks OK)
2) You have a group returned by the query with no attributes at all

Regards

Andy

simon · ‎04-13-2006

You have a group returned by the query with no attributes at all

This would mean that the FIRST group that is imported would be empty. We would see at least one "Adding group…" message in the log file if this wasn't the case, wouldn't we?

There is some traffic between Alfresco and AD when we run a tcpdump. This is the last package that is sent from AD to Alfresco, Alfresco doesn't reply on this package:

16:44:45.260752 adserver.company.be.ldap > alfresco.53519: . [tcp sum ok] 913711:915159(1448) ack 115 win 65421 <nop,nop,timestamp 204894928 950577113> (DF) (ttl 128, id 41979, len 1500)
E…..@…+..g…g.X……!………."…….6r.8…jectCategory1….3.1CN=Group,CN=Schema,CN=Configuration,DC=company,DC=be0….V..dSCorePropagationData1….9..20050218134851.0Z..20050218134818.0Z..16010101000417.0Z0……..d……3
CN=accountX,OU=Groups,OU=User.Accounts,DC=company,DC=be0…..0…….objectClass1…….top..group0…….cn1…….accountX0….:..description1….'.%Contract.account:.Anil.SurenameXl0…….member1….q.8CN=accountY,OU=Non-Personal,
OU=User.Accounts,DC=company,DC=be.5CN=accountZ,OU=Payroll,OU=User.Accounts,DC=company,DC=be0….N..distinguishedName1….5.3CN=accountY,OU=Groups,OU=User.Accounts,DC=company,DC=be0…….instanceType1…….40….&..
whenCreated1…….20020419075041.0Z0….&..whenChanged1…….20050215111125.0Z0…….uSNCreated1…….263980…….uSNChanged1…….263980….+..extensionAttribute101…….project.group0…….name1…….accountY0….$..objectGUID1…….
e…3..C."3m,.@.0…./..objectSid1…………………W.N~.RNZ..B$..0…….sAMAccountName1…….accountY0….!..sAMAccountType1…….2684354560…….groupType1…….-21474836460….I..objectCategory1….3.1CN=Group,CN=Schema,
CN=Configuration,DC=company,DC=be0….V..dSCorePropagationData1….9..20050218134851.0Z..20050218134818.0Z..16010101000417.0Z0……..d……1CN=accountW,OU=Groups,OU=User.Accounts,DC=company,DC=be0…..0…….
objectClass1…….top..group0…….cn1…….glcos0….7..description1….$."Activity.account:.Johan.SurenameW0….F..member1….8.6CN=accountV,OU=Non-Persona‍‍‍‍‍‍‍‍

Packages are not the nicest to read but the end seems interessting. Why is the Non-PersonaL string not finished. Could this be some timeout? Seems to me that the last package is not finished completly.

I still think there is something wrong with the configuration parameters for groups but can't figure out what exactly.

org.alfresco.repo.importer.ExportSourceImporterException: Failed to import‍

What does this mean? Failed to import the XML file? This would be normal, there is no group XML file (the file is 0 bytes) so importing won't be easy. Why isn't there an error while Alfresco exports the information from AD?

andy · ‎04-13-2006

Hi Simon

The debug for group import reports most of its info during the structure build. At the start it just reports the number for groups obtained.

Stuff is pulled from LDAP on demand, as I understand it. You probably have enough info to answer the next question ….the next packet …that is never requested… would continue from where this one leaves off…..

org.alfresco.repo.importer.ExportSourceImporterException: Failed to import

This is the containing Exception for the null pointer exception.
This is normal if anything goes wrong at all.

There was an issue dealing with groups that had no member attributes. This is fixed but I can not recall exactly when it went in. I think this is the most likely cause of your issue.

Regards

Andy

simon · ‎04-13-2006

Good news Andy!

I created an empty group in our OpenLDAP server and tried to import the groups, it failed like usual. Getting used to the error message by now…
When I remove the empty group and restart the job everything works fine. Your last remark solved it, empty groups are not allowed in the Enterprise 1.2 version. So it should be solved in the next release?
Can't do the test with our (corporate) AD, the empty groups are used for security purposes so we'll need the fix. Let's hope this was the only problem.

Thanks for your continuing support!

andy · ‎04-21-2006

Hi

1.2.0 has an issue with groups that contain no member attributes ? which you have found. 1.2.1 fixes this.

The error occurs while extracting the ldap group information and before the xml file is constructed.

There are two fixes:
1)       Upgrade to 1.2.1
2)       Remove the groups that do not have members
3)       Extend the ldap query so it does not find groups that do not have members ? as soon as they have members they will start to appear


 <property name="groupQuery">
            <value>(objectclass=group)</value>

 </property> 
‍‍‍‍‍‍

becomes:


 <property name="groupQuery">
       <value>(&(objectclass=group) (member=*))</value>
 </property>
‍‍‍‍‍

This should return groups that have members defined

FYI:

(member=*) is a presence filter that requires any value for the member attribute

(&()()) ?ands? multiple filters together

Option 3 should be easiest.

Regards

Andy

andy · ‎04-21-2006

Hi

1.2.0 has an issue with groups that contain no member attributes ? which you have found. 1.2.1 fixes this.

The error occurs while extracting the ldap group information and before the xml file is constructed.

There are two fixes:
1)       Upgrade to 1.2.1
2)       Remove the groups that do not have members
3)       Extend the ldap query so it does not find groups that do not have members ? as soon as they have members they will start to appear


 <property name="groupQuery">
            <value>(objectclass=group)</value>

 </property> 
‍‍‍‍‍‍

becomes:


 <property name="groupQuery">
       <value>(&(objectclass=group) (member=*))</value>
 </property>
‍‍‍‍‍

This should return groups that have members defined

FYI:

(member=*) is a presence filter that requires any value for the member attribute

(&()()) ?ands? multiple filters together

Option 3 should be easiest.

Regards

Andy

Hyland Connect

LDAP and people synchronisation not working