[SSSD-users] "could not store group" failures for lookups on Active Directory groups

[James Ralston]

Hi,

I think this problem may be part (or related to) the "FreeIPA/SSSD
LDAP cross-forest trust slow queries" issue, but I'm not sure.

We've been testing sssd on our RHEL6 and RHEL7 hosts, using the latest
available packages. We have a fairly simple sssd configuration. We
use the "ad" provider with LDAP id mapping:

[sssd]
config_file_version = 2
debug_level = 0x0070
domains = example.org
services = nss, pam

[nss]
debug_level = 0x0070
default_shell = /bin/bash
fallback_homedir = /home/%u

[pam]
debug_level = 0x0070

[domain/example.org]
access_provider = ad
auth_provider = ad
cache_credentials = true
chpass_provider = ad
debug_level = 0x0010
dyndns_update = false
enumerate = true
id_provider = ad
krb5_realm = EXAMPLE.ORG
ldap_id_mapping = true
ldap_sasl_mech = GSSAPI
ldap_schema = AD
offline_failed_login_attempts = 3

Although we have about 1200 groups in Active Directory, only about 900
of them show up in the output of "getent -s sss group". For the
remaining ones, attempting a lookup (e.g., with getent) returns no
entry, and we see these lines in the sssd_example.org.log file:

(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sdap_get_groups_next_base] (0x0400): Searching for groups with base
[DC=example,DC=org]
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sdap_get_generic_ext_step] (0x0400): calling ldap_search_ext with
[(&(name=my-group)(objectClass=group)(name=*))][DC=example,DC=org].
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_parse_entry]
(0x1000): OriginalDN: [CN=my-group,OU=Distribution Groups,OU=Microsoft
Exchange,DC=example,DC=org].
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sdap_nested_group_hash_group] (0x4000): AD group has type flags 0x8.
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sdap_nested_group_hash_group] (0x0400): Filtering AD group.
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sdap_nested_group_hash_group] (0x4000): The group's gid was zero
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sdap_nested_group_hash_group] (0x2000): Marking group as non-posix
and setting GID=0!
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sdap_nested_group_hash_entry] (0x4000): Inserting
[CN=my-group,OU=Distribution Groups,OU=Microsoft
Exchange,DC=example,DC=org] into hash table [groups]
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sdap_nested_group_process_send] (0x2000): About to process group
[CN=my-group,OU=Distribution Groups,OU=Microsoft
Exchange,DC=example,DC=org]

[user lookups omitted]

(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sdap_get_primary_name] (0x0400): Processing object my-group
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group]
(0x0400): Processing group my-group
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group]
(0x4000): AD group [my-group] has type flags 0x8.
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group]
(0x0400): Filtering AD group [my-group].
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sdap_attrs_add_ldap_attr] (0x2000): Adding original DN
[CN=my-group,OU=Distribution Groups,OU=Microsoft
Exchange,DC=example,DC=org] to attributes of [my-group].
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sdap_attrs_add_ldap_attr] (0x2000): Adding original mod-Timestamp
[20150429215241.0Z] to attributes of [my-group].
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sdap_process_ghost_members] (0x0400): The group has 5 members
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sdap_process_ghost_members] (0x0400): Group has 5 members
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sysdb_attrs_get_aliases] (0x2000): Domain is case-insensitive; will
add lowercased aliases
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group]
(0x0400): Storing info for group my-group
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sysdb_set_entry_attr] (0x0080): ldb_modify failed: [Attribute or
value exists]
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sysdb_set_entry_attr] (0x0040): Error: 17 (File exists)
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sysdb_store_group]
(0x1000): sysdb_set_group_attr failed.
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sysdb_store_group]
(0x0400): Error: 17 (File exists)
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sdap_store_group_with_gid] (0x0040): Could not store group my-group
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group]
(0x0080): Could not store group with GID: [File exists]
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_group]
(0x0080): Failed to save group [my-group]: [File exists]
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]] [sdap_save_groups]
(0x0040): Failed to store group 0. Ignoring.

I can't find any pattern for what groups are correctly mapped versus
the groups that aren't. But it's consistent across different sssd
client hosts. Stopping sssd and removing its cache has no effect.
Nor does disabling enumeration.

Since we just recently started testing sssd, we don't know for certain
whether group lookups worked properly in the past. All we know is
that it doesn't seem to work now.

Is this a known issue, or is this a new problem I should report to
Bugzilla and/or Red Hat support?

If the former, is a patch available? Because if so, I'll rebuild our
sssd RPMs locally with the patch until Red Hat releases official
updated packages...

Thanks in advance for any assistance anyone might have...

[Lukas Slebodnik]

This post might be inappropriate. Click to display it.

[Jakub Hrozek]

Post by Lukas Slebodnik

Post by James Ralston
Hi,
I think this problem may be part (or related to) the "FreeIPA/SSSD
LDAP cross-forest trust slow queries" issue, but I'm not sure.
We've been testing sssd on our RHEL6 and RHEL7 hosts, using the latest
available packages. We have a fairly simple sssd configuration. We
[sssd]
config_file_version = 2
debug_level = 0x0070
domains = example.org
services = nss, pam
[nss]
debug_level = 0x0070
default_shell = /bin/bash
fallback_homedir = /home/%u
[pam]
debug_level = 0x0070
[domain/example.org]
access_provider = ad
auth_provider = ad
cache_credentials = true
chpass_provider = ad
debug_level = 0x0010
dyndns_update = false
enumerate = true

I Hope it was just for testing purposes. We do not recommend to enable
enumeration.

You know, just this morning, I was thinking about enumeration. It
doesn't work for IPA views at all for example. It doesn't work for
trusted domains at all either (except for some limited support in AD
trusted domains that is very untested)

I wonder if we could just remove enumeration from IPA and AD back ends
in some major release.

It's just a legacy feature, so those who need it can fall back to the
LDAP provider..

[James Ralston]

Hi Lukas,

Post by Lukas Slebodnik

Post by James Ralston
enumerate = true

I Hope it was just for testing purposes. We do not recommend to
enable enumeration.

I know it's not recommended. I'll address this in a separate
response.

Post by Lukas Slebodnik

Post by James Ralston
ldap_id_mapping = true

You can remove this line, id-mapping is enabled by default for
id_provider ad

Post by James Ralston
ldap_sasl_mech = GSSAPI
ldap_schema = AD

Previous 2 lines are efault for id_provider ad as well

I know. (I added those lines more for illustration purposes than
anything else.)

Post by Lukas Slebodnik

Post by James Ralston
offline_failed_login_attempts = 3

This line shoudl be in [pam] section, it will have effect only if
"cache_credentials" is enabled in domain section.

Ah; good to know. I will correct that. Thanks.

Post by Lukas Slebodnik
I would be curious where did you inspire in sssd.conf. So we can
improve it.

I created it myself. So, blame me. ;-)

Post by Lukas Slebodnik
Distributions grups are filtered by default.
"Distribution groups are not security-enabled, which means that
they cannot be listed in discretionary access control lists
(DACLs). If you need a group for controlling access to shared
resources, create a security group."

OK.

A suggestion: it would have been very helpful if the debug messages
had contained some statement like "ignoring distribution group
my-group".

That would have made it much more clear what was happening, which (as
I understand it) was:

1. sssd was ignoring a distribution group. (This is normal,
expected behavior.)

2. sssd was trying to cache the distribution group, but failing
due to ticket/2588. (This is bug, not normal behavior.)

Post by Lukas Slebodnik

Post by James Ralston
(Wed May 6 00:03:06 2015) [sssd[be[example.org]]]
[sysdb_set_entry_attr] (0x0080): ldb_modify failed: [Attribute or
value exists]

^^^^^^^^^^^^^^^^^
here is a problem
It is very likely upstream bug[2] with binary objectGUID

OK, thanks for the explanation and the pointer.

Post by Lukas Slebodnik
For testing purposes you can try to use pre-release[3] of upstream
1.12.5 it should be released within few days and it contains fix for
bug[2] and also other fixes.

OK.

What do you recommend doing for RHEL6 (currently on
1.11.6-30.el6_6.4)?

1. Use your 1.12.5 packages on RHEL6?

2. Wait for Red Hat to backport the patch for ticket/2588 to
their 1.11.6 branch?

3. Wait for Red Hat to rebase RHEL6 to 1.12.5?

4. Backport the patch for ticket/2588 to 1.11.6-30.el6_6.4
myself?

5. Something else?

Thanks...

[Jakub Hrozek]

Post by James Ralston
What do you recommend doing for RHEL6 (currently on
1.11.6-30.el6_6.4)?
1. Use your 1.12.5 packages on RHEL6?
2. Wait for Red Hat to backport the patch for ticket/2588 to
their 1.11.6 branch?

Unlikely to happen w/o a support case open and even so, a RHEL-6.6
update would have to be justified.

Post by James Ralston
3. Wait for Red Hat to rebase RHEL6 to 1.12.5?

RHEL-6.7 will rebase to sssd-1-12. If you want to stay on the supported
patch, this is the best option. btw 6.7 beta was released just today.

Post by James Ralston
4. Backport the patch for ticket/2588 to 1.11.6-30.el6_6.4
myself?

The unsupported options are really up to you. If you can cherry-pick the
single patch itself, that would create the smallest possible delta
compared to the vanilla RHEL packages.

[James Ralston]

3. Wait for Red Hat to rebase RHEL6 to 1.12.5?

RHEL-6.7 will rebase to sssd-1-12. If you want to stay on the
supported patch, this is the best option. btw 6.7 beta was
released just today.

Agreed; we don't care about RHEL 6.6 if 6.7 is coming and has sssd
1.12.

[Jakub Hrozek]

Post by James Ralston

3. Wait for Red Hat to rebase RHEL6 to 1.12.5?

RHEL-6.7 will rebase to sssd-1-12. If you want to stay on the
supported patch, this is the best option. btw 6.7 beta was
released just today.

Agreed; we don't care about RHEL 6.6 if 6.7 is coming and has sssd
1.12.

Thanks; any testing of 1.12 is greatly appreciated.