A customer of an AD Bridge product ( https://github.com/BeyondTrust/pbis-open ) asked me:

What is the best practice around how to ensure the Users you want to “export” are available to define with PBIS settings in the named cell on the RESOURCE side. Essentially, what prerequisite needs to be met so from the RESOURCE side, we can see these users. Is it a universal group?

If it is a universal group, is the expectation that the admins of the USER domain would know when onboarding a user that requires linux access, to put them in that group on their side (the USER side) so they’re available for cell enabling on the RESOURCE side?

a Typical 1-way trust setup has users in the “USER” domain, and the computers joined to the “RESOURCE” domain.  We’ll use those names, but “RESOURCE” could be a DMZ zone, or a partner system, or anything else.  For this discussion, we’ll use the following 2 forests:


RESOURCE has a 1-way outbound trust to USER (RESOURCE trusts USER, but accounts in RESOURCE cannot access the USER or ROOT domains. ROOT users cannot log into RESOURCE either)

In Windows, there are 4 kinds of groups ,with different purposes and limitations.  These limitations change slightly depending on the AD forest functional level (nesting is much more restrictive in older Forest Functional levels).  This document describes the AD groups: https://technet.microsoft.com/en-us/library/dn579255(v=ws.11).aspx but to recap the AD group types are “Domain Local”, “Domain Global”, and “Universal”. The 4th is “Computer Local Group” which you manage in the “Local Users and Groups” plugin to the “Computer Management” snap-in in the Windows OS (not in AD).  On Linux, this last one equates somewhat to groups in /etc/group.

  • Universal Groups: can contain any: Domain Global group, Universal Group, or User/Computer account in the same AD forest as the Universal Group.  Universal Groups are stored in the Global Catalog, making updates to them slower than other group types.  You can use them to control rights on any resource anywhere in the forest.
  • Domain Global Groups: can contain any Domain Global group or User/Computer account from the same Domain as the Domain Global group. Domain Global groups can be used to control rights on any resource anywhere in the forest.
  • Domain Local Groups: can contain pretty much any Group from any trusted domain, including across 1-way trusts. Domain Local groups can ONLY go into Domain Local groups in the same domain, or Computer Local groups in Computers on that Domain.  Domain Local groups can only be used to control rights on a resource in the same domain as the local group.

Of note: Universal groups store (in the AD database) a DN reference, which is part of why they can only contain objects form the same domain.  Domain Local groups store a SID reference, which is why they work across one-way trusts. (Domain Local Group membership contains a “foreignKeyReference” as the members, whereas Universal Groups contain a link to the GC object in the DIT which is expanded to the DN of the object on retrieval (meaning that if you move the member, the Universal Group membership doesn’t need to be updated, because the UUID of the object referenced by the link didn’t change).

Think of it this way: Users go into Universal and Domain Global groups.  Domain Global groups go into Domain Local Groups.  Domain Local groups are used to control access to Local resources in the Domain.  What this means is that if USER\rob is in the Domain Local group USER\dlg1, and accesses a server file01.root.local: file01 doesn’t even see that USER\rob is in USER\dlg1, because USER\dlg1 isn’t in the ROOT domain that file01 is in.  AD doesn’t even bother returning any foreign Domain Local groups in the Kerberos Ticket PAC (in the TGS) to file01, since that group can’t be used to control resources outside of the USER domain.

For any other 1-way-enabled AD Bridge like PBIS or Samba, then, which groups can we provision into the cell / Computers in RESOURCE?  PBIS specifically is less restrictive than AD (samba is not), and will allow you to import the Domain Global and Universal, and even if you try hard enough, the Domain Local groups from USER into the cell in RESOURCE, and use LDAP calls to attempt to resolve the Universal and Domain Global groups and their membership, because everything I just stated above is pretty difficult to explain to a Unix admin on a tech support call. In other words: the BeyondTrust PBIS Product will export the USER Domain Local groups to the RESOURCE domain computers, so that the Unix admins don’t have to know anything in this post, but the Samba admins will need to grok all of this.

But, when I’m designing a PBIS Cell design for a customer (it allows assigning UID numbers, rather than generating per host like Samba or SSSD), I will try to design the customer to build Domain Local groups in the RESOURCE domain, and populate those groups into the Cell, with the membership coming from the USER domain.  This meets the AD design methodologies, means that the Unix team can be delegated rights to control the group membership in the RESOURCE domain themselves, but not have any rights to any user accounts in USER. This also means that any Active Directory auditing tools will pick up and audit the users properly, and when PBIS drops this “nice enablement”, the customers I have worked with won’t need to make changes to their environment.

The information in this post came primarily from: https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/dd861330(v=ws.11)
When I originally wrote this in late 2017.

The UPS on the lab servers is near the end of its life, so we’ve changed the shutdown script in power outages to simply hard-power off some of the servers (using Rob’s fork of ESXIDown from https://github.com/docsmooth/esxidown ). Of course, the second DC had to be one of them, so that there would be enough battery left for the SQL server and FSMO / PDC to shut down cleanly. THis means that the second DC often fails to boot properly when the power comes back after our yearly ComEd outage.

So, how do you fix this without doing the research every time? Thankfully, modern versions of Windows can be configured to boot to recovery mode. If you log in and launch a command prompt, you can run the following commands to find and repair problems:

  1. Launch the command prompt.

    recovery mode cmd.exe window example

    Launching CMD.EXE in recovery window

  2. navigate to your drive with the NTDS.DIT file (on our lab server, it’s always d:\windows\ntds:
  3. Run an ESENTUTL checksum on the Active Directory database file ntds.dit:
    d:\windows\system32\esentutl.exe /k ntds.dit

    ESENTUTL.exe completed checksum

    ESENTUTL.exe completed checksum

  4. Even though that’s successful, you’ll probably fail an integrity check:
    d:\windows\system32\esentutl.exe /g edb
    On our server, that always generates the error:

    The database is not up-to-date. Integrity check may find that this database is corrupt because data from the log files has yet to be placed in the database. It is strongly recommended the database is brought up-to-date before continuing! Do you wish to abort the operation?

  5. If the checksum passed and you get this error, try rebooting into Directory Services Repair Mode – exit the command prompt and hit “Restart” and then pound on F8 to get the DSRM boot option.
  6. If DSRM bluescreens, then you need to go deeper into esentutl.exe to repair your DB. If this is NOT your only DC, you will have data loss, but it should replicate from other DCs back into this one, so it shouldn’t be a huge problem.
    If DSRM works, try an ntdsutil in cmd.exe:
    activate instance ntds
  7. If ntdsutil file recovery errors with

    Could not initialize the Jet engine: Jet Error -543.
    Failed to open DIT for AD DS/LDS instance NTDS. Error -21478418113

    then you need to do more work with esentutl, but can continue inside DSRM, rather than having to reboot.

    If you couldn’t boot into DSRM, continue as below, but from the recovery install CMD.exe. I’ll continue these commands from THAT pathing, since it turns out not bothering to jump into DSRM makes for a faster recovery, with ensured data loss. I don’t care abuot data loss in our lab though.

  8. Try to recover the database: d:\windows\system32\esenutl.exe /ml d:\windows\ntds\edb
  9. If that doesn’t work, delete the edb.log files, then try recovery again. You’ll get an error like

    Operation terminated with error -501 (JET_errLogFileCorrupt, Log file is corrupt) after 2.123 Seconds.

    So backup your logs to a new location or delete them outright:
    mkdir log-backup
    move edb*.log log-backup
    move edb.chk log-backup

  10. Recreate the log files with a hard recovery of the database:
    d:\windows\system32\esentutl.exe /p d:\windows\ntds\ntds.dit
    You’ll get an error saying you should only run this on damaged or corrupt databases. The checks before this have proven that that is the case in this situation, so click “OK”.
  11. This should restore the database and complete successfully. Reboot and test it. If it doesn’t work, or doesn’t boot, try again in all offline mode without jumping to DSRM.
  12. If that still doesn’t work:
  13. Recheck the database integrity:
    cd \windows\ntds
    esentutl /g: ntds.dit
  14. Do a database repair again:d:\windows\system32\esentutl.exe /p ntds.dit
  15. And reboot when that completes successfully. You *should* now boot properly, and the edb.chk and edb.log files should get rebuilt.

If you are setting up a cross-forest trust with selective authentication (which requires a Windows Server 2003 Native mode level forest and domain), don’t forget to grant the “Allowed to Authenticate” right to the users from the trusted domain to the servers they’ll need access to in your domain. The error messages you’ll get back (replicated here in my test VM domains) don’t really say much helpful.

System Error 317 has occurred. The system cannot find message text for message number 0x*** in the message file for ***.

System Error 317

Further information about adding the “Allowed to Authenticate” right to the trusted users is available at Microsoft TechNet. If you have the opportunity to raise your forest and domain functional levels to take advantage of this, I highly recommend it. But I recommend also (even more strongly) documenting precisely what you set.