The UPS on the lab servers is near the end of its life, so we’ve changed the shutdown script in power outages to simply hard-power off some of the servers (using Rob’s fork of ESXIDown from https://github.com/docsmooth/esxidown ). Of course, the second DC had to be one of them, so that there would be enough battery left for the SQL server and FSMO / PDC to shut down cleanly. THis means that the second DC often fails to boot properly when the power comes back after our yearly ComEd outage.

So, how do you fix this without doing the research every time? Thankfully, modern versions of Windows can be configured to boot to recovery mode. If you log in and launch a command prompt, you can run the following commands to find and repair problems:

  1. Launch the command prompt.

    recovery mode cmd.exe window example

    Launching CMD.EXE in recovery window

  2. navigate to your drive with the NTDS.DIT file (on our lab server, it’s always d:\windows\ntds:
    d:
    cd\windows\ntds
  3. Run an ESENTUTL checksum on the Active Directory database file ntds.dit:
    d:\windows\system32\esentutl.exe /k ntds.dit

    ESENTUTL.exe completed checksum

    ESENTUTL.exe completed checksum

  4. Even though that’s successful, you’ll probably fail an integrity check:
    d:\windows\system32\esentutl.exe /g edb
    On our server, that always generates the error:

    The database is not up-to-date. Integrity check may find that this database is corrupt because data from the log files has yet to be placed in the database. It is strongly recommended the database is brought up-to-date before continuing! Do you wish to abort the operation?

  5. If the checksum passed and you get this error, try rebooting into Directory Services Repair Mode – exit the command prompt and hit “Restart” and then pound on F8 to get the DSRM boot option.
  6. If DSRM bluescreens, then you need to go deeper into esentutl.exe to repair your DB. If this is NOT your only DC, you will have data loss, but it should replicate from other DCs back into this one, so it shouldn’t be a huge problem.
    If DSRM works, try an ntdsutil in cmd.exe:
    ntdsutil.exe
    activate instance ntds
    files
    recover
  7. If ntdsutil file recovery errors with

    Could not initialize the Jet engine: Jet Error -543.
    Failed to open DIT for AD DS/LDS instance NTDS. Error -21478418113

    then you need to do more work with esentutl, but can continue inside DSRM, rather than having to reboot.

    If you couldn’t boot into DSRM, continue as below, but from the recovery install CMD.exe. I’ll continue these commands from THAT pathing, since it turns out not bothering to jump into DSRM makes for a faster recovery, with ensured data loss. I don’t care abuot data loss in our lab though.

  8. Try to recover the database: d:\windows\system32\esenutl.exe /ml d:\windows\ntds\edb
  9. If that doesn’t work, delete the edb.log files, then try recovery again. You’ll get an error like

    Operation terminated with error -501 (JET_errLogFileCorrupt, Log file is corrupt) after 2.123 Seconds.

    So backup your logs to a new location or delete them outright:
    mkdir log-backup
    move edb*.log log-backup
    move edb.chk log-backup

  10. Recreate the log files with a hard recovery of the database:
    d:\windows\system32\esentutl.exe /p d:\windows\ntds\ntds.dit
    You’ll get an error saying you should only run this on damaged or corrupt databases. The checks before this have proven that that is the case in this situation, so click “OK”.
  11. This should restore the database and complete successfully. Reboot and test it. If it doesn’t work, or doesn’t boot, try again in all offline mode without jumping to DSRM.
  12. If that still doesn’t work:
  13. Recheck the database integrity:
    cd \windows\ntds
    esentutl /g: ntds.dit
  14. Do a database repair again:d:\windows\system32\esentutl.exe /p ntds.dit
  15. And reboot when that completes successfully. You *should* now boot properly, and the edb.chk and edb.log files should get rebuilt.

Sometimes it’s nice to know what’s happening under the hood, so let’s talk about how Group Policy is built, by tearing down how to access a particular policy. First, Group Policy is implemented in 2 parts, an LDAP part and a file part, delivered via SMB (CIFS if you’re oldschool) via DFS (Distributed FIle SYstem). Because the DFS part is replicated completely differently than the AD part, there’s a version number for each Group Policy object that’s stored in both places to keep them in sync. Most GPO engines remember the last version they applied by remembering the lowest of the 2 numbers (the LDAP version and the file version in the GPT.INI), if they don’t match.

Let’s talk about the “Default Domain Policy” which everyone will have one of. To find where that policy lives, you have to ask AD. The policy doesn’t actually live in the OU or Domain where it’s linked, so we have to back out the link:
ldap_search_s(ld, "dc=company,dc=com", 2, "(objectClass=organizationalUnit)", gpLink, base, &msg)
We’ll get back something like:

gPLink: [LDAP://CN={6AC1786C-016F-11D2-945F-00C04fB984F9},CN=Policies,CN=System,DC=company,Dc=com;0]

Now, this is a multi-valued array, because multiple GPOs can be linked, in order, to a single OU or Domain or Site. But we only care about this one, so let’s see what’s in it:

ldap_search_s(ld, "CN={6AC1786C-016F-11D2-945F-00C04fB984F9},CN=Policies,CN=System,DC=company,DC=com", 2, "(objectClass=*)", gPCMachineExtensionNames;gPCFileSysPath;displayName;versionNumber, 0, &msg)

That’ll get us the Client Side Extensions (where the work actually happens), and what the file path to the files in the estension are stored, as well as the pretty name of the Group Policy Object:

displayName: Default Domain Policy;
gPCFileSysPath: \\child1.lwtest.corp\sysvol\child1.lwtest.corp\Policies\{31B2F340-016D-11D2-945F-00C04FB984F9};
gPCMachineExtensionNames: [{35378EAC-683F-11D2-A89A-00C04FBBCFA2}{53D6AB1B-2488-11D1-A28C-00C04FB94F17}{53D6AB1D-2488-11D1-A28C-00C04FB94F17}{D02B1F72-3407-48AE-BA88-E8213C6761F1}][{827D319E-6EAC-11D2-A4EA-00C04F79F83A}{803E14A0-B4FB-11D0-A0D0-00A0C90F574B}][{B1BE8D72-6EAC-11D2-A4EA-00C04F79F83A}{53D6AB1B-2488-11D1-A28C-00C04FB94F17}];
versionNumber: 15;

So we have the Default Domain Policy, as desired, but there are a bunch of client side extensions here. It’d be nice to know what they all do generically, without having to inspect each one.
And TechNet delivers on that desire: a list of all Client Side Extensions (in 2010) by GUID for easy reference. Now, I’m writing this, because someone asked where the Password Policy for the domain was stored. Well, that appears to be in: {827D319E-6EAC-11D2-A4EA-00C04F79F83A} Security, which our Default Domain policy applies. So, let’s go find the data!

One of the attributes in the list we last requested was gPCFileSysPath which returned a normal SMB share. If you browse to that share, you’ll see 3 objects:

  • A folder named “MACHINE”
  • A folder named “USER”
  • a file named “GPT.INI”

The GPT.INI will only have 2 lines:

[General]
Version=15

That’s the version number, that you can compare to the “versionNumber” property from the object. If they’re the same, you’re good. If not, your AD isn’t in sync.

In the “MACHINE” Folder are all the Computer Policy settings, and in the “User” folder are all the User Policy settings. Since we were talking about the Password Policy, which is affected on the SAM on the server, it’s a MACHINE setting. If you were to poke through, eventually you’d find this file:

\\domain\sysvol\domain\\MACHINE\Microsoft\Windows NT\SecEdit\GptTmpl.inf

with this data:

[Registry Settings]
MACHINE\System\CurrentControlSet\Services\Netlogon\Parameters\MaximumPasswordAge=4,15

And there’s your password policy, via LDAP and SMB only.

For a bit of additional background, when a computer processes this data, in this order, it will actually only apply CSEs from the gPCMachineExtensionNames that the computer recognizes and has DLLs (or whatever code, if it’s non-Microsoft vendor CSE) that can apply the CSE. This makes it technically safe to put multiple GPOs for multiple Operating sytsems on the same OU structure, knowing that the client computer won’t even bother downloading the files for un-recognized CSEs.

Now, that’s a lot of stuff to type into ldp.exe, how can we make a report on this a bit easier? Well, PowerShell could do it, but one of the products I work on is PowerBroker Open https://github.com/BeyondTrust/pbis-open and https://www.beyondtrust.com/products/powerbroker-identity-services-open/ which includes a CLI for browing ldap called “adtool”. With a bit of bash, we can list out all the group policy objects by name attached to a single OU:

$ cat report-gpos-by-ou.sh
GP=`adtool -a lookup-object --dn "$@" --attr gPLink`;
GPO=`echo $GP | sed -e 's/\[LDAP:\/\///g' -e 's/;[[:digit:]]\]/ /g'`
if [ -n "$GPO" ]; then
echo "";
echo "$OU";
for P in $GPO; do
G=`adtool -a lookup-object --dn "$P" --attr displayName`;
grep -q "$G" /tmp/gpos.txt;
if [ $? -ne 0 ]; then
echo $G >> /tmp/gpos.txt;
fi;
echo "$G";
done;
fi;
$ ./report-gpos-by-ou.sh "OU=Company,DC=domain,DC=com"
OU=Company,DC=domain,DC=com
PBUL Basics
GP-Preferences
$

Joe and Jorge posted these back in 2005 and 2006, but they’re impossible for me to find in Google lately, possibly because of age:

http://blog.joeware.net/2005/07/17/48/
http://jorgequestforknowledge.wordpress.com/2006/01/05/creating-a-taskpad-and-delegating-several-admin-tasks/

In order to move an object in DS, you need the following three permissions:
1) DELETE_CHILD on the source container or DELETE on the object being moved
2) WRITE_PROP on the object being moved for two properties: RDN (name) and
CN (or whatever happens to be the rdn attribute for this class, i.e. ou for
org units).
3) CREATE_CHILD on the destination container.

Dmitri Gavrilov
SDE, Active Directory Core
This posting is provided “AS IS” with no warranties, and confers no rights.
Use of included script samples are subject to the terms specified at
http://www.microsoft.com/info/cpyright.htm

But, what, specifically does that mean?

  1. To provide these rights, after delegating control for the Creation and Deletion of the object (Computer/User/etc.), open ADSIEDIT.MSC and navigate to the OU in question.
  2. Right-click the OU and choose “Properties”
  3. Click on the “Security” tab.
  4. Click the “Advanced” button.
  5. Click the “Add” button to add a new security right.
  6. Enter the group you want to delegate the control to and click “OK”
  7. Choose the “Properties” tab.
  8. In the pulldown, choose “Descendent Computer Objects”
  9. Grant:
  1. Read and Write canonicalName
  2. Read and Write name
  3. Read and Write Name

I built a Windows Server 2008 Server Core DC last week. It’s an interesting exercise because you have to use an unattend.txt file. I found quite a few places online that listed RODC unattend.txt files, but not full read-write DC unattend.txt files. So, attached to this post you’ll find the unattend.txt I used, but also, of more interest, I’m attaching the full help file directly from the server, which I used to create the file.

FIrst, you have to install the server and set an IP address – my previous posts on IP changes on DCs all used netsh commands as well, so if you followed thouse, you should be somewhat prepared for Server Core. I already had a WIndows Server 2003 DC in the environment, so that will be my primary DNS server for the install, untill DCPromo edits the settings.
netsh interface ipv4 set address local static 10.1.1.6 255.255.255.0 10.1.1.1 10
netsh interface ipv4 set dns local static 10.1.1.5
netsh interface ipv4 set wins local static 10.1.1.5

Now networking is set up, we can rename the computer: netdom renamecomputer %computername% /NewName:dc02 and join the domain with etdom join dc02 /domain:foo.local /UserD:FOO\Administrator /reboot:5 /PasswordD:*. The “5″ after the reboot flag says to reboot 5 seconds after completion, and the “*” at the end says to prompt you for your password. I join the system to the domain manually first, because then I can WSUS patch it (if WSUS is in the network), or open up the firewall for any other patching software I have.

Once the server is back from reboot, activate, update the firewall to allow remote MMC connections (if you’re not doing that through GPO already), and install new roles.
slmgr.vbs -ato
netsh advfirewall firewall set rule group="Remote Administration" new enable=yes

The following roles are optional, depending on the service of the server. Mine has DNS and the File Server roles, but not DHCP. None of these are required to install AD Domain Services!
start /w ocsetup DNS-Server-Core-Role
start /w ocsetup DHCPServerCore
start /w ocsetup FRS-Infrastructure
start /w ocsetup DFSN-Server
start /w ocsetup DFSR-Infrastructure-ServerEdition

If this is the first Windows Server 2008 DC in your environment, you’ll need to take the Windows Server 2008 DVD to the DC with the Infrastructure Master role (required for /gpprep only) and run the following (E: assumed as DVD-ROM drive):
e:\sources\adprep\adprep.exe /forestprep
e:\sources\adprep\adprep.exe /domainprep
e:\sources\adprep\adprep.exe /domainprep /gpprep
(Also run adprep /rodcPrep if you plan on building RODCs.)

Now you’re ready to do the DCPromo itself. Create an unattend.txt file. To add a DC to an existing domain, you can use:
[DCInstall]
AutoConfigDNS=Yes
ConfirmGc=Yes
DatabasePath=E:\Windows\NTDS
LogPath=c:\windows\NTDS
RebootOnSuccess=Yes
ReplicaDomainDNSName=foo.local
ReplicaOrNewDomain=Replica
ReplicationSourceDC=dc01.foo.local
SafeModeAdminPassword=passwordhere
SysVolPath=e:\windows\SysVol
UserDomain=foo.local
/Password:passwordhere

DCPromo will wipe out the passwords when it starts, or you can fill in “*” instead of the password, to be prompted. When it’s done, the server will reboot and be a new Global Catalog / DC in your domain. DCPromo will install neccessary binaries and configure the firewall for DC Services for you. It’s quite slick.

And as promised, here are the DCPromo Unattend Options for reference for creating your own unattend.txt.