Windows


I spent more time than I’d care to admit trying to write an LDIF import file for a customer today. I started with a file provided by someone else, which is of course the root of my problem.  After adding the appropriate ” ” after each “:” character (which is absolutely required), when importing it, I was receiving the following error:

There is a syntax error in the input file
Failed on line 21. The last token starts with '-'.
An error has occurred in the program

So I opened the file in Notepad, and saw nothing wrong. I sent it back to the Linux box it came from, opened it in vi, verified it had dos line endings, and still saw nothing wrong with the format, according to the MSDN Document on the subject of LDIF Schema modifications.

Only after scrolling through the file several times did I notice that line 20 wasn’t actually blank. it was a single horizontal tab character.

To recap:

  1. LDIF formatting is extremely specific, including breaking on whitespace appearance.
  2. LDIF formatting is extremely specific, including breaking on whitespace missing.
  3. LDIF formatting is extremely specific, including requiring the “-” to be a line literally on its own.

I recently had a co-worker trying to figure out a “lost email” problem we were possibly experiencing. He was blind to everything after hitting “send” because the intermediate servers weren’t sending delivery recipets, even though they were requested in the MIME header (Does any mail admin allow those to be sent out of the organization these days?) So, to help him out, I wrote up the following “how to test SMTP by hand” HOWTO.

First step is to determine which servers are responsible for mail delivery inbound for the domain you’re sending to. You do this by looking in DNS for the “MX” type records. These are provided in the format “priority servername.domain.” Priority is reverse-ordered. The easiest way to remember priority order is that it’s the order in which servers are attempted.

rob@rob-kubuntu3:~$ dig MX totalnetsolutions.net +short
10 docsmooth.isa-geek.net.
rob@rob-kubuntu3:~$ dig MX likewise.com +short
10 server1.inboundmx.com.
20 server2.inboundmx.com.

This tells you the servers, in order, that *all* mail will be sent to for the domain listed. So, anything to my likewise.com address will go to server1.inboundmx.com. The higher priorities are only used if the lower priorities fail to answer. If no server answers, the mail is held by the sender and retried, generally every 1 or 4 hours for up to 4 days, but this retry is configured on the *sending* server. That means, your own email admin (or you, if you’re the mail admin).

Next thing to check is: does the server work, and is it your sender, or their receiver? Check with telnet!
Stuff I type is in red:

rob@rob-kubuntu3:~$ telnet docsmooth.isa-geek.net 25
Trying 99.29.179.119...
Connected to docsmooth.isa-geek.net.
Escape character is '^]'.
220 totalnetsolutions.net Microsoft ESMTP MAIL Service, Version: 6.0.3790.4675 ready at Tue, 31 May 2011 08:43:08 -0500
HELO
250 totalnetsolutions.net Hello [12.130.116.175]
MAIL FROM: me@me.com
250 2.1.0 me@me.com....Sender OK
RCPT TO:you@you.net
250 2.1.5 you@you.net
DATA
354 Start mail input; end with .
from:me@me.com
to:you@you.net
subject:test manually
test
test
.

250 2.6.0 Queued mail for delivery
quit
221 2.0.0 totalnetsolutions.net Service closing transmission channel
Connection closed by foreign host.

The last “.” is SUPER important – it tells the mail server when you’re done sending that email. You could use that channel to send other messages, rather than sending “QUIT” if you’d like. You might notice that I entered the “From” and “To” lines twice. The first entries are for the SMTP header (analogy would be the message envelope), and the second entries are for the MIME headings (analogy would be the return address header in a formal postal letter, if anyone sends those). The MIME headings are what most mail programs display, and actually don’t technically need to match the SMTP header (but if MIME and SMTP don’t match many anti-spam programs will throw out the message).

The MIME header is pretty complex, but not order-dependant, although I prefer to enter it in order, so that I can be sure I don’t miss anything.
If you want to add an attachment, just base64 encode it first with:
perl -e 'use MIME::Base64; qw(encode_base64); print encode_base64("@ARGV");' cat attachment-to-send.zip
Then you can just paste it into the email. In the MIME heading (right after the subject), just add (with the appropriate mime coding, probably application/octet-string:

------=_NextPart_000_000D_01CC1C41.21F38080
Content-Type: application/zip;
name="attachment-to-send.zip"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
filename="attachment-to-send.zip"
<paste your base64 encoded attachment/ >
Content-Type: multipart/mixed;
boundary="----=_NextPart_000_000D_01CC1C41.21F38080"
------=_NextPart_000_000D_01CC1C41.21F38080
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit

Type your message here, ending with a “.” Standalone on a line.
.

Now that you know *how* to send an email message by hand, you can use the returned error codes to troubleshoot where the message may be disappearing. Remember, that this just gives you transport troubleshooting between yourself and the initial destination mail server. Many large (and even medium-sized) organizations will have a perimeter mail server which then forwards the message to one or more internal servers. If the mail is being dropped at that point, you’ll have to contact the reciever with the proof that their server is accepting your messages.

I upgraded the TNS lab this past week from Windows 2008 to Windows 2008 R2, including replacing the 4 Domain Controllers (rather than upgrading). It gave me a chance to review the procedure for moving a Certificate Server to a new system, which I hadn’t done since 2005. For those who haven’t tried, the procedure for moving a Certificate Server is reasonably well documented at the Microsoft Support site here: http://support.microsoft.com/kb/555012. The part of this that’s especially tricky, especially in our lab, is the renaming of the DC.

In our lab we have an empty forest root, as per the old (Windows 2000-era) Microsoft recommendations, to match several large customer environments. Because it’s a lab, and no clients connect to it, we only have a single DC. I snapshotted it as a backup, and went through the procedure to rename a domain controller, also well documented by Microsoft, this time at TechNet.

For review, the procedure we planned to run was:
netdom computername dc04 /add:dc01.lwtest.corp
netdom computername dc04 /makeprimary:dc01.lwtest.corp
shutdown -r -t 0
netdom computername dc01 /enum
netdom computername dc01 /verify
netdom computername dc01 /rem:dc04.lwtest.corp

I’m still not sure what caused it, but in this case, this command failed:
netdom computername dc04 /makeprimary:dc01.tns.lab
At this point, I couldn’t make the old name primary again (I would get an “Access Denied” error), so I rebooted to see which name had taken. And that’s where things went bad.

When the DC came up, we were getting this error: Netlogon EventID 5602. Source: NETLOGON, EventID: 5602, Data: “An internal error occurred while accessing the computer’s local or network security database.”

Because the DC rename hadn’t completed successfully, the computer couldn’t actually log into itself to load AD. Very bad for the root of the forest. I wasn’t able to find anything helpful in my searches, so thought I’d let you know the fix:

Name it back to the old name and try again:
Reboot into Safe Mode.
netdom computername localhost /makeprimary:dc04.lwtest.corp
shutdown -r -t 0

Boot normally
netdom computername localhost /makeprimary:dc04.lwtest.corp
netdom computername dc01 /enum
netdom computername dc01 /verify
shutdown -r -t 0

After *that* reboot, make sure, with the verify command, that the old name took, and that you can log in, and just try the rename again.

I couldn’t get the “rename back” to take untill after the attempt in safe mode. Strange, but it’s working great now! Hopefully this will help someone.

I recently upgraded the totalnetsolutions.net internal network from ESX 3.5 to ESXi 4.1. The ESX Host upgrade itself is simple, and not worth mentioning. When complete, however, you have an option to upgrade the Guest OS Virtual Hardware from v4 to v7. Support for USB devices, thin-provisioned disks, and supposed speed improvements come with the upgrade.

The process should always be:

1. Upgrade VMware Tools to the latest available version. This pre-stages the drivers for the newest hardware, even though it’s not “installed” yet.
2. Reboot the guest and make sure it boots and runs properly after all upgrades (host and guest) have been completed.
3. Back up the entire guest VM, including the VMX and VMDK files.
4. Upgrade the virtual hardware through vSphere
5. Boot the VM and verify all settings are working properly.

I started the upgrades in the Unix lab. The Red Hat Enterprise Linux (4 and 5) and Ubuntu (10) systems went without a hitch. VMware Tools automatic upgrade went properly, systems rebooted fine, and after upgrading the virtual hardware, I didn’t have to change a thing in the guests. The Solaris 10 x86 guest, had some issues, however. I believe a rescan was all that was required to fix it, but we were planning on rebuilding the box anyways, so used the issues as the final “nail in the coffin” to the old hardware.

On the Windows side, we have 2 pools in our ESX environment: one for test machines, and one running our production environment. We have Domain Controllers (and separate forests) in both environments, but all file and Exchange operations only live in production.

The Windows 2003 DC / Exchange 2003 server came up fine, although it lost its network configuration (adapter MAC changed), so that had to be reset, but is a simple fix.

All Windows 2008 DCs in the test lab, including the RODC, came up fine, but with the same “lost network configuration” hiccup. These systems all have the NTDS data and logs on the C: drive.

The Windows 2008 Server Core DC / File server, however, was a different story. Upon reboot, the server kept giving a BSOD and rebooting, so I couldn’t read the error. As this system is the primary (200GB) file server, primary DNS server (including conditional forwarding to the test lab), and the DC that handles the most load (DNS weight on the Windows 2003 is slightly lower), fixing the Blue Screen was of major importance. This is how it’s been fixed:

1. Safe Mode and “Last known Config” didn’t work, so hit F8 on the boot process to choose “Do not restart on system failure”. This allows you to read the BSOD message. In our case, it was simply “File Not Found”. Which means, no minidump, and you might be sunk.
2. On a whim, since it is a DC, I tried to boot into Directory Services Restore Mode, hoping the “not found” file was AD related… and was right.
3. This leads us down the path of this support article.
4. Immediately upon booting, I ran: ntdsutil files integrity which gave this error:
Could not initialize the Jet engine: Jet Error -566.
Failed to open DIT for AD DS/LDS instance NTDS. Error -2147418113
5. Searching shows there’s not much useful here, but we know it’s a failure to read the DIT. This could be security, or horrid corruption.
6. I quit ntdsutil to try to check the files on the E: drive, where they lived, only to find there was no E: drive. With no MMC, it’s diskpart to the rescue.
7. diskpart
DISKPART> list disk
Disk ### Status Size Free Dyn Gpt
-------- ---------- ------- ------- --- ---
Disk 0 Online 24 GB 0 B
Disk 1 Offline 100 GB 0 B
Disk 2 Offline 100 GB 0 B

8. I ran:
select disk 1
online
select disk 2
online
exit

9. Now I can read the E: drive, so try ntdsutil files integrity again… and get the same error message. Checking the disk, everything looked fine. In Linux, I’d check permissions with a quick “touch filename”, but notepad needed to be used here, only to discover the entire disk was marked read-only. Back to diskpart!
diskpart
select disk 1
attributes disk clear readonly
select disk 2
attributes disk clear readonly

10. Now ntdsutil runs properly, reboot into normal mode, and the system is fixed!

I haven’t seen posts of other people having disks get marked offline and unreadable on their VMs after an upgrade, but this only happened on the Windows 2008 system, and it’s non-system disks.

I’ve been fighting K9Mail for weeks now, trying to get it to sync with MailStreet who hosts “exchange.ms”) hosted Exchange. If you’ve already followed the instructions at the K9Mail Wiki with no success, read on.

Thanks to the k9mail wiki on debugging connection issues and the fact that I already had the Android SDK installed, I was able to solve the 2 related errors I was getting. I would either get an “HTTP 404 not found” or an “HTTP 501 Not Implemented” depending on the settings I chose. With no additional settings other than suggested in the Wiki, I’d get a “501 not implemented”. If I tried to set a mailbox path, or a WebDAV path, I’d get the HTTP 404 Not Found.

In the debugging log, I saw that the system was calling “http://mail.$domain.exchange.ms/”$webDAVpath/Inbox – if I set it to a full URL, the full URL was getting appended. When I attempted to hit those same paths in a full browser, I’d always get an HTTP 404. So, digging in my history in Firefox, I found the following (cleaned) path:
http://mail.$domain.exchange.ms/exchange/$emailaddress/
In this case $emailaddress was my Exchange mail address with the “@” stripped out. Appending “Inbox” to the end of this path resulted in a valid load of my OWA inbox.

Plugging then: /exchange/$emailaddress/ into the WebDAV box in K9Mail, and my email immediately loaded up.

Now I have Android syncing my calendars and contacts, and k9mail is handling my massive inbox!

« Previous PageNext Page »