As I mentioned in my last clustering post, there are some Exchange problems we’ve been working on over the past few weeks.  One of the simpler problems has a complex answer, so I thought I’d explain a bit.

As any good Exchange administrator knows, Exchange stores its data (for a store) in 2 files, the EDB file, and the STM file.  However, there’s not a really great explanation of the differences between the two files – the best I’ve found so far is at MessagingTalk.org, but they only explain that the STM is MIME formatted, and the EDB is MAPI content. Why, though, and how does it affect the end users? This is what we’ll explore.

What brought me into this topic was an end-user complaining that an email with a 7MB attachment was getting bounced, and the NDR said that it was over the mail system limit.  Normally, nothing worth investigating: zip the attachment, and resend.  The problem was, the user had already zipped it down, and the messaging size limit for the Exchange Organization is 10MB.  We had the user forward the message to several admins, and it went through ok.  About 500KB of HTML-formatted email, and a 7.1MB attachment, for a total size of 7.6MB.  When I sent it out to my account back here, it bounced back – over size limit, and the emails were being rejected by the Front End Exchange servers, so the emails weren’t even leaving the organization.

But Why?

Remember how we said earlier that the EDB file contains MAPI-formatted emails?  All messages submitted to Exchange via MAPI clients (Outlook) are stored directly as submitted in the EDB file for the user’s store.  All the message information is stored alongside that message in the store, and Exchange is able to serve the messages back to Outlook conversion-free. 

In the same way, all messages submitted to the store in MIME format (SMTP, POP3, IMAP4, or HTTP – OWA, WebDAV, OMA, or Exchange ActiveSync) are stored in the STM file in their original MIME format, and the message properties that Exchange needs to track are stored separately in the EDB file with pointers to the message location in the STM file.  This means that the STM file is straight plain text, so any data in it is 100% extractable if the store is dismounted (file unlocked).  I do have to say, grepping your STM file for your own email address is pretty interesting, if you’re into that kind of thing.

The thing about MIME formatted email is that any non-ASCII text has to be Base64 encoded, which causes a 33% increase to the size of the attachment. (See http://www.faqs.org/rfcs/rfc1521.html section 5.2 for the explanation.)  And aren’t most of your users writing “Rich Text” format emails with Word as their default editor?  Well, all that extra formatting most likely causes them to have to be base64 encoded when they’re converted to MIME.

Email sent between Exchange servers within the Organization are sent through the SMTP connector, but are kept in MAPI format by BDAT encoding the contents.  However, email sent outside of the organization (especially if the remote SMTP server doesn’t accept ESMTP) must be converted to MIME encoding before delivery.

So, looking at my particular user’s case:  The 7.1MB attachment grows to 9.5MB, and the 500KB of HTML formatted messaging replies adds up to 10MB on the nose.  Add SMTP headers, and the whole thing gets stopped by the SMTP connector on the Front-End Exchange server for just breaking the limit.

In this case, we used it as an opportunity to review how the client was sharing data with customers, and find a way to help them not use email as a file share.