Emailing SharePoint 2007 Lists and Libraries

Windows SharePoint Services (WSS) 3.0 and Microsoft Office SharePoint Server (MOSS) 2007 let users contribute content to lists and libraries via email. This ability adds to existing methods such as uploading through the browser and offers end users more flexibility in the way they contribute to team sites. Email is an especially useful way of contributing SharePoint content for the road warrior who might not always have a direct connection to the network.

Microsoft SharePoint Portal Server 2003 let you email documents into document libraries via an Exchange Server mail-enabled public folder—a cumbersome, circuitous method. WSS 3.0 and MOSS 2007 improve the process with the ability to send messages directly into many types of lists and libraries without relying on Exchange public folders. This is a welcome development, but there are some implementation issues that could trip you up along the way. In this article, I show you how emailing works with SharePoint 2007 and highlight some problems you should be aware of. SharePoint also supports outgoing email for notification of various events, but that’s outside the scope of this article.

Basic Architecture
First, I’d like to make it clear that SharePoint is not an SMTP engine, and it doesn’t care what messaging system you use; Exchange (or any other specific messaging system) isn’t required to be able to email items into SharePoint lists and libraries. Essentially, SharePoint monitors a folder location looking for correctly formatted SMTP messages. SharePoint doesn’t care how those messages arrive in the folder—it just opens them up and tries to associate each message with a list or library somewhere in the SharePoint farm. It doesn’t return delivery failures nor does it process requests for read or delivery receipts—which is what I mean by it not being an SMTP engine in itself.

In a typical environment, the folder that SharePoint monitors is the drop folder of the SMTP service running on web front-end servers within the farm. The general messaging infrastructure is configured such that SharePoint-bound messages are routed to the SMTP service. How this routing is achieved differs in each organization, but typically SharePoint is identified using its own SMTP namespace, and a combination of mail exchanger (MX) records and messaging connectors route traffic appropriately.

If an item in the monitored folder can be successfully parsed as an SMTP message, SharePoint uses the To address to look up the destination list or library in the current farm. Depending on various configuration options—such as who is allowed to send mail to a particular list and the type of list it is—the email and attachments are added to that list; they become normal list items and therefore benefit from core SharePoint services such as search and views.

Configuring Incoming-Email Settings
For this emailing structure to be successful, you have to configure how and where the web front-end servers monitor the drop folder, how to match incoming email addresses to lists and libraries, and how to control who is allowed to do what via email. The first step is to enable processing for incoming email at the farm level through the Operations tab in SharePoint Central Administration. With incoming mail enabled, SharePoint starts a background timer job called Windows SharePoint Services Incoming E-Mail that runs every minute on all web front-end servers to poll the drop folder for new messages. As with all timer jobs, this job is controlled by the Windows SharePoint Services Timer service; if the Timer service isn’t running, incoming email won’t be processed.

The incoming-email job runs by default on all front-end servers, but the configuration for the drop folder is set farmwide. Therefore, all web front-end servers in a farm that run the incoming-email service must have a drop folder in the same location. So, if the drop folder is configured as C:Drop, all web front-end servers must have a C:Drop folder and your mail routing topology must ensure that incoming messages get deposited in this folder.

You should enable the Windows SMTP service on your front-end servers to receive email from other SMTP servers— for example, so you can send messages from Exchange 2007 directly into WSS. The automatic configuration mode on the Configure Incoming E-Mail Settings page in Central Administration meets the needs of most organizations. However, if you’re not using the Windows SMTP service and want to populate a drop folder via some other mechanism, you’ll need to manually configure the settings using the advanced mode, which reveals configuration settings for the drop folder in the UI. Note that using the advanced mode disables the ability to specify safe SMTP servers.

In the automatic mode, you can indicate that all incoming mail is acceptable or you can specify a list of IP addresses that equate to those SMTP servers you’re happy to process mail from. Your choice depends on your overall SMTP routing topology. For example, you might have one central SMTP server that all mail passes through and you want this server to be the only one from which you’re willing to process mail. The incoming-email service analyzes the Received header inside each message it picks up from the drop folder to determine whether to accept the message.

If you’re not using the automatic mode, you have to ensure that the account used to run the Windows SharePoint Services Timer service has modify permissions on the specified drop folder so that it can delete messages after processing them. Failing to do so results in messages being delivered multiple times.

The last piece of configuration for this stage is to indicate which SMTP domain your WSS farm processes mail for. For example, with an email address of [email protected] .com, the SMTP domain is wss.spysrus .com. In this example, any email-enabled list or library would have an address of [email protected]. The challenge from an infrastructure point of view is to ensure that every email message addressed to wss.spysrus.com finds its way to a web front-end server with the incoming email service enabled. You could do this many different ways, but it most likely involves setting up MX records in DNS and connectors from other internal mail systems. For example, in Exchange 2007 you could set up a Send connector that handles the wss .spysrus.com namespace and indicate that you want to forward all messages for that address space to a smart host—the smart host being your web front-end server.

Enabling Delivery to a List or Library
After email is configured at the farm level, site administrators can email-enable individual lists and libraries. Not all list types can be enabled for email, but for those that can, you’ll see an Incoming email settings option under the Communications section when viewing the settings for the list. The settings available depend on the list type—for example, a document library asks you what you want to do with attachments and a blog list lets you choose whether the incoming item is published immediately. Figure 1 shows the settings available for a document library.

Every list you mail-enable needs to have an email address that’s unique across the entire farm. But the site administrator can choose only the user part of the address, which SharePoint stores in the AllLists table within the content database associated with the site collection. The domain part of the address is configured at the farm level.

This naming method for email addresses probably isn’t practical for most organizations. Why? First, SharePoint offers no governance on names, so it’s a first-come, first-served situation for picking the user part of the name. And people will most likely choose common names for common lists—for example, what would you want the email address for your Announcements list to be? Announcements@domain.com? But so does everyone with a SharePoint site in the farm, and there’s no way to check what other email addresses are currently in use other than by trial and error—SharePoint blocks you when you try to create a name that already exists. So this leads to end-user frustration. SharePoint also provides no way to validate that the email address conforms to corporate policies for formatting or the use of invalid or inappropriate names.

Continue on Page 2

Furthermore, there’s no way for the sender of a message to know whether it was successfully delivered to the correct SharePoint list—if, indeed, it was delivered at all—except by physically visiting the intended destination. The SMTP service initially receives the incoming email message, but the service isn’t tied to SharePoint in any way and therefore can’t validate that the email address points to a list or library.

The last aspect to mention at this stage is the security that you can set for incoming email. You can choose to accept mail based only on the permissions set on the list or library itself or to accept mail from any sender.

Directory Management Service
If you’ve sorted out your SMTP routing topology, you can now send items to SharePoint by entering the location’s email address in a message. However, raw email addresses aren’t necessarily user-friendly, so most email systems let you look up email addresses using common terms, such as first name and last name. WSS can leverage Active Directory (AD)—or a custom directory— for this very purpose via something called the Directory Management Service (DMS), a web service (SharepointEmailWS .asmx) that’s installed out of the box on a web front-end server and enabled at the farm level using the incoming-email settings. Let’s see how the DMS works and what problems it solves—and what problems it introduces to the mix.

The DMS creates Contact objects in AD to represent email-enabled lists and libraries. (It can also create distribution groups to represent the members of team sites, but that functionality is outside the scope of this article.) Your end users should be able to benefit from these entries with the same address look-up functionality and email distribution as with other email-enabled objects. I say should because enabling the DMS creates some problems you won’t have if you don’t use it. Note that the DMS isn’t mandatory for supporting incoming email.

Configuring the DMS is a matter of specifying an organizational unit (OU) in AD where the contacts will be created. Best practice is to create a MOSS-specific OU for easier all-around management. You need to provide the SharePoint Central Administration application pool account with write access to the OU, and therefore you’ll have to work with whoever’s in charge of your AD. You must also specify the name of the SMTP server for incoming mail— ultimately, this information finds its way onto the email addresses associated with the contacts that the DMS creates. Note that this address can be different from the Incoming E-Mail Server Display Address that you set during incoming-email configuration. The DMS-created address is the more user-friendly one that’s displayed through the SharePoint UI, and you might have to subsequently update the AD-created object with this address so that it’s a valid address to which mail can be routed. You can also indicate whether the AD Contact objects can receive mail only from authenticated users, which results in an Exchange-specific attribute being set on the Contact objects in AD. So, if you’re using a different email system, this setting won’t be applicable.

Using the GAL—or Not
Table 1 shows the main attributes the DMS sets on Contact objects in addition to standard system attributes. There are some limitations with these settings, however, not least the fact that in a pure Exchange 2007 environment, these settings are insufficient for a valid entry to appear in the Global Address List (GAL). Exchange 2003 and Exchange 2000 include an asynchronous component called the Recipient Update Service (RUS) that stamps AD objects with attributes such as proxyAddresses and showInAddressBook that Exchange relies on to identify recipients and route mail. This method of stamping addresses was removed from Exchange 2007; instead, all attributes are stamped at creation time— but only if you use Exchange Management Console (EMC) or Exchange Management Shell (EMS) to create your objects in the first place!

Therefore, to get your contacts correctly stamped in an environment that doesn’t have RUS, you need to modify them after they’re created. This process is discussed in the Exchange team’s blog entry, “Good bye RUS” (msexchangeteam.com/ archive/2006/10/02/429053.aspx). Essentially, you need to update all relevant address lists with the following EMS commands:

Get-EmailAddressPolicy |  Update-EmailAddressPolicyGet-AddressList | Update-AddressListGet-GlobalAddressList |  Update-GlobalAddressList

The second limitation with the Contact objects is that they aren’t configured for attachments to be correctly sent to them. This problem is documented in the Microsoft article “Attachment is missing from an e-mail message that is sent to a Microsoft Windows SharePoint Services 3.0 document library” (support.microsoft.com/default .aspx?scid=kb;en-us;926891), which explains that you have to set the mAPIRecipient attribute to false and the internetEncoding attribute to 1310720 before attachments will be sent correctly to the Contact object. Note that if you aren’t using the DMS to enable GAL lookup, these problems don’t occur. Exchange doesn’t correctly deliver to the DMS-created contact because of these missing attributes.

The third problem is with the display name that gets created and ultimately finds its way into the GAL. This name is usually the primary way someone chooses an entry from the GAL. However, it’s highly likely that users have similar list and site names, which means that there’s little to differentiate entries in the GAL. Remember, Share- Point has nothing in the UI to prevent sites and lists having the same titles. Figure 2 shows an example of two team sites, both called Group Team Site and both having the out-of-the-box Team Discussion list mail-enabled. Which one should users choose? To pick the right one, they’d need to know the email address of the appropriate list—which would defeat the purpose of having a GAL entry in the first place. The existence of Contacts in the GAL could be a hindrance rather than a help.

The last problem I’d like to mention is the lack of governance. There’s no administrative control in SharePoint over what gets put into the GAL—if you decide to enable the DMS, it’s a free-for-all and chaos can certainly reign. Can you imagine if every team site email-enabled, say, three lists? How many entries in the GAL would that equate to in your organization? At my company, that would reach the million mark. And, given the design of the DMS, all those entries would be in the same OU in AD. I’m not sure I know any AD administrators who would happily open up their AD to the potential for such abuse.

Emailing Ease
Correctly configuring all the pieces so that email makes it to the intended list or library can be tricky, but the ability to email items into SharePoint is certainly a useful feature, and the way SharePoint handles incoming items works well. That said, make sure you communicate correct expectations about what the features are, how contacts will appear in the GAL, and so forth, so that everyone—end users, AD admins, Exchange admins—knows what to expect when you email-enable SharePoint. Good communication is especially important if you decide to use the DMS—but remember, the DMS isn’t mandatory.