Monitor SharePoint User Profile Changes

SharePoint Server 2010 is a great platform for storing information about the users in an organization. Using the User Profile Service application and its related components, you can configure SharePoint to store a variety of information about users and synchronize that information with external storage, such as Active Directory (AD) or any other data source to which you can connect using Business Connectivity Services (BCS).

Users can then edit their information using the user profile pages. The external storage sources will then be updated automatically with this new information, thereby making the information more relevant and timely throughout the organization.

However, allowing users to update their own information introduces new problems when it comes to governance, particularly in terms of security of information, privacy, and ethical conduct. Fortunately, one component of the User Profile Service application is the User Profile Change Service.

The User Profile Change Service isn’t a true service but rather a series of tables within the User Profile Service’s profile database and a timer job (i.e., User Profile Change Cleanup Job) that’s used to clear stored changes older than 14 days.

By default, the User Profile Change Cleanup Job runs once a day. You can adjust this frequency by editing the schedule for this timer job.

You can use the User Profile Change Service to view and report on changes. Unfortunately, however, there isn’t a UI to view these changes, so you must use the SharePoint API to query the system to view changes.

But what kind of changes can you query the system for? The answer is simple: all user profile properties.

You can see the properties that are available by navigating to the Manage Service Applications page in the Central Administration site, selecting a User Profile Service application, and clicking Manage to view the User Profile Management page. (If a User Profile Service application doesn’t exist, you must create one in order to store and manage user details.)

Click the Manage User Properties link to see all the available properties. From here you can create, delete, or edit any properties that you want your users to populate with data. When a user edits any of the values for one of these properties, a change record will be created.

However, it’s important to remember that changes will persist for only 14 days, assuming the User Profile Change Cleanup Job is running. If you need to keep these changes longer, you can change the retention period by running a simple STSADM command. (There’s no Windows PowerShell equivalent way to do this.) The following example changes the retention period to 28 days:

stsadm -o profilechangelog  -userprofileapplication "My User Profile Service App"  -daysofhistory 28

(Although the command wraps here, you'd enter it all on one line. The same holds true for the other commands that wrap.)

You can gain access to these change records using the objects in the Microsoft.Office.Server.UserProfiles namespace, which is part of the Microsoft.Office.Server.UserProfiles.dll assembly.

Specifically, you use the UserProfileManager, UserProfileChangeQuery, and UserProfileChangeToken classes.

Retrieving User Profile Changes

To retrieve user profile changes, you must first create an instance of the UserProfileManager class. This class is the gateway for all programmatic manipulations of the user profile properties and their properties.

To create an instance of this class, you must provide it with a Microsoft.SharePoint.SPServiceContext object. This context object makes sure that you’re working with the correct User Profile Service application.

There are two ways to retrieve the SPServiceContext object:

Listing 1 at the end of this article demonstrates the first approach using PowerShell. Alternatively, you could use code in which the URL to the site collection is passed to the GetContext method. Either approach is perfectly acceptable.

Now that you have the SPServiceContext object, you can create the UserProfileManager object:

$manager = New-Object Microsoft.Office.Server.UserProfiles.UserProfileManager $context

The UserProfileManager class contains the GetChanges method. This method has three overloads:

All the GetChanges overloads return a UserProfileChangeCollection object. This object contains all the changes stored as either a UserProfileChange object or one of its derivative types:

Each object type stores the resultant change in a manner most appropriate for the given type. It’s important to note that a separate object will be returned for every change.

So, if a user edits two profile properties, you’ll have at least two objects (one for each property). As the phrase “at least” implies, you could have more than two objects.

For example, if the user edits the Skills property by adding several skills, a separate change object will be returned for each skill added (i.e., one object for each value entered).

Using the GetChanges Method

Let’s look at examples of how to use each overload for the GetChanges method. Figure 1 shows an example of running the GetChanges() method without any parameters.

Running the method in this manner will return all changes in the log, which can be quite numerous.
 

Figure 1: Example of the GetChanges() method

You can reduce the number of changes returned by passing in a UserProfileChangeToken object to the GetChanges method. This object lets you retrieve changes starting from a specific date.

To do so, you simply specify that date in the change token.

You can also retrieve all the changes created after a certain change event. In this case, you need to specify an arbitrary date and the change event’s ID because there’s no constructor available that accepts only an event ID. Only the event ID will be used; the date will be ignored.

Figure 2 shows an example of running the GetChanges method when you pass in a UserProfileChangeToken object with both the event ID (value of 32) and date ($date, whose value is 1/31/2011) set.

Alternatively, you could provide a value of [System.DateTime]::Now for the $date variable, as the value itself is irrelevant when the event ID is provided. (Note that the UserProfileChangeToken object values can only be set by initializing the object through its constructors.)
 

Figure 2: Example of the GetChanges(UserProfileChangeToken) method

Filtering in this manner, however, can still result in a large number of records returned, particularly if you’re only interested in a specific type of change. To address this concern, you can use the third GetChanges overload and pass in a UserProfileChangeQuery object.

This object gives you the ability to specify exactly what type of changes you want returned. For example, perhaps you only want to know when single-valued properties (such as the AboutMe property) are added or updated. Using the UserProfileChangeQuery object, you can specify what property types and change types you care about.

The UserProfileChangeQuery object’s constructor lets you quickly set all the filtering properties to false. You can then enable just the properties of interest by setting them to true.

For example, the code in Figure 3 sets all the properties to false when it initializes the UserProfileChangeQuery object, then sets the SingleValueProperty, Add, and Update properties to True.


Figure 3: Example of the GetChanges(ProfileBaseChangeQuery) method

As you can see, retrieving user profile changes is fairly simple, requiring knowledge of only a few classes and their various members. With PowerShell, you can easily format the returned data a variety of ways. For example, Figure 4 shows Figure 3’s query results formatted into tables and grouped by account.
 

Figure 4: Example of formatted query results

Empowering Privileged Users to Monitor Changes

Using PowerShell to retrieve information about user changes can be extraordinarily powerful and useful to administrators. But to enforce governance policies, you must bring the ability to monitor changes to privileged users (e.g., HR staff members) and give them the responsibility and authority to properly act on this data.

Let’s examine a real-world example that accomplishes the goal of empowering privileged users.

Suppose that an organization’s HR department has employee policies set in place that govern the communications of employees for email and other types of communication. These policies are extended to the communications in employees’ personal sites and their user profile information.

The HR department wants to monitor updates to visible fields like About me to ensure that the employees are complying with the policies. Further, it wants to have the freedom to review the changes from a central location and receive an alert when a new change has been posted to the change log.

The HR department’s requirements are easy to achieve if you consider that once you’ve gathered the changes and stored them in a list that HR staff members have access to, they can manage the information and set alerts using the list’s functionality. The only customizations you need to write are:

In this example, the list resides at the root of the My Site host, but the list could reside anywhere in a real implementation.

Creating the Timer Job

Let’s walk through the code that creates the SharePoint timer job to see how it works. This example is based on the Microsoft article “Creating Custom Timer Jobs in Windows SharePoint Services 3.0”.

The timer job begins by creating an instance of the UserProfileWatcherTimerJob class, which inherits from the SPJobDefinition class.

The SPJobDefinition class has a property bag that lets you provide properties to the Execute method for processing. In this case, the SubmitJob method is used to pass the userProfileFields and mySiteHost properties to the timer job instance and store them in the property bag, as you can see in the code that Listing 2 at the end of this article shows.

The Execute method is called when the timer job runs. As seen in Listing 3 at the end of this article, the Execute method begins by reading the values of the timer job’s properties. Next, it creates an instance of the worker class, UserProfileWatcherWorker, passing in those values.

UserProfileWatcherWorker then provides two methods, which the Execute method uses to query the user profile change log and update the target list when changes matching the query are found.

UserProfileWatcherWorker is the workhorse of the timer job, so let’s take a close look it. This worker class first declares several private fields to keep track of the user profile fields and the age of the changes. It also declares a list to store UserProfilePropertyValueChange objects:

public class UserProfileWatcherWorker{  private string _mySitesHost;  private int _changeAge = 30;  private string _profileFields;  private List _changes =    new List(); }

Next, it initializes the values that will be used to get the UserProfileManager object and construct the change query:

public UserProfileWatcherWorker(string profileFields, int queryInterval, string mySitesHost) {  _mySitesHost = mySitesHost;  _changeAge = queryInterval;  _profileFields = profileFields;}

Finally, the UserProfileWatcherWorker worker class provides two methods:

Creating the Administrative UI

Administration of the timer job is achieved through the use of a custom administration page that Figure 5 shows. The administration page, which inherits from the OperationsPage class, is added to the solution in an ADMIN mapped folder.

Figure 5: Administration page

Navigation to the new administration page is achieved by adding an elements.xml file that contains the CustomAction element code in Listing 6 at the end of this article. The CustomAction element adds a new menu item—Manage Aptillon SharePoint User Profile Watcher—to the Timer Jobs section of the Monitoring page, as seen in Figure 6.

Figure 6: New link to the administration page

When the new menu item is clicked, the administration page loads. During the loading process, the code shown in Listing 7 at the end of this article looks for a previous instance of the timer job and initializes the values of the page controls if a timer job is found.

In the administration page, the privileged user or administrator can choose to disable or enable the timer job. When a person selects Disabled and clicks OK, the OnClick method gets a reference to the SharePoint Timer Service and deletes all previous versions of the timer job. This prevents multiple instances of the timer job.

When the privileged user or administrator selects Enabled, enters the necessary information, and clicks OK, the administration page runs code that tests for the existence of the specified site and creates a schedule object based on the specified settings.

A new instance of the UserProfileWatcherTimerJob is created and the SubmitJob method is called to update the properties, which the code in Listing 8 at the end of this article shows. The privileged user or administrator is then redirected to the Monitoring page in Central Administration.

Testing the Sample

Testing the sample code requires you to have administrative access to your SharePoint farm.

After deploying the code, select the new Manage Aptillon SharePoint User Profile Watcher menu item in the Timer Jobs section of the Monitoring page in Central Administration.

On the administration page, enable the timer job, make sure the Profile Fields value is AboutMe, and set the time value to 1 minute for testing purposes. Supply the address of your farm’s My Site host and click OK. You’ll be returned to the Monitoring page in Central Administration.

Figure 7: Edit the user profile’s About me field

Open your profile page and choose Edit My Profile. Edit the About me field, which Figure 7 shows, choose Save, then click Close.

After the time set on the administration page has passed, navigate to the My Site host and click View All Site Content. There will be a new list named User Profile Changes. When you view that list, you’ll see a copy of your change, as Figure 8 shows.

Figure 8: Review the change made to the About me field

A Powerful Tool

The user profile change log is a powerful tool for monitoring change in your environment and can be used to help enforce governance policies for your My Site deployment.

Although SharePoint doesn’t provide a UI for the user profile change log, the SharePoint API provides great functions for accessing and using it to monitor user activity. You can also use PowerShell to access and monitor the change log.

Listing 1: Code to retrieve the SPServiceContext object

$app = Get-SPServiceApplication | ? {$_.Name -eq "My User Profile Service App"}$siteSub = [Microsoft.SharePoint. ?   SPSiteSubscriptionIdentifier]::Default$context = [Microsoft.SharePoint. ?  SPServiceContext]::GetContext( ?    $app.ServiceApplicationProxyGroup, $siteSub)

Listing 2: SPJobDefinition code with properties added

public class UserProfileWatcherTimerJob : SPJobDefinition{  public UserProfileWatcherTimerJob() {}  ///   /// Initializes a new instance of the  class.  ///   public UserProfileWatcherTimerJob(SPService service): base(Constants.UserProfileWatcherJobName, service, null, SPJobLockType.Job)  {    Title = Constants.UserProfileWatcherJobName;  }  ///   /// Submits the job.  ///   public void SubmitJob(string userProfileFields, string mySitesHost, SPSchedule schedule)  {    Properties[Constants.PropertyKeyProfileFields] = userProfileFields;    Properties[Constants.PropertyKeyMySitesHost] = mySitesHost;    Schedule = schedule;    Update();  }

Listing 3: Execute method

public override void Execute(Guid targetInstanceId){    string profileFields = Constants.ProfileFields;    int queryInterval = 30;    string mySitesHost = String.Empty;    Debug.WriteLine("[Aptillon] User Profile Watcher Job: Begin Execute.");    try    {      //Get the properties from the Timer Job      profileFields = this.Properties[Constants.PropertyKeyProfileFields].ToString();      queryInterval = (this.Schedule as SPMinuteSchedule).Interval;      mySitesHost = this.Properties[Constants.PropertyKeyMySitesHost].ToString();      Debug.WriteLine(String.Format("[Aptillon] User Profile Watcher Job: Fields={0}, Interval={1}, MySites Host={2}", profileFields, queryInterval, mySitesHost));    }    catch (Exception ex)    {      Debug.WriteLine("[Aptillon] User Profile Watcher Job: Error gettting Job Properties");      Debug.WriteLine("[Aptillon] User Profile Watcher Job: " + ex.Message);    }    //Create an instance of the watcher class    UserProfileWatcherWorker changes = new UserProfileWatcherWorker(profileFields, queryInterval, mySitesHost);    //Get the changes from the log    changes.RetrieveUserProfileChanges();    //Process the changes and add them to the list    changes.LogProfileChange();  }}

 Listing 4: RetrieveUserProfileChanges method
 

internal void RetrieveUserProfileChanges(){  Debug.WriteLine("[Aptillon] User Profile Watcher Job: Starting Retrieve");              using (SPSite site = new SPSite(_mySitesHost))  {    SPServiceContext context = SPServiceContext.GetContext(site);    UserProfileManager profileManager = new UserProfileManager(context);    DateTime tokenStart = DateTime.UtcNow.Subtract(TimeSpan.FromMinutes(_changeAge));    UserProfileChangeToken changeToken = new UserProfileChangeToken(tokenStart);    UserProfileChangeQuery changeQuery = new UserProfileChangeQuery(false, true);    changeQuery.ChangeTokenStart = changeToken;    changeQuery.UserProfile = true;    changeQuery.SingleValueProperty = true;    changeQuery.MultiValueProperty = true;                    UserProfileChangeCollection changes = profileManager.GetChanges(changeQuery);    Debug.WriteLine(String.Format("[Aptillon] User Profile Watcher Job: found {0} changes.", changes.Count));    foreach (UserProfileChange change in changes)    {      if (change is UserProfilePropertyValueChange)      {        UserProfilePropertyValueChange propertyChange = (UserProfilePropertyValueChange)change;        if (_profileFields.Split(',').Contains(propertyChange.ProfileProperty.Name))        {          _changes.Add(propertyChange);        }      }    }    Debug.WriteLine("[Aptillon] User Profile Watcher Job: Done with Retrieve");  }}

Listing 5: LogProfileChange method

internal void LogProfileChange(){  if (_changes.Count == 0) return;  SPList list = Utilities.FetchOrCreateList(_mySitesHost);  if (list == null)    return;  try  {    foreach (UserProfilePropertyValueChange change in _changes)    {      UserProfile profile = change.ChangedProfile as UserProfile;      if (profile == null) continue;            try      {        Debug.WriteLine(string.Format("[Aptillon] User Profile Watcher Job: Adding change details - Account={0}, Property={1}", change.AccountName, change.ProfileProperty.Name));        SPListItem item = list.AddItem();        item["Title"] = String.Format(""{0}" changed "{1}"",profile[PropertyConstants.PreferredName].Value, change.ProfileProperty.Name);        item["PropertyName"] = change.ProfileProperty.Name;        item["PropertyValue"] =            profile[change.ProfileProperty.Name].Value;        //Log the user too        SPUser user = list.ParentWeb.EnsureUser(change.AccountName);        item["User"] = user;        item.Update();        Debug.WriteLine("[Aptillon] User Profile Watcher Job: Add Complete");      }      catch (Exception ex)      {        Debug.WriteLine("[Aptillon] User Profile Watcher Job: Error gettting Job Properties");        Debug.WriteLine("[Aptillon] User Profile Watcher Job: " + ex.Message);      }    }  }  finally  {    list.ParentWeb.Dispose();    list.ParentWeb.Site.Dispose();  }}

Listing 6: CustomAction element
 

Listing 7: Code to look for previous timer job instances
 

public partial class UserProfileWatchSettings : OperationsPage{  protected void Page_Load(object sender, EventArgs e)  {  if (!IsPostBack)  {    jobStatusList.SelectedValue = "False";    userProfileFieldsTextBox.Text = Constants.ProfileFields;    intervalMinutesDropDownList.SelectedValue = "30";    mySitesHostTextBox.Text = String.Empty;    SPTimerService timerService = SPFarm.Local.TimerService;    if (null == timerService)    {      throw new SPException("The Farm's timer service cannot be found.");    }    foreach (SPJobDefinition oldJob in timerService.JobDefinitions)    {      if (oldJob.Title == Constants.UserProfileWatcherJobName)      {        jobStatusList.SelectedValue = "True";        userProfileFieldsTextBox.Text = oldJob.Properties[Constants.PropertyKeyProfileFields].ToString();        intervalMinutesDropDownList.SelectedValue = (oldJob.Schedule as SPMinuteSchedule).Interval.ToString();        mySitesHostTextBox.Text = oldJob.Properties[Constants.PropertyKeyMySitesHost].ToString();        break;      }    }  }}

Listing 8: Code to update properties
 

protected void SetTimerJobsButton_OnClick(object sender, EventArgs e){  SPTimerService timerService = SPFarm.Local.TimerService;  if (null == timerService)  {    throw new SPException("The Farm's timer service cannot be found.");  }  // delete the job for the current web application  foreach (SPJobDefinition oldJob in timerService.JobDefinitions)  {    if (oldJob.Title == Constants.UserProfileWatcherJobName)    oldJob.Delete();  }  // Enable the Timer Job if the enabled button is chosen  if (jobStatusList.SelectedValue == "True")  {    string mySitesHost = mySitesHostTextBox.Text;    if (!SPSite.Exists(new Uri(mySitesHost)))     {      ErrorMessageLiteral.Text = "The specified my site host does not exist.";      return;    }    // create a new instance of the job and schedule it    SPMinuteSchedule schedule = new SPMinuteSchedule();    schedule.BeginSecond = 0;    schedule.EndSecond = 59;    schedule.Interval =     Convert.ToInt32(intervalMinutesDropDownList.SelectedValue);    UserProfileWatcherTimerJob job = new UserProfileWatcherTimerJob(timerService);    job.SubmitJob(userProfileFieldsTextBox.Text, mySitesHost, schedule);  }  this.RedirectToOperationsPage();}