SharePoint Server 2007 and Records Management: Part 3 of 4

This is the third post in a series on Records Management in SharePoint:

1. Introduction to Records Management, and Definition of Terms

2. Records Management using Standard SharePoint Features

3. Using the DoD 5015.2 Records Management Resource Kit (this post)

4. Commonly Requested Enhancements and Features 

In the previous posts in this series I examined what Records Management is, and how standard SharePoint Server 2007 functionality can be used to implement a basic Records Center. This post discusses a publicly-available add-on pack for SharePoint, the DoD 5015.2 Resource Kit.

Using the DoD 5015.2 Resource Kit

The DoD 5015.2 Resource Kit was created to demonstrate that custom Records Management solutions built using the MOSS platform could be made to be compliant with legal standards. Increasingly the most common standard, and the one chosen for certification by Microsoft, is the United States Department of Defense (DoD) 5015 Chapter 2 standard. The certification against this standard was achieved in May 2007 with a solution built using the Resource Kit created for Microsoft by a Microsoft Partner. The certification allowed Microsoft to position SharePoint as a valid solution for US (and often other government) Records Management requirements, and serves as a good yardstick as to the capability of the SharePoint platform to be extended to meet differing customer requirements and scenarios.

An evaluation version of the Resource Kit is available for Microsoft Partners or customers to download from the Microsoft website, but is only available commercially as a production solution by engaging Microsoft Consulting Services or a Microsoft Partner. This is because Records Management implementations are complex and should only be implemented by teams with experience in designing and implementing complex records management solutions.

• • The Resource Kit provides additional features that build on the default SharePoint records management capabilities and extend the standard functionality to include several important new areas. Although there are many enhancements that the pack provides on top of SharePoint, let’s take a look at the most important new additions:Enhanced file plan management
  • Enhanced retention
  • Enhanced disposition
  • Enhanced search
  • Clearance-based access control
  • Related versions and records

DoD Enhanced File-plan management

The DoD Resource Kit allows for better management of a hierarchical file plan. Specifically, it allows the creation of a tree-like folder structure, with the ability to set metadata and control other functionality on either the top-level category, or lower level folders.

The file plan consists of two primary entities:

Categories , or top-level folders. Categories are represented in SharePoint as Document Libraries and serve as the actual physical repositories for the records. Several features can be controlled at the category level, including

· Disposition instructions (including Retention)

· Vital Record Review period and reviewer

· Cutoff instructions

· Metadata such as category unique id and description

Record folders, or lower-level folders. Record folders are represented in SharePoint as folders within the document libraries that represent categories. Features that can be controlled at the record folder level include

· Vital Record Reviewer and Review period

· Whether the folder is open to records or closed

· Supplemental Markings

· Cutoff Trigger

· Metadata such as unique ID

The enhanced file plan management of the DoD Resource Kit allows for much greater control over the structure and order of the file plan. The control over items such as Vital Record Review Period on a folder level allows different content to be grouped by folder and have some of the functionality of cutoffs and retentions applied at that level.

Enhanced Retention

The DoD Resource Kit improves the flexibility and control exercised over the retention of records, specifically allowing for greater control over the triggers for the cutoff of a category or folder, and allowing for event-based disposition of categories. To create a trigger for a cutoff, a new Global Event or Global Period is created. As many of these as necessary can be created and used as the triggers for cutoff of a category or folder.

Global Event

Global Events allow for an event-driven cutoff. For example, an event could be “Termination of employment of John Smith”, and the cutoff could start 2 years after the event is triggered.

Global Period

A global period represents a repeating, set period of time. For example, a global period could be “Financial year”. A global period can be created to represent any period of time.

For triggering a disposition, a Global Event is used, which allows for control over when records are disposed of on a category level. Optionally, a field on a record can be used to decide when disposition occurs – for example, “ten years after record create” can be set as an instruction.

Enhanced Disposition

The DoD Resource Kit allows for greater disposition control than standard SharePoint. Specifically, the pack allows the creation of a sequence of disposition instructions which are followed when the disposition is triggered. The actions are applied on the category level, which means that different top-level folders in the file plan can have different disposition instructions applied to them, and generally trigger a workflow which, on approval, causes the instruction to be applied.

Actions

The disposition instructions that can be applied are

· Destroy

· Transfer

· Destroy After Transfer

· Do Nothing

A Destroy instruction results in the Destroy workflow initiating, in which the record is reviewed and deleted if the destroy is approved. Transfer results in the export of the records to a specified location in a packaged file, and Destroy After Transfer will export and then delete the records.

Sequence

A series of disposition instructions created for a category, which are executed in sequence. For example, the instructions for a category could be

1. Transfer 5 years after event X

2. Destroy 10 years after event x

The events are followed in sequence. The sequence does not necessarily need to follow logically, for example the first instruction in a sequence can be a Destroy instruction. When the workflow for the Destroy instruction is triggered, the record manager can decline the destroy, at which point the next instruction is followed.

Triggers

There are two triggers that can be used to initiate a disposition instruction

Global Event Trigger, where the disposition is triggered based on the occurrence of a Global Event. An example could be the closure of a project or the end of a pending court case.

Record event trigger, which allows the disposition to be initiated on an individual record basis, triggered by a date field on the record. An example of this type of Instruction would be “Destroy ten years after the record was uploaded”. One of the most commonly used fields for this type of trigger is the Cutoff Date which, in combination with the Cutoffs specified on the category or folder, allows for folder-level control of the disposition of records

Enhanced Search

The DoD Resource Kit supplies its own specialized search user interface for records managers, adding some new features which allow for greater property-level searching of records, and the export of the search results into Excel. This provides a basic reporting structure for the records managers, allowing them to search through records based on various properties, and export those results.

This search does not interfere with the standard SharePoint search, and both can be configured side-by-side.

Clearance-based access control

The DoD Resource Kit adds a useful tool for controlling the access to records within the Records Center, based on clearance level. For example, a clearance level of “Top Secret” can be placed on records, and only those users in the “Top Secret” group have access to the record and can change the permissions on the Record.

Although utilizing standard SharePoint permissions under the hood, the feature allows for several levels of clearance/categorization to be applied to a single record, and the intersection of users in these groups is used to determine the Access Control List for these records. For example, tags that could be applied to a record include “Top Secret”, “US Eyes Only” and “Department of Defense Only”. With these tags applied to a record, only those users in all of these groups would have access to the record. This is contrasted to normal permissions in SharePoint, which allows a user in any of the applied groups access to the item.

Related versions and records

The DoD Resource Kit allows for related versions of a record to automatically be linked together. If a record is declared, and the source document is subsequently modified and then re-declared as a record, the pack will automatically link the two records together. In addition, when the earlier record is opened, the pack will warn the user that a newer version exists and allow the user to view that version.

In a similar way, records can be related to each other using a custom-created “record relationship”. An unlimited number of Record Relationship Templates can be created and used. By default, the templates available are Succession, Rendition and Attachment, although this list can be customized and extended.

When viewing the properties of a record, the versions and related records are visible to the user and can be browsed.

DoD Design considerations

The DoD Resource Kit is extremely useful in extending the built-in records management functionality in SharePoint. However, there are some potential (and some deliberate) design constraints that implementers should be aware of.

Routing

The DoD Resource Kit assumes that records will be manually uploaded into the correct location in the file plan by browsing the Record Center. Although the automated routing of SharePoint can be used with the DoD Resource Kit solution, much of the metadata requires manual updating. Practically, this means automated routing of documents is quite tricky, and must be carefully planned (and may require additional custom development).

Clearance level Security

The DoD Resource Kit allows for the adding of protective markings such as “Top Secret” to records, which limits their permissions based on group membership.

However, technically this mechanism works by analyzing and applying the individual SharePoint users that have been granted access by name as individuals to all of the clearance level tags, and then reducing the list of SharePoint users that have been found to be in all applied tags.

Practically, in most cases this means that the users that are in each of the allowed groups for tags must be named SharePoint users. The relatively standard practice of managing permissions via Active Directory groups may result in unexpected results, as the DoD Resource Kit solution is unable to expand out the AD group membershipwhen performing the required group membership intersections. As such, if Active Directory groups are to be used, careful planning and testing must be undertaken. If possible, use individual named SharePoint users when managing permissions within the Record Center to avoid these issues.

Folder-level retention

Practically, it is difficult to apply folder-level retention periods, disposition triggers and disposition actions in a DoD Resource Kit solution. For designs where the disposition can be defined on the category level, this is not a limitation. However, designs must be careful not to build folder-level dispositions in to the system or to design careful processes around folders if this is necessary.

Definition of control structures

Many of the control functions within the DoD Resource Kit, such as Global Events, are list items in specialized lists within the SharePoint site.  This means that care must be taken that the number of items in these lists does not exceed the recommended software boundaries for SharePoint (approximately 2000 items per view) to avoid gradual performance degradation. For example, if a folder for an employee must be cut off when the employee leaves the company, then a Global Event must be created for every single employee in the company, which may exceed the 2000 item per view performance boundary guideline. This takes careful planning to implement correctly.

Bulk Workflow Processing

The disposition workflow process for records review and disposition is applied at the Record level in the DoD Resource Kit (and standard SharePoint). This means that when a folder expires, if it has a thousand items in it, then a thousand workflow tasks are created for the administrators. Although bulk processing of tasks can be achieved, the DoD Resource Kit does not apply fields such as folder name to the tasks, and so this functionality must be carefully analyzed when implementing the pack.

Scalability

The DoD Resource Kit follows the implementation of a standard SharePoint Records Center, in that typically only one Records Center would exist at any one time – it is technically possible to create many separate Record Centers sites, but they would be separate entities, managed separately, not sharing a file plan by default and certain other features such as automatic version relations would not work automatically.

Records Centers are also subject to other standard SharePoint planning constraints; for example a site collection (or Records Center) is stored within a single contentdatabase, which has a recommended maximum size of 100GB. This size guideline is intended primarily to keep database backup and restore times within typical IT Department Service Level Agreements (SLAs) and so is arguably a flexible number, depending on the specific SLAs in place at a specific customer and on the tools and techniques they use to backup and restore databases. However Microsoft’s standard guidance is to keep individual content databases to 100GB in size.

Conclusion

The DoD 5015.2 Resource Kit provides many additional records management features and controls that are required to meet the DoD 5015.2 standard. It is a great tool for extending the SharePoint functionality and as long as careful design and planning is undertaken, can meet many of the ERM requirements for medium to large enterprises that need to comply with the DoD 5015.2 standards.

However, implementing the solution is not a trivial exercise and must be undertaken by engaging with either Microsoft Consulting Services or an appropriate Microsoft Partner. Customers who would like to enhance the records management functionality of SharePoint 2007 with particular features available within the DoD 5015.2 Resource Kit but are not required to run their system in a certified configuration should not use the DoD 5015.2 Resource Kit. Alternatively, sample code and documentation is available for the most frequently requested features available within the DoD 5015.2 Resource Kit. For a list of the available samples and documentation please refer to this page: https://sharepoint.microsoft.com/product/capabilities/ecm/Pages/dod-resource-kit.aspx

In Part 4 of the blog series, we examine how the SharePoint platform can be extended with additional records management functionality via further custom development.

 This post was written by Peter Reid and modified by James Kemp and Graham Tyler: