Making sense of a multi-workload ECM SharePoint installation, Part 2, Planning

Introduction – Review of Part 1

Wow, nearly a year without writing, time goes way too fast!  In the first article, the objectives were to properly explain the high level terms that are related in deploying SharePoint in the field: Enterprise Content Management, Document Management, Collaboration, Records Management, eDiscovery,and Web Content Management.  It is necessary to understand these concepts as they directly relate to what you are trying to do with SharePoint. 

While it is typical to have multiple products, by understanding how each of these areas interact between each other is critical as you will have to either integrate, or migrate.

The goal isn’t about technology, it’s about defining the right scenarios and focusing on bringing solutions to these – this will equal to adoption.

Alright, I get it, Scenarios, Scenarios, Scenarios.  But what are the SharePoint workloads?

There’s a lot of viewpoints for this, but the typical ones are about Collaboration, Enterprise Content Management, Social, Search, Web Content Management, and Business Intelligence.  Let me try to break these down a bit.

Collaboration – This is about capturing all sorts of information right up from the end-users.  This can be in various formats: documents, micro-blogging questions & answers, emails, structured web pages (i.e.: page layouts), unstructured web pages (wiki), opinions through blogs, forums, etc.  Remember: think scenarios!  When elaborating collaboration scenarios, they tend to be split between Team-Based Collaboration, and Document-Centric Collaboration (often referred in ECM, or as a subset in ECM).

Enterprise Content Management – This is your large document stores.  In my experience, this essentially come in 2 types of usage with SharePoint.  (1) As a document center for some document-centric departments (HR, Legal, Finance) where these end-users are working directly in these workspaces for storing and finding back documents.  (2) as an archive where you’d like to massively store documents with no collaboration and limited read operations.  These files may be uploaded by a system (i.e.: financial/insurance documents), or through a process for when collaboration has ended on some document types.

Social – This is an expanding one.  Today, this is about enterprise micro-blogging, colleagues and following them, finding expertise, and following communities.  This is impacting how we build these communities, forums, and even simple things such as Questions & Answers.  The main goal is to allow for people to quickly share information, and to have a broader reach for getting answers.

Search – Most people defines search as that little textbox that allows you to perform keyword searches.  While this is certainly true, there’s more to this than meets the eyes.  With proper planning, you can create search-driven applications. For example, why would potentially create an HR resume Document Center along with a Finance PO Document Center; compared to, say, a Record/Document Center containing both?  The reason would likely be because your usage scenarios are very different and driven by customer requirements – automatic search-driven pages makes it more compelling for end-users to adopt the system.  Enterprise Search is also more than simply crawling all your data sources – it’s about making relations between them and about automatically categorizing the information (automatic metadata entry based on patterns).

Web Content Management – This is your typical web site for broad audience.  However, trends are changing this space.  For the public Internet areas, social is impacting these web sites.  You may need to allow people to provide feedback and respond to them, it could be end-user tagging & ratings.  For intranet sites, social and collaboration are impacting what you need to present as your end-users requires to have centralized dashboards for both the enterprise/CEO messages, along with their tasks, community messages, etc.

Business Intelligence – BI is a world in itself and is completely driven by usage scenarios.  SharePoint 2010 offers different BI capabilities that can meet most, if not all, of the end-user scenarios through the help of SharePoint Designer, PerformancePoint, Excel, and other surrounding products and technologies.

Define your general enterprise strategies

Collaboration Strategy

Collaboration is typically where most of your information entry will be done – unless the information is coming in from automated means (scans, faxes, etc.).  The important part here is that there’s an end-user involved and adoption will be determined on how easy and intuitive it is for that user.  A good collaboration strategy starting point is to address what kind of initial solutions will be provided (team sites, project team sites with specific metadata, meeting boards, etc.)  -- this isn’t so much about features but rather high-level generic end-user solutions.

The 2nd element I’d strongly suggest is the surrounding services framework such as providing a solution for Site metadata.  This provides a good solution to find back sites, but also to automatically tag information based on these site metadata.  I also look at providing a framework for communicating change back to users, such as new solutions or features, upcoming maintenances, etc.

3rd, I’d include some sort of collaboration portal which will act as the starting point for creating a collaboration workspace, providing information on how to use the platform and what the solutions are, what the policies are, etc.  This area must also provide an area for end-users to provide feedback and request assistance.  If there are business requirements leading to it, it’s okay to have templates for “Generic Project Site”, “IT Project Site”, “Marketing Project Site” – the reason for this is that they may have different types of content, policies, workflows, and metadata associated to them.  By doing this, you make it easier for end-users to comply to the corporate policies and making it easier for them to use the system.

Last, I’d consider levering the site metadata solution (#1) to include site security classification.  At the site creation, with simple questions regarding what the site may contain, you could automatically apply infrastructure security parameters depending on whether it will host (for example) public, secret, or top secret information.  Your security compliance requirements may define that top secret information be on a segregated network, uses encrypted database, and requires a 2-form authentication.  Also, you may need to tag all information (in the records management scenarios) with confidentiality parameters – by doing it at the site level, you could automatically tag all the information without requiring end-user input.

Collaboration is where SharePoint has excelled and brought awareness to end-users of the other workloads.  Its integration point with other Microsoft points are numerous and essential to end-user adoption.  This is also where we have a typical gap with traditional ECM products that are mostly document-centric collaboration, often without an intuitive interface for most end-users.

Note that SharePoint is designed for your to maximize the number of Site Collections, but also encourage you to have some sort of site collection retentions.  Putting it simply, create a Site Collection for a specific but unique purpose.  For example, you would create a Site Collection for each IT projects, and maybe another Site Collection for the IT Department which would list all of their projects through site metadata and search.  Each projects have a different site collection because they have a different retention than the next project, different members and security, quotas, etc.  You could share information policies and content type through the Content Type Hub for example.

However, You would not create an IT site collection with sub-sites for each project as this would make it much more difficult to maintain and operate.

ECM and Records Management

Your general ECM strategy must define the high level flows of information, compliancy requirements, and information security requirements.  Note that this isn’t just for documents but for all types of information (paper, IM, social, etc.).  The following could be an example of a high-level information flow for a company using SharePoint, SAP, Documentum, and another tool for archival:

Records Management is your typical deep tree (folders) of documents/information kept for retention and compliancy purposes.  The scenarios are typically “record keeper driven”.  If all end-users have access, it’s likely through search.  The goal is not to have end-users enter the content manually there – they generally get lost in the folder structure and that defeats the purpose. 

Instead, the document is routed (automatically for known content, at the right time; or manually for selected content by end-users) to a drop folder which, based on metadata rules, will move the document to the right folder.  This ensures that the record center is kept optimal for storage and adequate performance – while following the compliance requirements, without impacting end-user adoption.

SharePoint provides this capability with the “Send to” configurations which can be defined per Web Applications, and attached on content types and/or information policies.  This “Send To” effectively provides the capability to Copy, Move, or Move and leave a link.  The later automatically tags the document with a unique identifier providing access to the document from anywhere (transparently provided by the SharePoint/FAST search engine).

While this warrant a specific post on just that subject (and I’ll try to do it soon, promise!), integration between ECM products must be looked at properly.  Of course, all of them claims to integrate with SharePoint, but that depends on your interpretation of what integrating means.  In my opinion, Web Parts are NOT integration (it’s more like a window to looking outside, or to some other black box).  While they can fulfill some scenarios, they do not constitute a proper end-user integration. 

From a high level view, if you want SharePoint as the front-end and also have other ECM products such as OpenText LiveLink or EMC Documentum, you cannot simply state that ‘all document will be in Documentum’, install web parts or one of their respective deeper integration product, and be done with it.  Before integrating, you’ll have to define what and when integration will occur.  By far, these products are mostly designed as document-centric collaboration and/or for retention archives.  The challenge in integrating these will be on the configuration of this what and how – this is no easy feat.  In fact, typically, when information stems from end-user driven collaboration, it’s easier to use the SharePoint “Send To” features to send it to the SharePoint Record Center (using the drop folder, or using the content organizer altogether) in between, and then integrate that with the other ECM product.  Of course, you’ll have to ask yourself why you are using 2 tools at that point – but, in my experience, it’s been (by very far) the easiest/lowest impact to the end-user/cheapest way to integrate them.

As a final note regarding ECM integration, for this post, it should be driven by the archival users requirements, not specifically by IT.  Compliance requirements starts before the information is entered through SharePoint so you will likely have to define the information rules both in SharePoint and that other tool.  If you have end-user requirements, they will be done through SharePoint and most of these do not work well if SharePoint is not the back-end.  Be very careful, complete end-to-end integration with other ECM products is not easy, if possible.  It is possible that future implementation of CMIS may help, but it's no silver bullet today.

ECM integration, in my opinion at this time, should be kept for specific departments that already have document-centric collaboration scenarios with that other ECM product.  For example, Finance already uses Documentum and would like to have SharePoint as a front-end and for other features.  You could then add a My Documentum for SharePoint Web Part to represent that segment of Documentum in the Finance site collection’s home page.  Features such as Office Web Apps, SharePoint alerts, SharePoint workflows, and SharePoint permissions would not apply for the information presented by that web part.

Lifecyles

With SharePoint, you have to look into both the Site Lifecycles, Document Lifecycles, and Information Lifecycles (not everything’s a document).  Here’s an example:

This describe a process on which site owners must renew the site for continued use.  The process should also define, based on the solution domains and the types of information that they contain, if and how the important information are moved to a longer term storage (i.e.: Document Center –ish solution).  At the latest, this would need to occur automatically prior to a site being deleted.

For example, the structured information will get routed automatically to the right location.  On the other hand, you have a choice regarding unstructured information – whether you allow them to be deleted, or whether you move the last version of that document to a specific ‘unstructured’ container of your record manager, which would have a generic retention but would also allow for a process to be define so that owners are notified to tag the information at that time – if necessary.

Note that it’s impossible to structure all information from the beginning (if ever) – you need to provide guidance on what you will do with the unstructured information.  No matter what, if you aren’t completely structured today, you’ll have to answer that question for any information you have today.

Search strategy

An enterprise search strategy must not only look at the repositories that must be crawled, but also how security trimming will apply.  For example, the SharePoint Indexing Connector for Documentum provides the ability to use a SQL database that will map the Documentum LDAP accounts (if not using Kerberos) to the corresponding AD account used in SharePoint.

The search strategy must also look at how results are brought back to end-users in form of ‘central search center’, contextual search, search-driven application, profile-influenced search (i.e.: FAST Search returning results based on your company role, for example).

Last, it should also look at how the indexing pipeline can be leveraged to automatically tag information on documents, based on its content.

This strategy is particularly complex when dealing with multiple different types of systems, across geographical locations (WAN), and different authentication mechanism.

Social strategy

Typically, when I mention a “Social Strategy”, the first thought from the customer is about the social policy that they have or need – which details what employees are allowed on public social sites such as Facebook and Twitter – and their corporate obligations to it.  While this is definitely true, required, and part of the social strategy, there’s more to it.

It’s not just about providing blogs and wikis capabilities, those are merely a format that might be used.

In the enterprise, this strategy must provide guidance on how to use social features and technologies to address business requirements.  For example:

• How can employees find people by their name, roles, region, expertise and skills, or even by project;
• How can employees identify information that should be tagged as an Intellectual Property (IP), possibly may even making them IP, Community IP, and Managed IP depending on how many people refers that information;
• How can employees ask questions to the right audience;
• How is a community managed, who can create them, what kind of communications are allowed, what is allowed in the community, etc.;
• Where and how information can be tagged and rated by individuals
• How do we provide a repository for any employees to enter content in a free-form format that makes it easy for them to do so, how are these maintained/approved, and if/how this employee participation can be identified, evaluated, and rewarded in some fashion.

The last is important – times are changing, employee retention is difficult and so is documentation.  Providing to much of a fixed format with large document models is cumbersome, time-consuming, and essentially not efficient – it takes too much time to find back the information and/or to read all the documents.  Think about providing some kind of enterprise Wikipedia for people to easily share information that they deem valuable, with the ability for tagging/rating by end-users.  Popularity and search (which requires planning) will make the system invaluable.

This is all about how you connect people to people and to information.

Governance

Big topic, and not one I’ll cover in details in this post.  There are several posts on this subject too.  The reason I wanted to point it out is that governance has been coined as an issue in SharePoint.  Depending on your point of view, it’s both true and false.  Does SharePoint have some governance enforcement features in it, of course.  Are there other governance enforcement features that are often needed? very likely.

The reason it’s been a problem is multi-fold: SharePoint is often installed without much of a vision and grows like most file storage system – you can’t find back the information properly, it’s a bunch of disconnected features, there’s no end-user solution, capacity planning is an afterthought, etc.  A governance is a continual process on how you glue in business requirements and technical requirements.  It’s a set of policies, procedures, and guidance to help IT build the right solutions to the end-users, and to provide the end-users the flexibility they require.  It’s a necessity now since most IT organizations are big, complex, and overwhelmed by procedures (for better and worst) that it may have lost its touch with the business.

It’s not a one time job, and one size doesn’t fit all.  The governance should outline the strategies that are mentioned in this article, and provide an easy way for end-users to request solutions.  These strategies must evolve as the business evolves.

Also, although most governance work today are separate engagements, it should be part of any project that augments the overall governance.  Personally, I built the documentation of governance deliverables in wiki-ish format, and it’s also built around stories that are meaningful to the readers – whoever they are in the organisation, from the end-user to the architects, including the developers and all roles associated with the platform.

Wrapping up Part 2

I hope this was helpful in looking how to breakdown standard installations of SharePoint.  SP2010 provides a lot of capabilities but making them useful to your business takes some planning and guidance.

Part 3 will be looking at how to integrate ECM products together in more details.