Publishing – Made Simple

“What happened to my publishing options?” is one of the frequently asked questions from sharp-eyed project managers who have just upgraded from Project Server 2003 to Project Server 2007. The answer is complex enough to deserve this blog entry.

My design needed to deliver in two areas:

- Simplicity, our research showed that very few customers understood the nuances of each publish option, especially when combined with the even more hidden Tools/Customize/Published Fields options

- Scalability, our larger customers were hitting bottlenecks because of the serialized nature of project publish.

Simplicity

To address the simplicity I decided to strip project publish to its bare bones, changing from the two dialogs below:

To the single option, placed next to the Save command:

It doesn’t get much simpler than that - this should make the publish option much less of a dice throw for most of our customers – however “power publishers” will be asking for more functionality, you should read on to the Power Publishing section for more information.

Scalability

History Lesson: Project Server 2003 used a background service called “Views Notification” that was responsible for moving data from the Project client sql tables to the Project Server and Views sql tables in the database. This process was serialized on a single project and the whole process was single threaded, using a single cpu (although later service packs allowed it to be run on multiple servers), as you might expect publishing rapidly became a bottleneck on larger systems. It also had other issues in terms of manageability, capacity planning and availability but more on that in other blog entries.

The new server architecture has radically changed how publishing works. Publish requests are placed on the Project Queue and are handled by a queue processor that:

  • Is multithreaded and multi-server and serializes per project rather than across all projects
  • Pulls data from the Draft database into the Published database through the MSP_*_WORKING_VIEW sql views
  • Invokes the corresponding Report Publish that handles the transformation of the data into a report friendly format into the reporting database
  • Optimizes (“folds”) multiple publish requests against the same project into a single request

Even with all the additional work done by publishing (such as moving all custom fields, and serializing data from our internal binary formats) internal tests have shown sustained publishing rates of around 1,400 projects per hour for non-trivial projects on a farm infrastructure.

And the Project Server 2007 queue infrastructure allows for much improved capacity planning and remote management as it exposes performance counters that the Windows System Monitor (fancy name for perfmon) and MOM can catch and track.

Power Publishing

The Project Server 2003 dialogs reflected the relative underlying complexity of the publishing process - much of that complexity was driven by the need to cope with the shared schema (now split across distinct databases) and the need to avoid the performance hit of a full publish. The server now publishes all changed information each time you request a publish operation.

Changes are tracked using revision counters on our primary entities and their children (for instance Projects own {tasks, assignments, specific custom field values, calendars and local resources} – these counters increment each time the project is saved, deleted rows are tracked in our _SHADOW tables.

The two main operations a power publisher wants to control are:

  • When a team member sees a specific task assignment (aka “Phasing”)
  • Who approves task progress (especially when the primary project manager is on vacation)

Both these actions are now controlled through the task sheet where the settings are now visible and editable (Yay!).

If you add the “Status Manager” and “Publish” fields to the sheet (as seen above) you can control the publishing process. Note that both these fields can be set as a group by filtering then using the mouse to drag the value down to more cells.

Firstly – use the Publish Yes/No flag to control whether a task assignment is placed in the Statusing (“My Work”) system – this flag can be toggled at any time & the project republished to make it active. This empowers you to publish a project a phase at a time, allowing you to avoid bombarding your team members with future assignments.

I made the flag at the task level because statusing is all about gauging progress against the whole task - assignment progressing is best managed in the Timesheet sub-system where work can be approved by resource managers rather than the project manager.

In the reporting database the flag is actually stored on the assignment:

 msp_epmassignments_userview.AssignmentIsPublished

There isn't a sinister reason for this, it just reflects the order in which we did the development work against a changing schema.

Note that if you toggle Yes-->No then the assignment disappears from the Team Members My Work (any approved work doesn't get lost though!) so use this power carefully! Also be aware that if the line is already in a timesheet it won't be pulled back.

Secondly - the Status Manager field has some strange rules (that echo those of project Server 2003) - it can be set to another pre-existing Status Manager on a task in the current project or the current user (ie the person with the project checked out and open) - this allows the value to be set even when the project is off line, and ensures that the Status Manager hhas (at least had) the ability to edit the project so that status data from team members can be applied once approved.

So if you are going on vacation you'd ask your deputy to open the project, filter on an appropriate time window and set themselves as the status manager for the tasks that will be active while you are out. On your return you can easily find those tasks and reset them back.

Phew! If you have any questions about other "mysteries of publishing", then please reply to this posting and I'll do a followup.

 

Technorati tags: Project Server 2007, Project Manager, Statusing, Publishing, Status Manager, Project Reporting

 

del.icio.us tags: Project Server 2007, Project Manager, Statusing, Publishing, Status Manager, Project Reporting