Delivering End-to-End High-Productivity Computing

  • Article

Here's a very good paper outlining the work by Marc Holmes (MTC) and Simon Cox on producing an end to end solution using WinCCS (HPC) and many other technologies - it would be great to get feedback from others if they have setup their WinCCS cluster this way or if there are architectures they are using for scientific research .

Here's the technologies they have incorporated -

Delivering End-to-End High-Productivity Computing

by Marc Holmes and Simon Cox

Summary: Performing a complex computational science and engineering calculation today is more than about just buying a big supercomputer. Although HPC traditionally stands for "high-performance computing," we believe that the real end-to-end solution should be about "high-productivity computing." What we mean by "high-productivity computing" is the whole computational and data-handling infrastructure, as well as the tools, technologies, and platforms required to coordinate, execute, and monitor such a calculation end-to-end.

Many challenges are associated with delivering a general high-productivity computing (HPC) solution for engineering and scientific domain problems. In this article, we discuss these challenges based on the typical requirements of such problems, propose various solutions, and demonstrate how they have been deployed to users in a specific end-to-end environmental-science exemplar. Our general technical solution will potentially translate to any solution requiring controlling and interface layers for a distributed service-oriented HPC service.

<...>

Conclusion

Architecting for high-productivity computing is not just a case of ensuring the "best" performance in order to compute results as quickly as possible; that is more of an expectation than a design feature. In the context of the overall value stream, the architecture must drive value from other areas, such as ease of access and decreasing cost of specialist skills to operate the system.

A successful architecture for high-productivity computing solutions involves consideration of the overall process alongside the computationally intensive activities, and, therefore, might use several integrated components to perform the individual aspects of the process. Microsoft Cluster Compute Server Edition is easy to include as a service gateway inside a general n-tier application structure, and is simple to integrate via command-line or API hooks.

Other available technologies can provide the basis for an HPC solution. In particular, Windows Workflow Foundation is well-suited to provide an application interface to CCS, because the features and extensibility of WF, such as persistence and tracking, lend themselves to the requirements of HPC-based solutions. The use of WF also opens up the available choices of user-experience technologies to be applied in a given domain.

Source: Delivering End-to-End High-Productivity Computing