Announcing: MSVC Conforms to the C++ Standard

Ulzii Luvsanbat [MSFT]

Achieving conformance with the C++ Standards has been a long road for the Visual C++ team. If you’ve seen us at any conferences lately, you’ve probably seen the MSVC Conformance slide. (You can grab a copy of the slide or watch the 2017 CppCon talk here.) Finishing the features on this slide – which includes features from C++11, C++14, and C++17 – has been one of our team’s main goals for the past few years.

We’re happy to announce that in Visual Studio 2017 version 15.7 we’ve implemented these features. (The Technical Specifications are work-in-progress as they make their ways into the Standard.)

The MSVC compiler toolset in Visual Studio version 15.7 conforms with the C++ Standard!

We’re not going to stop our conformance effort – there are always new features, defect reports, etc. There are places within these features where we have known bugs, but we believe that, at this point, there are no feature areas in the C++ standard that you should avoid with the MSVC compiler (except for the preprocessor as described below, and floating-point <charconv> which is not yet implemented). Expect to see more future communications from our team as we alleviate the remaining caveats. See the details below in the blog.

Getting to conformance

Our compiler has a long history: February marked the 25th anniversary of the Visual C++ product. The Microsoft C product on which Visual C++ was built is 35 years old. And through the entire history of compilers our first concern has been maintaining compatibility so that your code isn’t broken. We carefully document all changes made for conformance, making our fixes source-compatible when possible and providing workarounds when changes are needed in your code. Three years ago, we unveiled our compiler rewrite. We’ve been doing a major overhaul of the compiler in a process we’ve referred to as “Rejuvenation”. Both the old YACC parser and the “Rejuv” Recursive Descent parser run side-by-side in our compiler today. We’ve moved features individually from the old code to the “Rejuv” code as we’ve been able to implement them. This technique has enabled us to continually make progress while minimizing breaking changes. Rewriting features has occasionally caused some regressions, and we are truly sorry for any inconveniences. However, overall it’s been far easier to implement modern C++ features on a modern compiler than it was on our decades-old parser, where some features just could not be implemented altogether.

Standard Library Conformance

The MSVC implementation of the Standard Library has gained major new features recently: Parallel Algorithms, Filesystem, constexpr char_traits, Special Math, and support for class template argument deduction. A detailed table of our feature status is below in this post.

Compiler Conformance

With Visual Studio 2017 version 15.7 we’re shipping a complete implementation of almost all features in the C++ Standard, including all versions up through C++17. The remaining features have been implemented (and will ship soon) or are being implemented today. This includes the completion of some of the more difficult features that we’ve been working on from C++11/14: two-phase name lookup, expression SFINAE, extended constexpr, and pack expansions. Every compiler has bugs—we reported some conformance bugs in other implementations while implementing old features with fresh eyes. We’ll continue to address our bugs and continue to implement new features as they are adopted in draft Standards.

Technical Specifications

MSVC also leads on many technical specifications. MSVC has the most complete implementation of the Extensions to C++ for Modules TS. We’ve got the oldest implementation of the C++ Extensions for Coroutines and have recently rewritten our optimizer for coroutines. Microsoft is shipping products that substantially use both Modules and Coroutines. We’ve been working on the C++ Extensions for Ranges TS, both to improve the TS and to bring MSVC to the point where we can support Ranges. And while most of the C++ Extensions for Concepts has been merged into the C++20 standard draft, we’re committed to implementing the feature early in our C++20 work.

“But what about…?”

We’ve driven our conformance effort by focusing on popular OSS libraries while maintaining source compatibility for large codebases. Some major libraries still don’t compile correctly with MSVC, and we are actively working on them.

• Boost.Hana:
  • Extended constexpr has also been a challenge in our compiler. We have some known bugs, but we believe our implementation is conforming. For a concrete example, we’ve discovered about 10 blocking constexpr bugs in the compiler when building Boost Hana, the “standard library for metaprogramming”. While we are actively working on fixing these compiler bugs, if we apply source workaround changes to Hana then we are able to pass about 30-40% of the Hana tests. We’re currently completely rewriting our constexpr evaluator to address the remaining issues.
• Pack expansion:
  • Pack expansions are quickly becoming the fabric of C++. Our implementation is also evolving as we move constructs to use the new “rejuvenation” compiler parse tree. We have some bugs with variadic generic lambdas, pack expansions over arrays, and partial ordering of function templates that will be addressed by using the new parse tree. We’re also making some fixes to the rewording of inheriting constructors to work correctly with pack expansions.
• Range V3:
  • Alias templates are used heavily in many Modern C++ libraries. MSVC has bugs with alias templates that prevent some of these libraries from compiling, e.g. Range v3. We’re re-implementing parts of the feature on the new “rejuv” parser. The new parse trees will help us to fix all the remaining bugs with alias templates in MSVC. (Range v3 is the basis of a proposal to add range support to the C++ Standard. We have a fork of Range v3 that works with MSVC but it’s significantly behind the Range v3 trunk.)
• Preprocessor:
  • Most codebases don’t use esoteric macros so we’ve only just started our major preprocessor conformance effort. We’ve fixed bugs as we’ve encountered them in OSS libraries but we haven’t made changes that would break existing code. We’ll introduce an opt-in switch in a later Visual Studio 2017 release that will allow you to use a conforming preprocessor that no longer supports some of the old MSVC-isms that exist in many codebases.
• As always, any bugs that are reported at the Developer Community portal will get looked at and responded to by the Visual C++ Team.

Feature status tables

Here are feature status tables and notes prepared by Stephan T. Lavavej (without the usual color highlighting).

• C++20: We’re working on finishing C++17 before starting C++20, so not-yet-implemented C++20 features aren’t listed in these tables. In the STL, there are minor exceptions to “17 before 20”. P0809R0 “Comparing Unordered Containers” and P0858R0 “Constexpr Iterator Requirements” were already achieved by our implementation, and P0777R1 “Avoiding Unnecessary decay” was purely a compiler throughput improvement.
• N/A: For clarity, we’ve omitted a number of papers that are Not Applicable (nothing for implementers to do, or users to take advantage of), such as wording clarifications.
• [throw()]: In /std:c++14 mode, dynamic exception specifications remain unimplemented, and throw() is still treated as a synonym for __declspec(nothrow). In C++17, dynamic exception specifications were mostly removed by P0003R5, leaving one vestige: throw() is deprecated and required to behave as a synonym for noexcept. In /std:c++17 mode, MSVC now conforms to the Standard by giving throw() the same behavior as noexcept, i.e. enforcement via termination. The compiler option /Zc:noexceptTypes- requests our old behavior of __declspec(nothrow). It’s likely that throw() will be removed in C++20. To help with migrating code in response to these changes in the Standard and our implementation, new compiler warnings for exception specification issues have been added under /std:c++17 and /permissive- as documented here.
• [twoPhase]: Two-phase name lookup is now supported in /permissive- mode. Please read our Sept 2017 post for more details.
• [exprSFINAE]: Expression SFINAE is now supported, regardless of whether /permissive- mode is active. There are a few bugs remaining to be fixed, but in general, the “unique tag type” workaround is no longer necessary, and we’ve removed this workaround from our STL implementation.
• [preprocessor]: Support for C99’s preprocessor rules is unchanged (considered partial due to support for variadic macros, although there are numerous bugs). We’re overhauling the preprocessor, and we’ll experimentally ship those changes under the /permissive- mode soon.
• [ifConstexpr]: “if constexpr” is supported in /std:c++14 with a warning that can be suppressed, delighting template metaprogramming library authors everywhere.
• [launder]: std::launder() is defined in /std:c++17 mode. When the __builtin_launder compiler intrinsic is available (as is the case for MSVC’s front-end C1XX), std::launder() uses it to communicate information to the back-end, suppressing certain optimizations. Otherwise (as is the case for Clang 6.0.0), std::launder() returns the pointer unchanged. We’re using “__has_builtin(__builtin_launder)” to automatically adapt to future versions of Clang providing this intrinsic.
• [14]: These C++17/20 features are implemented unconditionally, even in /std:c++14 mode (the default). For some features, this was because they predated the introduction of MSVC’s Standard mode options. For other features, conditional implementation would be nearly pointless or undesirably complicated.
• [17]: These C++20 features are implemented in /std:c++17 mode.
• [DR]: These papers were voted into the Working Paper after C++17, but as Defect Reports, meaning that they retroactively apply to C++17 (as bugfixes).
• [atomic]: This was almost completely implemented in VS 2017 15.3, and the remaining differences are difficult to observe (some signatures differ from the Standard, as observed by taking their address or providing explicit template arguments). The STL’s next major binary-incompatible version will fix the remaining differences.
• [byte]: std::byte is enabled by /std:c++17, but has a fine-grained opt-out macro (_HAS_STD_BYTE can be defined to be 0). This is because given certain patterns of using-directives, it can conflict with the Windows SDK’s headers. This has been reported to the SDK team and will be fixed, but in the meantime the escape hatch is available.
• [C11]: First available in VS 2015, the Universal CRT implemented the parts of the C11 Standard Library that are required by C++17, with minor exceptions. Those exceptions (which are tracked by bugs) are: missing C99 strftime() E/O alternative conversion specifiers, missing C11 fopen() exclusive mode, and missing C11 aligned_alloc(). The strftime() and fopen() functionality will be implemented in the future. aligned_alloc() will probably never be implemented, as C11 specified it in a way that’s incompatible with our implementation (namely, that free() must be able to handle highly aligned allocations).
• [charconv]: from_chars() and to_chars() are available for integers. We’re currently working on floating-point from_chars(), to be followed by floating-point to_chars().
• [depr] and [rem]: See C++17 Feature Removals And Deprecations.
• [parallel]: See below.

New Features: Parallel Algorithms

C++17’s parallel algorithms library is complete. Note that this doesn’t mean every algorithm is parallelized in every case; the most important algorithms have been parallelized and execution policy signatures are provided even where algorithms are not parallelized. Our STL implementation’s central internal header, yvals.h, contains the following “Parallel Algorithms Notes”: C++ allows an implementation to implement parallel algorithms as calls to the serial algorithms. This implementation parallelizes several common algorithm calls, but not all.

• The following algorithms are parallelized.
  • adjacent_difference, adjacent_find, all_of, any_of, count, count_if, equal, exclusive_scan, find, find_end, find_first_of, find_if, for_each, for_each_n, inclusive_scan, mismatch, none_of, reduce, remove, remove_if, search, search_n, sort, stable_sort, transform, transform_exclusive_scan, transform_inclusive_scan, transform_reduce
• The following are not presently parallelized:
  • No apparent parallelism performance improvement on target hardware; all algorithms which merely copy or permute elements with no branches are typically memory bandwidth limited.
    • copy, copy_backward, copy_n, fill, fill_n, move, move_backward, remove, remove_if, replace, replace_if, reverse, reverse_copy, rotate, rotate_copy, swap_ranges
  • Confusion over user parallelism requirements exists; likely in the above category anyway.
    • generate, generate_n
  • Effective parallelism suspected to be infeasible.
    • partial_sort, partial_sort_copy
  • Not yet evaluated; parallelism may be implemented in a future release and is suspected to be beneficial.
    • copy_if, includes, inplace_merge, is_heap, is_heap_until, is_partitioned, is_sorted, is_sorted_until, lexicographical_compare, max_element, merge, min_element, minmax_element, nth_element, partition_copy, remove_copy, remove_copy_if, replace_copy, replace_copy_if, set_difference, set_intersection, set_symmetric_difference, set_union, stable_partition, unique, unique_copy

New Features: Filesystem

C++17’s <filesystem> is supported. This is a completely new implementation, incompatible with the previous std::experimental version, necessitated by symlink support, bug fixes, and changes in standard-required behavior. Currently, including <filesystem> provides the new std::filesystem and the previous std::experimental::filesystem, and including <experimental/filesystem> provides only the old experimental implementation. The experimental implementation will be REMOVED in the next ABI-breaking release of the libraries.

Performance and Throughput Improvements

• Refactored some of the metaprogramming in <variant>, reducing compile time of programs that use large variants by as much as 30%.
• The STL will now detect some inputs to std::swap_ranges that are safe to vectorize, providing 8x (times, not percent) performance wins for these special cases.
• std::stable_sort had an optimization applied to avoid doing “is the temporary memory we allocated big enough” checks on each element movement, resulting in a 2-3% performance win in release builds and a 26% win in debug builds when sorting ints.
• Refactored the vector::emplace_back family into separate “fast and nonallocating” and “slow and allocating” paths, allowing the former to be inlined into callers.
• Refactored some parts of the binary tree implementation that don’t depend on T out of templates for increased throughput.
• /std:c++17 now activates the STL’s internal usage of “if constexpr” to improve throughput, mitigating the throughput cost of providing more features. (More “if constexpr” refactoring remains to be done.)

Bug Fixes

• Fixed a bug in pmr::unsynchronized_pool_resource and pmr::synchronized_pool_resource that resulted in severe failures when allocating objects of multiple sizes from the same pool resource.
• Applied noexcept in more places in std::array, especially in std::array<T, 0> where we didn’t even provide the standard-mandated noexcepts.
• We use destructors rather than catch and reraise in more places in the STL, improving the debugging experience for unhandled exceptions by preserving their original throw site.
• Iterator debugging now runs order checking for homogenous ranges only, avoiding cases where we required more from comparison predicates than the standard requires.
• priority_queue now avoids unintentional argument-dependent lookup when calling std::make_heap() etc.
• <cmath>’s fma() and remquo() overloads are now constrained for consistency.
• Fixed truncation warnings in vector and string (on x64 with allocators providing 32-bit size_type).
• Added constexpr and noexcept to std::ignore.
• Implemented more LWG issue resolutions. Our tracking spreadsheet is massive, so here’s a summary. The LWG issues resolved in C++14 were completed in VS 2017 15.0. Of the LWG issues resolved in C++17 and the C++20 Working Paper (plus a few that are still open):
  • 130 are N/A
  • 64 were implemented before VS 2017
  • 47 were implemented in VS 2017 15.0
  • 32 were implemented in VS 2017 15.3
  • 13 were implemented in VS 2017 15.5
  • 8 were implemented in VS 2017 15.6
  • 33 were implemented in VS 2017 15.7
  • 14 have been implemented in VS 2017 15.8 so far
  • 4 have been implemented in our “WCFB02” binary-incompatible branch
  • 2 have been reported to the Universal CRT team (these involve ISO-deprecated headers)
  • 2 affect not-yet-implemented C++20 features
  • 6 remain to be implemented

Other Significant Changes

• The STL is now tested with Clang/LLVM 6.0.0 and no longer supports 5.x.
• Added more [[nodiscard]] attributes, especially in <complex> and <cmath>.
• The non-Standard <allocators> header has been deprecated with the same technique as <hash_map> (an impossible-to-ignore #error with an escape hatch). The message is: “#error The non-Standard <allocators> header is deprecated and will be REMOVED. Consider using the C++17 <memory_resource> header. You can define _SILENCE_STDEXT_ALLOCATORS_DEPRECATION_WARNING to acknowledge that you have received this warning.”

In closing

We’d love for you to download Visual Studio 2017 version 15.7 and try out all the new C++17 features. (Remember you’ll often have to use the /permissive- conformance switch as well as /std:c++17 or /std:c++latest as appropriate.)

As always, we welcome your feedback. We can be reached via the comments below or via email (visualcpp@microsoft.com). If you encounter other problems with MSVC in Visual Studio 2017 please let us know through Help > Report A Problem in the product, or via Developer Community. Let us know your suggestions through UserVoice. You can also find us on Twitter (@VisualC) and Facebook (msftvisualcpp).

Ulzii Luvsanbat [MSFT] Principal Group Software Engineering Manager, Microsoft C++ Language Team

Follow