As you may know, Visual Studio now supports building Android and iOS applications using Clang. We realize the need of our users to write cross-platform apps and we know that C++ offers massive benefits in the cross-platform arena. Clang has made impressive strides in standards conformance and diagnostics and is fast becoming the preferred C++ compiler for non-Windows platforms. In order to extend the benefits of Clang to C++ developers building apps for Windows, we have been working on an experimental implementation of Clang on Windows. First preview of that was demonstrated in this //build talk by Jim Radigan.
The Clang and LLVM community has already done tremendous work towards providing a toolchain that builds C++ code for Windows. This blog post first announced it. Subsequent progress was also noted through other posts such as this and this. They have also provided neatly packaged toolset builds for Windows that integrate very nicely with Visual Studio. Our work builds on top of all the impressive work that the community has already done.
On our end, the primary motivation for doing this work is to enable the cross-platform scenario where a significant portion of your app is cross-platform C++ code.
If you already build cross-platform apps, you probably have a code-base that somewhat resembles the above structure. Here (1) and (3) are platform specific pieces of code, either because they call into OS-specific API or they happen to use programming language features that are provided only by some platform specific compiler tool-sets. There are usually genuine reasons for such code to exist. However, at the same time, a big chunk of your app logic is (or can be) written using proper standards conformant cross-platform C++ code. This is the block marked (2) above.
Imagine an app whose code structure resembles the above diagram being built for various platforms:
- For Android and iOS, the entire apps would be built using Clang
- For Windows, the entire app would be built using Visual C++
This essentially means that, even though we are calling the block (2) cross-platform, we are still building it using different compilers for different platforms. So, any minor divergence in features of Clang and Visual C++ will cause you as a developer to have to worry about these compiler differences or limit you to using the smallest subset of the two compiler features. This is definitely not ideal.
- What if you could use a single compiler for your cross-platform code irrespective of what platform you target?
- What if you could still use a separate compiler to build all of your platform specific (or compiler specific) code that you have accumulated over the years?
- What if you could have rich inter-op between the above two while still making use of the excellent, secure, debugging-friendly code-generation engine that powers Visual C++ and builds some of the most complex software on Windows (almost everything.. including Windows!)?
This is now possible with the work we have done. What this enables is a scenario in which you compile (1) and (3) using the Visual C++ compiler, (2) using Clang and for code-generation, all of these would still use the Visual C++ back-end (We call it C2). All of this while still enjoying the rich end-to-end developer experience within Visual Studio.
We believe using our back-end as the core of this technology gives us a lot of benefits since we look at C2 and the linker as a part of the ABI of the Windows platform. Having dealt with code-generation for all supported processor architectures under various conditions, we believe it is best placed to power the new technologies like .NET Native and now Clang (for C++ and even for ObjectiveC).
This is work in progress and we will be talking a lot more about this work in coming months. As always, we look forward to your feedback on this scenario and our approach.
Thank you
Visual C++ Team
Will it be possible to use Clang to compile (1) and (3) too?
Eddie, current focus is box 2. we want to significantly reduce the friction to making cross plat code work well.
This appears to be wishful thinking. Clang is standard for Mac, GCC in all its GPLv3 goodness is standard for Linux.
You've made a falsifiable claim here. What are your numbers that support the claim you're making?
@ David Gerard
How about Android and iOS ?
Also more and more linix distros support clang too.
Guess i'll have to wait until Google finishes the working their doing to bring Clang to Windows (that targets the MS ABI). Meanwhile i'll stick to Mingw-w64, i can use Clang with it.
Anyway, thanks for the quick response Steve.
That sounds great! Nicely done!
One question: if the Microsoft linker is considered part of the Windows API, any chance of getting it to support reproducible builds? Zeroing out timestamps etc.?
Eddie, I want to make sure I understand your feedback. Why the preference for a Google version? We are just trying to say that we are focusing our efforts on portable code for Clang/C2. If your code doesn't have MS extensions, you should be able to use it.
I like the above, and as a first step, building my cross-platform code with Clang would be a gigantic help. But here's some elucidation of the pushback you are seeing.
The ask is that we can build Windows binaries start to finish with Clang, for multiple reasons. For the foreseeable future, Clang tracks the C++ standard quicker than Microsoft (I'd like C++14 constexpr, and so on). But it would also be cool to build cross-platform binaries on a single machine, instead of using multiple machines. I can build Android binaries on Windows, for example, and this is cool. I'd like to build Windows binaries on a Linux or Mac box.
I love the Visual Studio IDE, and the compiler and linker have always been fast compared to the competition, and usually generate much better code to boot. But I don't like the slowness in getting new features in the language or libraries. Hence – Clang.
If Visual Studio would build Mac and Linux binaries and it was bleeding edge in terms of language and library support, I'd use it for everything.
Yeah my code, which only targets Windows at the moment, doesn't use any MS extensions. So if i understand it right, as long as my code doesn't use any MS extensions, i should be able to build a native Windows executable with Clang.
That's great then. I'm looking forward to try it.
Thanks.
I'm wondering WHEN or if it's NEVER the Clang would support compiling COM/ATL code…
Re: Brian Fitzgerald – you are totally right-on. Clang is by far the most forward-looking compiler for C++ and a pleasure to use. Given Herb Sutter's wonderful evangelically effusive advocacy of writing "Modern C++" (his term, which I completely agree with) and using C++ in a truly portable fashion, it's a shame to see MS still weighed down by obligations to the visual compiler in this day an age. I applaud these new moves by Microsoft, bravo to them – but I feel like you have to commit all the way to be meet the promise. Support vc completely of course, get optimisations for the Windows platforms of course, but I would expect in the contemporary context that you should be able to write C++ which compiles with clang anywhere and compiles with clang in Visual Studio just the same for any platform including Windows, and the optimisations from writing vc++-compliant code should be done only when necessary and at as late a stage of development as possible – if that means MS has to throw resources at LLVM and clang then that's what needs to happen, and the results will be better for everyone including MS in the long term. In the same spirit as Herb's "Conformance And Cross-Platform Mobile Development" (channel9.msdn.com/…/311) pitch for VS 2015, how can C++ be portable with the need for vc-compliant code when building for MS products? Surely that part should be optional rather than a requirement? Xamarin is great, but if you are pitching yourself into the same space in the context of C++ instead of C# then there are obviously a lot more steps to be taken.
I'm a young developer who loves C++ but I'm not a Windows dev so I'm not trying to proclaim any laws at all, I just see myself as precisely one of the targeted market demos for switching to VS2015 and these are just the obvious thoughts that arise for me. I watched Herb's cross-platform presentation and feel like that's the gulf between the 'promise' and the reality.
But still, bravo!
Re: Eddie – exactly. Windows binaries with clang out of the box is the expectation. Let us know how it works.
p.s. I had to remote into my Win8 box at home to post this as it didn't like Safari on my Mac laptop. Guess the leopard can only change it's spots so quickly!
Why is it so hard to come up with fully cross-platform code, even for UI?
We have wxWidgets, Qt, GTK, FLTK, …. for desktops, so what's the problem with mobile?
How about fixing the /Fa switch for the listing of 64bit assembly so it can be modified by scripts to implement static instrumentation and code coverage assessment without resorting to PIN. Right now it won't even assemble with Masm64 in Visual Studio 2015, your ability to export the intermediate steps is getting worse. Also it would be nice to open up more intermediate steps between the compiling steps and even the intermediate representation for tweaking and tooling, and instrumentation, like clang offers.
@Exim wrote:
I think for that to happen, we will probably have to wait for C++17 features to go viral to have standard graphics libraries.
This is a brilliant news for xplat scenarios. Thanks for the community! 🙂
On a related note, it will be icing on the top when VC team will make CL as an independent compiler environment and provide a version manager, like DNX (https://github.com/aspnet/dnx) (esp. DNVM) from .NET/apsnet teams.
Based on that, I drafted these proposals for VC team:
//gist.github.com/am11/157897417d1eef4d7116
//gist.github.com/am11/0c8c1306cf311315bf07
Please vote here, if this is something which would help your use-case as well: visualstudio.uservoice.com/…/6742251-make-c-c-compiler-cl-exe-independent-of-ide
So, if my understanding is right, you will be providing the possibility of using Clang as the compiler frontend with C2 (MSVC) as the compiler backend for Windows?
That would be perfect: full feature compliance with standards (Clang) and full compliance and compatibility with Windows (MS).
Is this correct?
Does this mean a Cland compiler frontend (full language compliance) with C2 (full Windows compliance) compiler backend?
That would be a match made in heaven for Windows developers.
Might as well get rid off the front-end completely once that happens.
Next step: make Visual Studio 2015 cross-platform and 64 bit. It's the best ever IDE – so focus on that rather than compiler frontends.
What about exceptions? I seem to remember the Clang folks saying in so many words that they were unable to support throwing / catching exceptions on Win32 (thus putting a severe limitation on using Clang for serious work on Windows) due to Microsoft having patented the ABI?..
Will we be able to target other platforms supported by clang? I use FreeBSD and I have a MSDN subscription.
@Lando
If Microsoft has patented the API, then they are able to use it freely, correct? so they can do the work to make it work with clang, if the clang developers can't.
Clang is available under the BSD or MIt license, so Microsoft doesn't have to release the source changes they make at all.
Now, I don't know if you can patent an API, and I am pretty sure you can't, as an API is a description of services offered. You might be able to patent specific pieces of it that describe novel ways of implementing it, but that doesn't stop someone else from implementing a different novel way of implementing it. At least in the US in a perfect world.
You might try Oracle's copyright strategy on the API, but that is a different story altogether.
This is great news!
What's the story with STL in this scenario? Would Clang be able to use STL from MS or would it need a port of libc++?
@BillDStrong They're talking about the ABI (application binary interface), not the API. Since it is more concrete and specific about implementation, patents sound entirely plausible to me. And since C++ doesn't have a standard cross-platform ABI, in constrast to C, so each compiler/OS combination is free to use whatever interface is desired (constrained by requirements of the OS, of course). Though it would indeed be convenient if all compilers on a single OS would at least use the same ABI. If I recall correctly, Herb Sutter has been recommending a standardization proposal along these lines. Pushing for a single ABI across all operation systems, on the other hand, is much harder to design, justify, and get consensus support for.
Great! Now can you let us use CLANG as a C99 compiler on windows?
excellent
Please support both C cross-platform code compiled by Clang and not just C++. Porting C99 (designated struc/union initializers) and C11 code (stdatomic.h) to C++ is just too much effort for too little gain esp. since the code works on both iOS and Android just fine.
Can you clarify the "(2) using Clang and for code-generation" bit. When you say code generation does this mean you take the IL from clang and have written a way to convert and read that into the VC back end? If so what does this look like from VS, is it a new Platform Toolset? What do you do when you come across some language feature that clang can compile to IL but you can't represent in VC ABI or that can't be consumed by the VC back end for some other reason?
Would have been great if there had been a Build talk on this as the ones that did focused mostly on phone stuff.
Also any comments on LLVM project trying to target the VC ABI out of the back end of LLVM?
I believe the Embarcadero C++Builder 64-bit compiler already uses Clang and LLVM on Windows. Will you be working with them at all, or re-using any of the work they've submitted back to the project?
Steve, this looks interesting, but when you say (in a comment) "If your code doesn't have MS extensions, you should be able to use [Clang]", does it mean that MS's Standard Library headers will not use any MS extension? Last time I checked, that wasn't the case for e.g. <iostream>, which used "__try" blocks. Will I be able to include those headers? If I do so, does the code in those headers count as "my code"?
Thanks for the great feedback everyone. These are exactly the kind of discussions we want to have. So, please keep it coming either here or feel free to write to me [rasharm at microsoft dot com]
Now for some responses:
@Brian Fitzgerald, @Eddie, @BR: Building full binaries using clang would largely be a function of how many Microsoft extensions your code has, Clang's level of support for those extensions (remember they aim for quite a lot of MSVC compatibility) and how much time/effort you want to spend on converting that code to pure portable C++ code. As Steve mentioned, our focus right now is on the portable pieces (block #2 in the diagram).
@Simon Dan: Again, if by "clang support COM/ATL code" you mean replicating the specific support that VC compiler had added to support COM/ATL then I am not sure what is the level of compat right now. However, if you are just talking about using C++ header files to call into COM API, then clang should already be able to do it.
@Exim: Cross-platform UI is admittedly a hard problem. Every platform likes to differentiate by providing unique UI controls and API. Putting a "common framework" on top of them somewhat dulls the richness. Maybe that's the reason. Just hypothesizing of course :). Having said that there is work ongoing to try and have some standard description of that too.
@Adeel: We have already been thinking about some of these. There are some complexities here that we don’t have a good solution for. For example, how to balance between steady, stable stream of compiler/libraries on one hand and introducing binary breaking changes due to new features/improvements on the other. This is an important problem for us and we want to solve it right. I will go through your proposals as well.
@Istvan Fabian: Clang as the front-end and C2 as the back-end. That’s exactly what this is. Getting rid of current Visual C++ compiler front-end (C1)? Absolutely not the goal. There are multiple reasons (too many to enumerate here) why C1 will continue to be important. Our investment in Clang/C2 is “and” not “or”.
@Krzysztof Kawa, @Andy Prowl: Still thinking hard about what libraries to use in which scenario. In our tests the current MSVC STL has played out reasonably well while building the cross-platform pieces using clang (for Windows). However, we haven’t ruled out the idea of using the same library (like libc++) for x-plat code just like we are using the same compiler (clang). Under investigation!
@David Brower, @Scott Tsai: When we are done, yes you will be able to use Clang for that. Which specific pieces of C99/C11 do you rely and are currently missing in MSVC?
@David Hunter:
“When you say code generation does this mean you take the IL from clang and have written a way to convert and read that into the VC back end?”
Correct. What the final VS experience would be (a new Platform toolset or something else) is still under discussion. We are currently focused on the compiler toolset. Did you watch this //build talk: channel9.msdn.com/…/3-610
Is the ObjC runtime your own and if so are you planning on open sourcing it?
It is possible with the Clang API to write C++/ObjC diagnotics and refactorings like it is possible now with Rolyn for C# and VB.NET. If Clang becomes a front end in VS, this means that we are not very far from being able to integrate these C++ diagnotics and refactorings in the IDE, like Roslyn is integrated in the IDE for .NET languages.
Is there any plan for that? It would be very much interresting and very much appreciated I think.
@Raman Sharma Which specific pieces of C99/C11 do you rely and are currently missing in MSVC?
From my experience, many C99/C11 projects tend to rely on being able to:
1) declare variables inline with a function body, instead of at the top of a function or scope block
2) dynamic arrays, where you can define an array on the stack with dynamic size determined at runtime
3) compound literals for struct initialization
4) designated initializers for struct initialization
One helpful step to help people write platform-independent code for (2) would be to get at least the standard C and C++ library headers (and as many of the other platform headers as possible) to compile under /Za (MS extensions disabled). Otherwise it's hard to detect nonstandard code until you bring it over to Clang and it breaks.
As a fallback, would it be possible to change the /Za – /Ze switch through a pragma? Then you could set /Ze in your headers, but client could could be /Za.
As a second fallback, it would be great to just document the effect of that switch. Some of the extensions trigger warnings, but not all, I think. There use to be a page for this purpose (msdn.microsoft.com/…/2tb15w2z.aspx), but it's been blanked.
@Jeremy (aka P. M. Baty): Stop spamming your license violating tool chain.
@Luke Howard: See this page for more details about the iOS support: dev.windows.com/…/project-islandwood
More details will come soon.
@Pierre: That is definitely feasible. We haven't yet fully considered the E2E scenarios outside of the core compiler area. We will get to it in due time.
@S. Colcord: I know how it sounds.. but let me tell you .. it's on the backlog 🙂
First step in the right direction!
Hopefully we'll also get the backend at some point for high performance code generation.
So far the existing clang platform toolset @ http://llvm.org/builds has been the only viable alternative to cl.
Does your solution solve the problems arising from clang-cl having to masquerade as cl which leads to big problems with boost's countless old msvc bugfeature branches?
Does EH work?
Will clang's diagnostic capabilities be integrated in the IDE? Like the links between warnings/errors and notes, static analysis results inside the code editor, later with the backend optimization pass hints…
Why not pour your C++ compiler efforts into Clang? Especially the optimizer? While at it, add extensions to fully support Qt without the precompiler!
clang is working fine with VS2013 for generating 32 bits code (without exception and without using <thread>)
However it does not work for generating 64 bits. I speak about the clang version for Windows obtained from the llvm site.
Do you plan to have a different port? Do you plan to have a compiler generating 64bits code?
I've given this more thought and I can think of only one business reason to continue development of CL (Visual C++ compiler): to force C++ developers to upgrade Visual Studio to get compiler fixes and enhancements. However, there a many compelling business reasons to put development effort into Clang. Above all, it will avoid duplicate effort and provide consistency.
One thing the tools division seems to forget is that their purpose is to get developers to write Windows Apps which sells more Microsoft products (versus being a profit center in, and of itself.) That said, Microsoft is already giving away CL, which is why I don't get why the company is turning away from the obvious.
(To be clear; all in all, CL is the best C++ compiler, even if not the most C++11 compliant, Visual Studio is the best IDE and I've yet to use any debugger in any language that's better. Still, it would be nice to not have to wait for C++ features, especially when some projects still have to be backward compatible with Windows 2010 (so remote debugging works with XPe.))
Thanks for leaving Objective-C in the new clang compiler for Windows 10
and helping it along with libraries too …
Is Objective-C also present in the clang compiler you use in VS2015 when building for Android
or taken out like in the Android NDK from Google.
If not … please just leave it in.
Then Objective-C will become an equally valid cross platform language for the #2 level than C/C++
A task for which it is much better suited … in my opinion …
Peter
Well, well done – seems like MS is doing great stuff lately. Very curious to see how you've implemented Foundation and the runtime. Is NSMachPort built on LPC? Might be time to reimplement D'OLE!
@Joe,
I think you missed this post:
blogs.msdn.com/…/c-11-14-17-features-in-vs-2015-rc.aspx
In summary, VS2015 is:
Fully C++11 compliant minus "Expression SFINAE".
75% C++14 standard compliant.
40% C++17 standard complaint.
The missing piece from post C11 support: out of five headers en.cppreference.com/…/header, only
<uchar.h>is supported in VS2015 RC.Now if you say "VS2015 is not fully C++11 standard complaint" that would be too "mathematically true" and too "absurd" in my honest opinion.
Compare it with VS2010 and assert some positivity and you will be the happiest person on the planet..
@Meh
You missed the point, but inadvertently made it at the same time. Why should I have to upgrade Visual Studio to get a better compiler?
Why not put the same effort into Clang that's being put into CL? Especially now that MS is duplicating effort anyway?
Also note that most code analysis tools use Clang.
@Joe said:
> Why should I have to upgrade Visual Studio to get a better compiler?
I saw a proposal for the Microsoft team to make cl.exe independent from Visual Studio. So your compiler will always stay up to date. While the front-end compiler (responsible for intellisense and whatnot) can be upgraded later with Visual Studio.
> Why not put the same effort into Clang that's being put into CL?
I think Microsoft will not dump cl.exe anytime soon. In my opinion, they should have rather invested in making CL cross-platform instead of opting in for Clang. No disrespect to your favourite Clang-based tools and "everything Clang is RAD" notion, but I have found Visual Studio tooling much better and mature.
When can we actually use Clang instead of the buggy cl1? I've some projects using C++11/14 features heavily to be ported to Windows, after weeks trying to make them compile with VC I gave up and turned to MinGW.
Is there any early access I can have a test drive?
@windoze.net: We don't have a public release date yet. We will have more information on that soon. In the mean time, we would love to know what kind of issues you are running into while porting your code to Windows using VC. Please free to write to me <sharma.raman@microsoft.com>
Well, at least clang-cl has complete C99/C11 support, unlike even the VS2015. So in fact it is possible to use it as a good C compiler for Windows targets too clang.llvm.org/…/MSVCCompatibility.html
Clang lets you do so much more than that. because of it's modular nature, you can integrate the parser and lexer and semantic analyzer into visual studio, meaning that the IDE could some of the most common bugs and underline it like a spelling error using the exact same engine that the compiler uses to turn the code into machine language. Clang also compiles faster code usually and is a fully compliant C++14 implementation. If Microsoft adopted Clang for all C/C++/Obj-C programming, Microsoft would be right there with Apple, most linux distros that include clang, and google in being 100% compliant with the latest editions of C/C++.
Expression SFINAE is actually great tool for template meta-programming. Its a shame that VS 2015 will not support this.
The problem is C++ mangling crap any linkers created for its own greed. When you know you can port a DLL using a text editor changing mangling name of function from the new one and change RT link chain, you already know something is retard from Microsoft strategy.
Why nobody in Microsoft made a tool to port DLL and LIB when pusblishing a new RT … too many managers and not communicants and not enought workers ?
The reason clang is going ballistic is because Cling will bring salvation for any C++ users waiting for C++ scripting and C++ interpreter. This tiny little effort from Microsoft compiler team was never made, why not make CL.exe interactive at some point to make it a REPL ? It blocked millions of C++ users for decades.
I am waiting Cling like any of the millions C++ users, and for that we need Clang to be abble to generate ABI / API compatible exe on windows. The minute Cling can be launched on windows with both world ( scripting and compiled joined) , I remove Visual Studio C++ and switch forever on C++ Cling.
As for your idea, I think the main point would be the linker capable of digesting Clang output and a reverse side.
The general idea is , javascript will have soon a compiler , rendering .NET and JAVA useless technology or marginal as we can already convert peer reviewed library from C++ to JS, not understanding that point is only digging a grave for Microsoft technology strategy.
If C++ can't digest itself as a script, and javascript compiler is achieved, you won't have any room to sell .NET and Oracle to sell Java.
Is any version of this "clang on Windows with the C2 backend" work (be it preview, CTP, beta or otherwise) available yet?
@Gino Microsoft can (and does) already do that with their own C++ compiler and its syntax highlighting in the Visual Studio IDE.