What’s this ‘CoApp’ all about?


Last week, I blog’d about a new open source project that I’ve launched called “CoApp” (The Common Opensource Application Publishing Platform). As I’ve mentioned on the project site, “CoApp aims to create a vibrant Open Source ecosystem on Windows by providing the technologies needed to build a complete community-driven Package Management System, along with tools to enable developers to take advantage of features of the Windows platform.”

Ugh—a mouthful—and all chocked full of them shiny marketing words.(Uh.. yeah, I know wrote that.).

 

So, what does that mean?

Well, while Windows provides some pretty good stuff for packaging applications in the form of Windows Installer* technology (aka “MSI”), the down side is that the open source community hasn’t really picked it up in the same way that they have picked up packaging on other platforms where they create repositories and distributions of software, and so we’re missing out on having these nice, consistent collections of all these great open source apps.  That’s where I really want to be.

‘Course, my pappy always used to tell me “it don’t take a genius to spot a goat in flock of sheep” … Sure, it’s easy to see what the problem is, question is, how do we go about fixin’ it?

 

Last fall, I started to sketch out what that should look like, and what it would take to get there.  After a few months of poking the right people, I started to get agreement here at Microsoft that this really is a great idea, and we should be spending time on it. (And, ‘course, by ‘we’ I mean ‘me’)  I know from personal experience with building open source software on Windows, that things are not only sometimes tricky, but often downright impossible to build correctly, and even harder to make sure the software is built in such a way that anyone on Windows could use it.  I’ve come up with a plan for building a set of tools to help open source software build better on Windows, along with automating the packaging in such a way that will allow us to build yet more shiny tools to locate and install them.

Along with the tools, we’re going to need to lay down some guidance on how to use them to build packages that play nice with each other—I want to make sure that I’m never running into “DLL Hell”, never having to search for missin’ bits, and always getting the right package for the right job.  At the same time, I really want to use some optimization techniques to help open source software run better on Windows.

 

Starting with ApacheCon last fall, I began to reach out to people I know in open source communities, not only to get their buy-in that this is a good idea, but solicit their help. I’ve already secured a handful of folks who are interested in helping, and I can always use a few more.

Over the course of the next month or so, we’ll be the filling in the details on what all of this looks like on the project site, and discussing the merits on the mailing list. From there, we’ll begin to build the tools, and with a bit of luck, we’ll start producing packages a few more months after that. We’ll probably start with the packages that make the most sense (Apache, PHP and Python) and work our way out from there.

 

And just how does Microsoft fit into all of this?

Well, the folks here at Microsoft have recognized the value in this project—and have kindly offered to let me work on it full-time.  I’m running the project; Microsoft is supporting my efforts in this 100%. The design is entirely the work of myself and the CoApp community, I don’t have to vet it with anyone inside the company. This really makes my job a dream job—I get to work on a project that I’m passionate about, make it open source, and let it take me where it makes sense.

 

Sure, it’s a large project, but I’m pretty sure that we’re headed in the right direction—if you’d like to come out and help (or even just come get more details about what I’m talking about), you can start at http://coapp.org.

 


* I know, some people don’t particularly like MSI, but trust me, it’s all in how it’s used—ya don’t blame the horse for throwin’ a shoe.



Comments (15)

  1. commenter says:

    I thought of a slogan.

    Windows Installer – "if you’re losing fingers you’re placing your hands too close to the huge whirling blade"(tm)

    😉

  2. commenter says:

    Also, this sounds like a great idea and I hope it’s a success.

  3. Chris says:

    Are you going to use other good open source technologies like cmake, gcc etc. as these would make porting to Linux/Unix easier?  ie. build on one platform, know that it will build with only minor changes on all.

  4. GarrettS says:

    @Chris

    I’ve tried that. Unfortunately, Windows ain’t unix. automake is not really not designed to work with Windows, and even if it did, the results are not really compelling.

    What’s needed is a slightly different approach.

    Most of the software already builds for Windows, admittedly with a bit of pain and suffering.

    I’m looking to fix that by streamlining and organizing it all.

  5. Jonathan Brandmeyer says:

    I would like to point out two critical pieces of tech that you will need in order to use WinSxS appropriately with OSS libraries. You need to establish a clear mapping from libtool-style library interface version numbers to WinSxS compatibility information. Then, you will need to modify pkg-config such that it produces the right -l and -L arguments to gcc (or cl) to direct the compiler into the correct WinSxS directories.  Finally, you will need to modify either collect2 or GNU ld to generate SxS manifest information for the compiled DLL’s and EXE’s based on the manifests of the linked-in SxS DLL’s.

    While some OSS will work with MSVC, most do not have the requisite project files and MSVC does not support C99. If you want to support the broadest variety of OSS, then you should aim towards standardizing on the use of MinGW+MSYS as the standard build environment.

  6. GarrettS says:

    @Jonathan Brandmeyer -> part of the CoApp project is building out a repository of shallow-forked OSS apps and libraries to build MSVC build scripts and optimizations.

    of the 100+ OSS libraries and apps I’ve been working with in the lab, every one will compile on MSVC given enough love.

    The CoApp toolset will address and automate most of the hassle around these.

  7. Steve Walker says:

    There’s a reason why most UNIX-based software is hard to build on Windows – it’s a different platform! I would be much more interested to know what Microsoft are doing to foster/encourage open source development on their own platform rather than simply making it easier to ‘steal’ the work of all those UNIX guys.

    😉

  8. Anonymous says:

    dammit, do you guys heard of cygwin, which is opensource and have kinda simple package managment?

    Even though, this stuff is not required, cause windoze is terrible os and when you need some oss apps, you should use oss os. Logic, huh?

  9. Markt9 says:

    Open Source on Microsoft (or other non-free OS)?  Is that an oxymoron?

    Are you a crack dealer offering MSDN licenses to developers?

    Have you tried talking to Richard Stallman to understand what and why Open Source is important to people?  (please save the "Hippie" comments and focus on his vision.)

    Are you following the standard Microsoft "Embrace, extend, and extinguish" play book?  Collecting a paycheck and whoring yourself out?

    It’s not too late to join the Debian team and enjoy the true benefits of apt-get.

  10. Faldegast says:

    I actually have an idea that would make building UNIX/Linux software easy.

    While i have tried cygwin and do not like the solution i have done some thinking.

    Generic Linux Compatibility Layer (http://brainstorm.ubuntu.com/idea/22842/).

    The ability to run Linux applications in Windows the way Wine run Windows applications in Linux.

    "Ubuntu/OSS Desktop" Application suite (http://brainstorm.ubuntu.com/idea/22840/).

    With the Linux Compatibility Layer the applications of full distributions like Ubuntu or RHEL could run even without recompiling the packages.

    It would give the flexibility of a Virtual Machine but run with native speed as it is only a compatibility layer.

  11. scetical says:

    Fear not the almighty M$ will do the right thing by Unix/Gnu/Linux. "NOT" And the writer of this article will be treated fairly when M$ rams it up his rump!

  12. blueQ says:

    @Markt9

    Don’t mention "Open Source" to Richard Stallman. 😉

    But I would agree with "There might be a CoApp Windows trap".

  13. E.T. Anderson says:

    I’m glad to hear of this project. Centralized package management is one of the (perhaps not the most important, but still) things that makes Linux/Unix more secure than Windows. Increased security, by any amount, benefits everyone.

    Furthermore, "Embrace, Extend, Extinguish" can work both ways. By getting people used to F/OSS on Windows, they may be more willing to try the total Free experience (Linux). Even if they do not, it hurts no one for Free Software to be more easily used on Windows.

  14. James K. Lowden says:

    Garrett, cmake != autoconf.  cmake deserves your careful consideration.  

    As the maintainer of FreeTDS, I’ve run into a few headaches supporting Win32:

    1.  nmake.  I can’t distribute a .dsw or .sln file with any certainty it will work because I don’t know — and, please, shouldn’t care — what version of the compiler the user has.  I distribute an Nmakefile to build the project.  nmake, however, lacks many features that make writing makefiles easier, and the result was hard to create and is difficult for others to modify.  (cmake might have been a better choice except that then for my one project you would need to get/build cmake first.  If it’s part of the standard build environment, that problem disappears.)

    2.  missing/nonstandard C library functions.  There’s no getopt(3), no readdir(3). Winsock functions don’t set errno.  

    3.  syscalls.  There’s no fork(2), no chown(2).  There’s no USE_POSIX option, so there’s no chmod(2), only _chmod(2).  (POSIX might be "deprecated" in Redmond, but it’s alive and well elsewhere.)  Try writing anything like a daemon, and you’ll soon find a short list of crucial functions that aren’t there.  And, you know, comrade, that’s no accident.  

    I’ve frequently wished for a "libcxor" project that would implement missing POSIX functions in terms of Win32.  It wouldn’t be hard, technically, and it would make porting code much easier.  If done as open source, I think it would get community support.  

    4.  VC++.  Some genius decided strcpy(3) deserves a warning.  I can’t turn stuff like that off with one well-defined command-line option, so I have to keep adding new -D preprocessor macros to produce a clean compile.  

    5.  Installation.  What should "make install" do?  In Unixland we have hier(7), PREFIX, and install(1) to go by.  There’s no system/user division in Windows, so should I plop my DLLs into %SYSTEMROOT%?  

    As you can see, the need isn’t so much for fancy tools and as for functionality in the base system.  If Windows came with GNU/BSD make, a complete C runtime library, and an installer that operated with command-line options and text files, it wouldn’t be a step-child target environment for my project.  

    –jkl

  15. Vivien says:

    I maintain an application which I cross compile on Linux for Windows. The reason I don’t generate MSI or others is that they don’t work fine in a cross compiling environment.

    Do you plan to make the software being able to work on Linux in a cross compiling environment?