Don't package osx-gcc-installer

[mistydemeo]

I noticed that when a Homebrew user was having some trouble installing things (in Homebrew/legacy-homebrew#20087), they mentioned that Rails Installer distributes osx-gcc-installer on Mac. I wanted to suggest that it'd be better to look at alternatives.

There are a few problems with osx-gcc-installer, especially when using it on modern Mac versions:

• osx-gcc-installer is essentially a stripped-down version of Xcode 4.1, with all of the headers and compilers from that Xcode. That means that some of the tools it includes are more than two years old at this point. For example, the clang it includes, 2.1 build 163, is very very old and can't compile some C++ software correctly.
• Since osx-gcc-installer is a stripped down copy of an old Xcode, installing it for users who already have Xcode and/or the Command Line Tools for Xcode overwrites their new tools with older ones. That can cause some really nasty problems that are hard to track down later.
• The newest version of osx-gcc-installer is built for OS X 10.7; Xcode 4.1 was never released for Mountain Lion. As a result, some of the system headers it installs are outdated and will break some builds, like anything that uses the availability macros. You can see examples of build failures caused by this in mactoolboxglue.c preventing Python installation Homebrew/legacy-homebrew#20062 and Glib failed to build on 10.8.3 Homebrew/legacy-homebrew#20026.

There are some good alternatives to osx-gcc-installer at this point - Apple's official CLT is free and a small download, for example. If what you're specifically looking for is a package that provides Apple's gcc-4.2, one of AT&T's gfortran packages includes the full gcc-4.2 compiler and installs cleanly alongside Xcode/CLT.

[mpapis]

CLT is the plan, the problem is that Apple has nasty licenses and we need approval from them to embed CLT in installer - without embedding or automating the download the whole idea of all in one installer is busted.

I have tried to get some kind of approval from two angles, but still waiting on it, as soon as we get it we will build new version of installer with CLT.

[mistydemeo]

Aha, makes sense. I didn't know you were asking Apple about it.

Keep in mind that, since osx-gcc-installer is an unauthorized derivative of Xcode, it's not actually any more legal for you to include it without permission from Apple than it is to include the CLT.

(And given the problems osx-gcc-installer causes, you really might be better off telling people to download the CLT on their own - even if that means one extra installer package.)

[mpapis]

actually it is legal to use osx-gcc-installer all sources of the programs included in it are published because Apple is bound by the licenses of this programs - so yes we can use it as it's all legal - but it is not supported by Apple, that's another problem, as long as they do not give a license to use freely CLT (without registration) we will be forced to use other software that can provide compilation toolchain for OSX, at this time most complete is osx-gcc-installer and it is what we can provide. We are working on getting the license - I call it get a license that allows free use of free software, that's all on Apple. As soon they let us use the software freely we will do it.

[mistydemeo]

osx-gcc-installer also includes many Apple system headers, not just external open source tools.

[mpapis]

do you mean that there is anything more over http://www.opensource.apple.com/release/developer-tools-41/ ? do you have any pointers to what it is?

[mistydemeo]

Yes - if you take a look at the file list, you'll find that it contains contents from many of the core frameworks in OS X, like CoreServices, etc.

osx-gcc-installer wasn't built from the source from Apple's developer site. It's just .pkg files ripped out of the Xcode 4.1 installer and tossed into a new one.

Here's the full list of files installed by osx-gcc-installer: https://gist.github.com/mistydemeo/5667947 As you can see, it contains a ton of proprietary Apple framework headers along with the open-source commandline tools. (They're the frameworks found in /System/Library/Frameworks, /System/Library/PrivateFrameworks/, /Library/Developer/4.0/Instruments/Frameworks/, /Library/Developer/4.0/Instruments/PrivateFrameworks/)

[mpapis]

what I understood from this article http://kennethreitz.com/xcode-gcc-and-homebrew.html from @kennethreitz is that all what's included in https://github.com/kennethreitz/osx-gcc-installer is available as opensource / free ... if it's not the case we can repackage excluding what's not should be in there for free distribution.

[samueljohn]

Very interesting discussion.
All I can add is that the osx-gcc-installer really breaks a lot of builds these days :-(
Keep us updated @mpapis if you get approval from Apple. But I am not very hopeful due to their developer license you have to accept to download.

[mistydemeo]

@kennethreitz is incorrect about that; regardless of not including the SDKs, he still has hundreds of the headers from those same frameworks, along with some associated proprietary Apple binaries, etc. For example, the comment at the top of AppleEvents.h from CoreServices as packaged in osx-gcc-installer:

/*
     File:       AppleEvents.r

     Contains:   AppleEvent Package Interfaces.

     Version:    AppleEvents-527.7~1

     Copyright:  � 1989-2008 by Apple Computer, Inc., all rights reserved

     Bugs?:      For bug reports, consult the following page on
                 the World Wide Web:

                     http://developer.apple.com/bugreporter/

*/

All of these files are proprietary, Apple doesn't license them for any kind of redistribution.

If you do decide to strip these out and distribute just the open-source contents, why not do that from a current version of Xcode instead of Xcode 4.1?

[MikeMcQuaid]

Even then you can't just go stripping random binaries even if you know they are open-source; you really need to build them from sources yourselves and redistribute and repackage them. Even if/when you do that there's a bunch of stuff that will break as it expects the proprietary headers.

[mistydemeo]

Also, keep in mind that while osx-gcc-installer specifically omits the SDKs, those don't come with the CLT to begin with. They're essentially the same package. Here's the list of what comes with osx-gcc-installer, according to pkgutil --bom:

gcc4.2.pkg/Bom
clang.pkg/Bom
llvm-gcc4.2.pkg/Bom
DeveloperToolsCLI.pkg/Bom
DeveloperToolsSystemSupport.pkg/Bom
VersionedDeveloperToolsSystemSupport.pkg/Bom
XcodeUserSystemSupport.pkg/Bom
DevSDK.pkg/Bom
X11SDK.pkg/Bom

Whereas the CLT comes with only these two packages:

DevSDK.pkg
DeveloperToolsCLI.pkg

The compilers are now directly a part of DeveloperToolsCLI; otherwise, DeveloperToolsCLI and DevSDK include essentially the same things, aside from the CLT being newer. For comparison, here are the files from the two .pkg files in the CLT: https://gist.github.com/mistydemeo/5670162 You can see that they're the same contents, barring Xcode and OS version differences.

[mistydemeo]

Even if/when you do that there's a bunch of stuff that will break as it expects the proprietary headers.

Those headers are one of the two things causing problems with osx-gcc-installer, also - installing the Lion versions of the headers on Mountain Lion leads to mismatches with what's on the system, along with some of the headers the CLT does install.

[mpapis]

all I can say it's thanks to Apple, I will investigate legality of the osx-gcc-installer and will wait a bit for approval to use CLT, if Apple will not be able to cooperate then the only option for railsinstaller will be to build it's own toolchain as suggested by @MikeMcQuaid

[MikeMcQuaid]

I think the ideal outcome for all parties would be Apple allowing redistribution of the CLT. We've tried there before and been unsuccessful but you're able to resolve that: great. Otherwise I'd suggest railsinstaller put it's own toolchain somewhere that only it finds (and not things like e.g. Homebrew).

[samueljohn]

"I think the ideal outcome for all parties would be Apple allowing
redistribution of the CLT"

-> That would be wonderful.

[kennethreitz]

Not going to happen :)

[mistydemeo]

Ping! Have there been any updates here?

[MikeMcQuaid]

Well, kinda. 10.9 now will auto-install the CLT (like it does with Java on 10.8) if you run something related to it. Would still be nice if we got a 10.8 and below solution that didn't screw with Homebrew.

[mpapis]

Sorry I have no news yet, still waiting on response, maybe @emachnic can tell more, I have a fallback plan but no time to implement it.

[kennethreitz]

10.9 has most of CLTs built-in.

[mpapis]

oh, this is good news (10.9) will save a lot of time

[kennethreitz]

Don't bother contacting apple or investigating, there is legal ambiguity in OSX-GCC-Installer. That's why it exists. Apple can't provide it outside of their lengthy contracts, so I've decided to.

[MikeMcQuaid]

In addition, osx-gcc-installer is still distributing non-open source Apple headers, no? In that case it would be good for @mpapis and @kennethreitz to work out something that is legal and doesn't screw with Homebrew.

[kennethreitz]

Also note that my repository does not include a license.

[MikeMcQuaid]

@kennethreitz If there's a legal ambiguity why not just redistribute the CLT?

[kennethreitz]

GCC-Installer predates CLT, and is actually why CLT exists. Apple has been working hard to make it as distributable as possible within their legal boundaries. Now it's built in w/ 10.9, so problem solved :)

[MikeMcQuaid]

@kennethreitz The problem is that people still install the GCC-installer on 10.8 which breaks Homebrew and will continue to do so until we stop supporting 10.8 (which is at least 3 years away).

[MikeMcQuaid]

@kennethreitz Would you consider removing the binaries for 10.7/10.8?

[kennethreitz]

How does it break homebrew? I've never heard of this before. I've heard from dozens of people that the 10.7 versions works perfectly for 10.8.

[mpapis]

@MikeMcQuaid maybe consider building "something" - your own binary bootstrapper for homebrew - that does not require compilation - I would be happy to use that instead

[MikeMcQuaid]

And having a single download for all versions of OSX defeats the point of the progress in later versions. As said earlier: just redistribute the CLT if you don't care about the licensing; osx-gcc-installer violates the licenses in the same way but with the added bonus of breaking a bunch of stuff like Homebrew for your users.

[mpapis]

@MikeMcQuaid there is no single version, there are two versions right now, and most likely 3rd will be created for OSX 10.9.

Also can you point me to a claim from apple that it does violate it? https://github.com/kennethreitz/osx-gcc-installer states:

References

The source code from Apple for these packages is available here:

• http://www.opensource.apple.com/release/developer-tools-41/

Because when downloading CLT there is a license which explicitly says you need ask apple to download it, as long as Apple does not have any issues with using osx-gcc-installer I feel safer with it then with directly breaking a license ... anyway this project dropped on my priority list as there is this unresolved legal issue which EngineYard as sponsor was not able to help with - what more me simple developer could do?

[mistydemeo]

As stated earlier in this thread, osx-gcc-installer wasn't created from that source - it was taken from the Xcode .dmg, which had the very same license the CLT does now. And osx-gcc-installer does include proprietary Apple headers and tools.

[mpapis]

please show me any apple claims against it, anything else then debugging the files

[mistydemeo]

I'm not aware of Apple prosecuting anyone for redistributing the CLT either. That doesn't make either of them legal.

[mpapis]

as said earlier this is more corporate stuff and I can not do much more personally, I expect @emachnic as a representative of EngineYard to get into resolving it.

[emachnic]

I am still working on getting a company developer license but it just hasn't happened yet. I keep kicking the tires so hopefully there should be some more news soon.

[rafalwojcik]

What about 10.9 version?

[emachnic]

Ilewy what exactly is your question?

On Saturday, November 2, 2013, ilewy wrote:

What about 10.9 version?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/19#issuecomment-27632937
.

Evan B. Machnic
[email protected] [email protected]
Twitter: @emachnic
Skype: emachnic

[rafalwojcik]

RailsInstaller compatible with Mavricks (10.9)

[emachnic]

I haven't had a chance to test it yet. If you have issues, create a GitHub
Issue.

On Saturday, November 2, 2013, ilewy wrote:

RailsInstaller compatible with Mavricks (10.9)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/19#issuecomment-27635515
.

[MikeMcQuaid]

Please use xcode-select --install in Rails Installer on 10.9.

[celynch8]

@MikeMcQuaid can you explain a little more what that means? Sorry - new to this.

[MikeMcQuaid]

@celynch8 On 10.9 to install the compilers you no longer need an Apple Developer or App Store account. You can type xcode-select --install into a console and it will pop up a prompt which, if accepted, will download the Command Line Tools (which includes all the compilers you'd need for Rails).

[celynch8]

@MikeMcQuaid Thank you!!

[giancarlo3g]

I'm stuck for a couple of weeks with the same problem. I installed the osx-gcc-installer on 10.9 and not able to remove it correctly even with the uninstall-devtools -mode=all command or after reinstalling Xcode above it. It seems that for Hombrew, the Xcode installed is 4.1 that is the version of the osx-gcc-installer. Any clues on how can I tell brew that the new version is Xcode 5.0.2?

[MikeMcQuaid]

You can't really tell Homebrew anything. You need to completely remove osx-gcc-installer or Homebrew will break randomly when building tools, unfortunately. This is why we wish osx-gcc-installer wasn't used.

[giancarlo3g]

Right. The only problem is I can't get to fully remove the osx-gcc-installer. I'll keep trying.

[MikeMcQuaid]

Anyone who installs osx-gcc-installer on Mavericks will overwrite system files and break e.g. clang putting it in an unrecoverable state without OS reinstall or backup restores, it seems.

[mistydemeo]

In 10.9, /usr/bin/clang and friends are now xcselect shims that call the real binaries from either /Library/Developer/CommandLineTools or /Applications/Xcode.app/Contents/Developer, as appropriate. osx-gcc-installer overwrites the shims with actual compilers. Looks like neither CLT install restores the shims (as they come with the OS), so installing the CLT over top still leaves the old clang in place.

[MikeMcQuaid]

Ping @kennethreitz for the latest updates here too.

[mtheoryx]

In 10.9.2, xcode-select --install will do a fine enough job, avoiding license issues, prompting the user, and take care of everything.

[azimpradhan]

I have mavericks and I already installed osx-gcc-installer to install command line tools. Now when I try to upgrade to ruby 2.1.1 from ruby 1.9.2 it appears, I get all kinds of errors related to an outdated version of clang. I suspect this is because of osx-gcc-installer
------------------------------------------------------------------------------------------------------------------------------.
checking for clang option to accept ISO C89... (cached) unsupported
checking dependency style of clang... (cached) none
checking how to run the C preprocessor... clang -E
checking whether clang and cc understand -c and -o together... yes

configure: error: No C-89 compiler found

I also get the following message when I run brew doctor
Homebrew doesn't support osx-gcc-installer. It causes many builds to fail and
is an unlicensed distribution of really old Xcode files.

I have also tried to manually install command line tools through the developer.apple.com website. Deleting xcode temporarily gets rid of the error above , but the moment I get it back I get the above message (and I need xcode).

Is there any way I can remove osx-gcc-installer and use the command line tools packaged in xcode-select once I have already installed the osx-gcc-installer. I am asking specifically for the case of Mac OS 10.9 (mavericks). Thanks in advance!

[mistydemeo]

Hi, @azimpradhan,

The only foolproof way I'm aware of to restore the commandline tools on 10.9 after installing osx-gcc-installer is to reinstall the OS.

[mtheoryx]

I believe that running this int Terminal.app should launch a CLI tools install dialog:

xcode --select-install

David Poindexter
Sent from my mobile

On May 22, 2014, at 11:33 AM, Misty De Meo [email protected] wrote:

Hi, @azimpradhan,

The only foolproof way I'm aware of to restore the commandline tools on 10.9 after installing osx-gcc-installer is to reinstall the OS.

—
Reply to this email directly or view it on GitHub.

[mistydemeo]

That won't work in this case.

10.9 comes with shims for the commandline tools in /usr/bin. The real commandline tools are installed into /Library/Developer/CommandLineTools/. When the commandline tools are installed, the xcode path gets updated so the shims will call the CLT versions instead.

What's happened here is that osx-gcc-installer overwrote the shims, and the CLT install does not install anything to /usr/bin that would overwrite them. The only way to get them back is an OS reinstall, because they only come with the OS.

[azimpradhan]

Thanks @mistydemeo , I was hoping it wouldn't have to come to this, but I guess i'll have to reinstall the OS to upgrade ruby 2.1.1 and rails 4.