Created on 2011-07-26 17:02 by jonforums, last changed 2015-03-14 01:46 by ned.deily. This issue is now closed.

Messages (86)

The use of -mno-cygwin at http://bit.ly/qAIsZV causes build failures with versions of MinGW GCC. For example http://bit.ly/llK3f3 I recently ran into it trying to build a Mercurial snapshot using a MinGW GCC v4.6.2 flavor http://pastie.org/2274486 Why is -mno-cygwin still be used, and is there any reason why it can't be removed? Thanks, Jon

msg141173 - (view) Author: Ruben Van Boxem (rubenvb) Date: 2011-07-26 18:06

I can confirm the option has been removed. For those that don't want to believe a random person's comment on a bugtracker, here's the commit: http://repo.or.cz/w/official-gcc.git/commit/2637551aa9314c46cf4205d435ab5e8944e9ac8a The changelog is a bit cryptic, but note the last two items "removing documentation from removed option".

msg141230 - (view) Author: Éric Araujo (eric.araujo) * Date: 2011-07-27 15:03

Hi! Thanks for the report. This can’t be fixed in 2.5, 2.6 or 3.1, which are in security-fix only mode, but we can do something for the active branches. A quick web search finds reference of this deprecation/removal as far as 2007. Does anyone have more official documentation? If we can find the first gcc version that deprecates this option, then I’ll be able to make a patch with a version check (in order not to change previously working code). (In the future, it would be nice of you not to use URL obfuscators or pastebins. Such services offer no guarantee of continued availability of hosted resources or links, but it is useful to keep information on this bug tracker for future reference. Thank you. The first link is this: http://hg.python.org/cpython/file/4feb889d3bed/Lib/distutils/cygwinccompiler.py#l296 The second: http://stackoverflow.com/questions/6034390/compiling-with-cython-and-mingw-produces-gcc-error-unrecognized-command-line-op The compile log from the third link is attached as a file.)

msg141231 - (view) Author: Éric Araujo (eric.araujo) * Date: 2011-07-27 15:07

I forgot to answer this: > Why is -mno-cygwin still be used, Simply because distutils had no dedicated maintainer for a long time, and because nobody was aware of this change. I don’t think any Python core developer is using Cygwin. > and is there any reason why it can't be removed? For distutils, we have to work on eggshells when fixing bugs to make sure we don’t break third-party code that relies on known bugs or works around them. That’s why I propose adding a version check* instead of just removing the option. * i.e. removing -mno-cygwin if self.gcc_version >= some version

msg141265 - (view) Author: Ruben Van Boxem (rubenvb) Date: 2011-07-27 19:03

> Does anyone have more official documentation? The commit I linked to has the full option removal at October 7 2010 (see the fifth item in the changelog entry). Any GCC (major) version released after that will have it completely removed. Deprecation happened (according to the first doc change mentioning this option as being deprecated) on 2009-03-25: http://repo.or.cz/w/official-gcc.git/commit/6609be22531f447aeddbb3f670ad7883036cb23f It looks like GCC 4.4 and up have this change in them, by looking at their commit logs. Anyone using GCC 4.3 on MinGW should be shot (but that's just my humble opinion, rather radical I admit ;-) ) Note this has really nothing to do with current Cygwin or the cygwin platform; MinGW was part of the Cygwin source for a long time, and every MinGW compiler was burdened to search the Cygwin header and lib paths. These commits fixed that problem. MinGW(-w64) is a native compiler for Win32 just as MSVC is. Links to msvcrt.dll and has no POSIX translation/compatibility layer like the Cygwin DLL. May I ask for a reconsideration to commit a fix for this for Python 2.7 at least? With the version check it doesn't hurt anyone, instead it helps prevent further confusion and support requests on the MinGW side. Distutils pop up everywhere, and the projects depending on them rely on them working correctly. Thanks

I have confirmed on Win7 that the following 32bit MinGW flavors still recognize -mno-cygwin option and build without error: (4.5.2) http://tdm-gcc.tdragon.net/download (4.5.4) http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Automated%20Builds/ using the mingw-w32-1.0-bin_i686-mingw_20110624.zip download ...by compiling with: [i686-w64-mingw32-]gcc -Wall -mno-cygwin -o helloworld.exe helloworld.c ...where helloworld.c was: #include <stdio.h> int main(int argc, char *argv[]) { printf("Hello World!

"); return 0; } I have not yet checked the latest mingw.org download from the following, but I expect that it also recognizes -mno-cygwin. I will check later and report back if appropriate. (4.5.2) http://sourceforge.net/projects/mingw/files/ Given Ruben's 4.4 comment, something is odd here. Looks like some gcc source spelunking as these doc links make no mention of the removal: http://gcc.gnu.org/gcc-4.6/changes.html http://gcc.gnu.org/gcc-4.5/changes.html http://gcc.gnu.org/gcc-4.4/changes.html and the gcc manual is inconsistent. This summary mentions the option http://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/Option-Summary.html#Option-Summary but the details sections don't mention the option http://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/i386-and-x86_002d64-Windows-Options.html#i386-and-x86_002d64-Windows-Options http://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/i386-and-x86_002d64-Options.html#i386-and-x86_002d64-Options Ah, our good friend grep.

should the question be "what's the first mingw gcc version that -mno-cygwin usage unnecessary" rather than finding the first version the option was removed? and, does it matter whether you're building on win for win, or cross compiling for win from nix? sadly, i don't know gcc internals but a post to the mingw and/or mingw-w64 ml's should get the right people weighing in. btw, can distutils do a lightweight configure-like check for feature presence rather than going a gcc version check?

msg141296 - (view) Author: Éric Araujo (eric.araujo) * Date: 2011-07-28 14:12

> May I ask for a reconsideration to commit a fix for this for Python > 2.7 at least? With the version check it doesn't hurt anyone There’s a misunderstanding: I explained why 2.5, 2.6 and 3.1 can’t be fixed, but if you look at the versions at the top of this page, you’ll see 2.7, 3.2 and 3.3 listed. This will get fixed in these versions. > Ah, our good friend grep. Thanks for the exploration. This looks complicated. > should the question be "what's the first mingw gcc version that > -mno-cygwin usage unnecessary" rather than finding the first version > the option was removed? If removing the option in these versions causes no change (i.e. doesn’t break any code), I agree. > and, does it matter whether you're building on win for win, or cross > compiling for win from nix? I’m afraid I don’t know enough about Windows and MinGW to answer that. If we can’t be sure about versions and consequences here, I’ll go to the MinGW ML. > btw, can distutils do a lightweight configure-like check for feature > presence rather than going a gcc version check? There is a config command that’s not very used, but it would be IMO overkill to use it in the compiler code.

>> and, does it matter whether you're building on win for win, or cross >> compiling for win from nix? >I’m afraid I don’t know enough about Windows and MinGW to answer that. If we can’t be sure about versions and consequences here, I’ll go to the MinGW ML. Ruben's on it :) http://sourceforge.net/mailarchive/message.php?msg_id=27864860

are you ok with a targeted patch similar to what's being discussed at http://sourceforge.net/mailarchive/message.php?msg_id=27895558 assuming the regex search the output of `gcc -dumpspecs` idea is valid?

msg141614 - (view) Author: Roumen Petrov (rpetrov) * Date: 2011-08-03 20:33

it is save to remove -mno-cygwin from Mingw32CCompiler

msg141661 - (view) Author: Éric Araujo (eric.araujo) * Date: 2011-08-05 15:28

The necessity of walking on eggs with the distutils codebase restrains me. I’ve read the thread on sourceforge (thanks Ruben) but don’t have enough information yet to decide whether to do a version check, call gcc -dumpspecs or remove the option altogether. BTW, there’s no need to use re.search if you’re not using a regular expression: “'no-cygwin' in output” will work.

shortly after opening this issue i removed -mno-cygwin from my 2.7.2 install and have had no issues on win7 32bit. but i understand you're hesitation. regardless what you decide, please consider placing a summary note in the source comments as a safety net. if you'd like me to try building a limited set of extensions to help with your decision, let me know. i have the following mingw toolchains on my win7 32bit system: * tdm gcc 4.5.2 - http://tdm-gcc.tdragon.net/ * mingw-w64 prefix-stripped gcc 4.5.4 - http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Automated%20Builds/ * Ruben's gcc 4.6.2 personal build - http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/rubenvb/4.6.2-1/ i'm not speaking for Ruben, but as he maintains https://github.com/rubenvb/MinGW-w64-build-scripts you might try cajoling him if you feel you need more test results ;) re: re.search, understood, thanks...quick-testing.

Would it be practical to have a trivial compilation test to see if we are capable of using GCC with -mno-cygwin and if so, use it, otherwise drop off? I think GNU autotools uses a similar strategy for detecting compiler capabilities.

msg146499 - (view) Author: Éric Araujo (eric.araujo) * Date: 2011-10-27 16:03

> Would it be practical to have a trivial compilation test to see if we > are capable of using GCC with -mno-cygwin and if so, use it, otherwise > drop off? I think GNU autotools uses a similar strategy for detecting > compiler capabilities. The config command does such things, but the compiler classes don’t. I’d prefer something more lightweight, like a version check (see msg141231).

msg161514 - (view) Author: PJ Eby (pje) * Date: 2012-05-24 14:53

Just adding a data point: I use the "mingw32" compiler setting with the Cygwin GCC installation, to compile extensions for standard Windows Python (*not* Cygwin Python). It would be good if any change doesn't break this scenario -- assuming it's not already broken in newer Python versions. (For this scenario, AFAIK, the no-cygwin option is required, or at least I believe it was the last time I intentionally compiled something.)

msg165178 - (view) Author: Craig McQueen (cmcqueen1975) Date: 2012-07-10 08:20

I've come across this issue when trying to build extensions for Python 3.3 on Windows, needing a recent enough MinGW to provide a library for msvcr100. See issue #15315.

msg165181 - (view) Author: Martin v. Löwis (loewis) * Date: 2012-07-10 09:30

I have gcc 3.4.4 in my cygwin installation, and it still needs the -mno-cygwin option, else the resulting binary will link with cygwin1.dll (which is undesired).

msg165182 - (view) Author: Martin v. Löwis (loewis) * Date: 2012-07-10 10:11

I just installed the cygwin gcc4 package; this gives me gcc 4.5.3. In this version, -mno-cygwin is still recognized, and gives this error message: gcc: The -mno-cygwin flag has been removed; use a mingw-targeted cross-compiler. So it seems that removing the -mno-cygwin flag is definitely incorrect, as it will result in a cygwin binary. I then installed the mingw-gcc-core package, which gave me the utility i686-pc-mingw32-gcc, which indeed is able to create a (nearly) correct binary. So I think we should check whether i686-pc-mingw32-gcc exists in the path, and if so, use it (without a -mno-cygwin flag). If it doesn't exist, we should continue to use gcc -mno-cygwin. My remaining concern with i686-pc-mingw32-gcc is that it still links with msvcrt.dll in addition to linking with msvcrXY.dll; this is incorrect. There is another concern that this applies to 32-bit mode only; in 64-bit mode, i686-w64-mingw32-gcc should be used. However, this is issue 4709.

msg165184 - (view) Author: Martin v. Löwis (loewis) * Date: 2012-07-10 10:19

The 64-bit compiler is actually called x86_64-w64-mingw32-gcc.

msg165210 - (view) Author: Craig McQueen (cmcqueen1975) Date: 2012-07-10 23:31

It would be great if this could be sorted out in time for Python 3.3. Otherwise I don't think we'll be able to use MinGW to build extensions in Windows. Unless there is a version of MinGW which supports the -mno-cygwin option, as well as libmsvcr100.

msg167620 - (view) Author: Ruben Van Boxem (rubenvb) Date: 2012-08-07 13:02

Checking for a compiler's file name is stupid. Native Windows gcc is just "gcc.exe", Cygwin native GCC is also "gcc". Some have a lot of toolchains in PATH at the same time. That's not the right way to handle this kind of thing.

msg179175 - (view) Author: Geert Jansen (geertj) * Date: 2013-01-06 09:48

*bump* I just installed MinGW 2.6.2 32-bit on Windows XP. It doesn't accept -mnocygwin and there is no binary "i686-pc-mingw32-gcc" either. It would be great if you could agree on an approach and get this fixed. This impacts a lot of users that want to build extensions on Windows. In the mean time users can find a hack to work around the issue here: https://gist.github.com/4466320

msg179176 - (view) Author: Martin v. Löwis (loewis) * Date: 2013-01-06 10:07

Geert: Someone will have to provide a patch, else the issue will go nowhere. I will insist that the patch continues to support "older" gcc installations (from 2012), at least if implemented as a bug fix to older Python releases.

msg181830 - (view) Author: Roumen Petrov (rpetrov) * Date: 2013-02-10 18:27

In scope of this issue I would like to propose following patch set. First step is remove checks for versions used in past millenium, i.e. to avoid checks for 15 year old binaries.

msg181831 - (view) Author: Roumen Petrov (rpetrov) * Date: 2013-02-10 18:30

Next step is to propose customization for cygwin&mingw compilers.

msg181834 - (view) Author: Roumen Petrov (rpetrov) * Date: 2013-02-10 18:34

And optionally If someone disagree options m{no-}cygwin to be removed I would like to propose a patch '..check if cygwin/mingw... -m{no-}cygwin' to restore it for GCC before 4.6x.

msg181865 - (view) Author: Matthias Klose (doko) * Date: 2013-02-11 00:36

2.7, 3.3 and 3.4 now have AC_CANONICAL_HOST. So the correct compiler should be picked up. please could somebody validate this? or are explicit --host and --build options required for configure?

msg182090 - (view) Author: Martin v. Löwis (loewis) * Date: 2013-02-14 11:52

doko: this issue is about Windows; autoconf is not being used here (plus the "correct" compiler would be MSC).

msg182277 - (view) Author: Roumen Petrov (rpetrov) * Date: 2013-02-17 16:31

Hi Matthias, This issue is only for windows. In scope autotool based builds compiler customization is used to 'transfer' some build settings (flags, options) to distutils. This include compiler set in make macro (variable) CC. Transfer is not complete but this is distutils issue out of scope here. P.S. (uploaded a patch to avoid syntax warning)

Guys, this looks really bad and inconveniences a lot of users. You install the latest MinGW and Distutils from their default location, try using them on **anything that requires compilation**, and get the cryptic gcc -mno-cygwin error (after having to edit the obscure distutils.cfg, of course). Aren't Python / distutils supposed to be cross-platform? It's already hard enough to find distutils / pip setup instructions for Windows, shouldn't they at least **work**? After removing -mno-cygwin from cygwincompiler.py, I get another obscure -mdll error. This is ridiculous. If you can't agree on a patch that detects both new and old compilers, can't you split cygwincompiler.py into several versions, or somehow provide separate mingw32-old and mingw32-new options?

msg182737 - (view) Author: Roumen Petrov (rpetrov) * Date: 2013-02-23 12:40

> Dan added the comment: > > Guys, this looks really bad and inconveniences a lot of users. You install the latest MinGW and Distutils from their default location, try using them on **anything that requires compilation**, and get the cryptic gcc -mno-cygwin error (after having to edit the obscure distutils.cfg, of course). > > Aren't Python / distutils supposed to be cross-platform? It's already hard enough to find distutils / pip setup instructions for Windows, shouldn't they at least **work**? After removing -mno-cygwin from cygwincompiler.py, I get another obscure -mdll error. This is ridiculous. Yes . This is reason to pack many changes in one archive "issue12641-modernize_cygwin&mingw_compilers.tar.gz <http://bugs.python.org/file29030/issue12641-modernize_cygwin%26mingw_compilers.tar.gz>", i.e. to remove all checks for tools used in previous millеnium. My oldest compilers are : a) i386-mingw32msvc-gcc (GCC) 3.4.5 (mingw special) Copyright (C) 2004 Free Software Foundation, Inc. b) gcc.exe (GCC) 3.4.5 (mingw-vista special r3) Copyright (C) 2004 Free Software Foundation, Inc. Check for -m{no-}cygwin flags is optional. I can not found reason this patch to be applied, as with implementation of compiler customization this is for developer guide. Roumen

msg183032 - (view) Author: Martin Fiers (Martin.Fiers) Date: 2013-02-26 08:49

This also affects our software. I agree with Dan (danmbox): I don't understand; so many people depend on it and yet an out-of-the-box solution doesn't work. I don't want to break the distutils package of our users because we use mingw. Within one Python script, I managed to fix it using this before the setup call: if isWindows(): """ Fix bug in cygwinccompiler: removed -mno-cygwin. This is fixed in cygwinccompiler_new. We hacked the distutils.ccompiler : def new_compiler : It uses sys.modules to fetch the compiler By modifying the sys.modules, we can choose our own compiler version. (this is a bug that's out there for quite some time) """ import cygwinccompiler_new import distutils.cygwinccompiler import sys sys.modules["distutils.cygwinccompiler"] = cygwinccompiler_new ..if I then later run setup(...), it will use my new cygwinccompiler_new, that has the '-mno-cygwin' line removed. However, when you want to install new packages using pip from the command-line, I cannot find a suitable fix (except if I would replace the distutils.cygwinccompiler before pip'ing, then put it back). For afaik, distutils cannot be virtualenv'ed, right? So we cannot even fix the issue in a virtual environment. If it is not possible to find out what version of gcc implemented it first; can't you simply use a pragmatic solution and run "gcc -mno-cygwin": if it gives an error, then remove the option. That would need the least testing and would fix the issue.

Nice partial work-around. I think it's quite clear that the decision makers for this bug have not been making rational decisions for a year and a half, so we can't really expect change. This being the open-source world, the only recourse is publicizing the issue (or is there some bounty system for python?). So I suggest you blog about it.

msg183077 - (view) Author: Éric Araujo (eric.araujo) * Date: 2013-02-26 20:00

> I don't understand; so many people depend on it and yet an out-of-the-box solution doesn't work. It is because of the combinations of some facts: - Python contributors are volunteers; - distutils is used a lot and quite brittle, which is a combination that makes us extremely cautious with changes; - I am not a Windows user or developer; - Martin knows Windows but apparently not Cygwin; - nobody submitted patches until this month; - nobody reviewed patches after they were attached. You can choose to believe that we are unresponsible, stupid, etc. or accept that in an open, volunteer-based project, subtle changes in a niche part of the code are not committed right away. Whatever we do, we get blamed for moving too slowly and for changing Python too fast (not by the same people :). The good news is that Roumev submitted clear patches, and that Trent Nelson put together Snakebite, a set of computers for Python developers that run various OSes including Windows. Using these machines to test the patches, we can now make progress.

msg183081 - (view) Author: Martin Fiers (Martin.Fiers) Date: 2013-02-26 20:28

Dear Eric, I never said that anyone of these volunteers is unresponsible/stupid/whatsoever. It was also never my intention to express myself in this way, so I apologize if you felt harmed in any way. I was just suprised that this issue exists for so long. And since I only recently started developing on Windows, I was a bit knocked over by the additional effort that was needed to get things working (for which I blame Windows, not Python!). I've been developing for some time now and I understand completely that it's far from trivial; given the complexity of the software and fragility of distutils. So kudos to the developers that put their hours into this! Anyway, I have a workaround using some distutils manipulations now (see my previous post), which works for now, and I'm looking forward to a permanent fix! With best regards, Martin

That's bull, Eric. This is not about a corner case in cygwin. This is about mingw, which is the **main free software that builds executables on Windows**. You know, for when you don't want to require your users to install Visual Studio. Additionally, both you and Matthias imposed artificial conditions that made it unlikely for patches to be created (search for "will insist"). Now, I have to agree that the larger python community (and not an under-resourced team like your good selves) should be involved in distutils (or choose and **support** a different package manager). But I'm not sure where I could file a bug for that (again, blogging may be the best choice).

msg183224 - (view) Author: Éric Araujo (eric.araujo) * Date: 2013-02-28 17:17

[Martin] Thanks for the message. My previous message actually quoted Dan, not you, so no apology is necessary. [Dan] > That's bull, Eric. Could you rephrase this? English is not my native language. > This is not about a corner case in cygwin. Oops I confused cygwin and mingw, apologies. > This is about mingw, which is the **main free software that builds > executables on Windows**. You know, for when you don't want to require > your users to install Visual Studio. Let me state that I fully sympathize with people who use Windows and still want to use free software as much as possible, even though some version of Visual Studio is now available at no cost. Again, the fact is that many core developers don’t use Windows, and the official CPython builds use MSVC, so from this viewpoint, MinGW is a niche platform for CPython. It does not mean that patches are rejected, just that it’s harder for me to assess them. > Additionally, both you and Matthias imposed artificial conditions that > made it unlikely for patches to be created (search for "will insist"). Martin used that in a comment about backward compatibility, which we do take seriously. Roumen’s patches don’t break backward compat if I read them correctly. > Now, I have to agree that the larger python community (and not an > under-resourced team like your good selves) should be involved in > distutils Yes, more feedback, bug reports, patches and reviews from the community would help. > (or choose and **support** a different package manager). But I'm not > sure where I could file a bug for that An effort is ongoing to define the successors of distutils. One problem is that it lacks features (like dependencies), and at the same time it tries to be too much (build, upload, install). The community (Python core developers, packaging tools developers, users) is working on these things. > (again, blogging may be the best choice). If you want to get involved in the packaging discussions, you can join the distutils-sig mailing list. If you want to push this specific issue forward, testing the patches on your system would help. If you want to support Python on MinGW in general, you can contribute a buildbot to our continuous integration system, so that we can see when things break.

[Eric] >> This is about mingw, which is the **main free software that builds >> executables on Windows**. You know, for when you don't want to require >> your users to install Visual Studio. > Let me state that I fully sympathize with people who use Windows and > still want to use free software as much as possible, even though some > version of Visual Studio is now available at no cost. Again, the > fact is that many core developers don’t use Windows, and the official > CPython builds use MSVC, so from this viewpoint, MinGW is a niche > platform for CPython. It does not mean that patches are rejected, > just that it’s harder for me to assess them. I'm going to advocate why mingw-based toolchains (mingw.org and mingw-w64) should be considered important platforms for CPython by giving some background info. An important part of this issue has nothing to do with the politics of "free software" and everything to do with the technical benefits of using mingw-based toolchains on Windows. While you may be aware of the technical benefits, the following are important to me: 1a) MSVC forces you to link against a specific CRT version, and I don't get to decide which CRT version via a build setting. To the best of my knowledge the only way to link against another CRT version is to use another MSVC version. Many times it's important for all parts of a codebase to link against the same CRT version to prevent well-known cross-CRT object usage failures. While Mingw toolchains typically link to msvcrt.dll by default, they allow you to specify a CRT version via spec file tweaks. 1b) Often the most practical/stable way to build mixed codebases using CRT functionality is to link everything against msvcrt.dll. Using MSVC, the only way I'm aware of is to use old-and-slow VC6 which is not easily available. Updated and performant mingw toolchains enable one to link against msvcrt.dll and other CRT versions via spec file hackery. 2) While VC++ is a good IDE, many of us would like to simply use the command line compiler tools and not clutter our environment with a huge IDE we never use. MSFT has a poor record on this front, and after many fine years of making the command line tools available as part of the Windows SDK, they've regressed back to their lame ways in Windows 8 by removing the cmd line compiler tools from the SDK and embedding into the IDE similar to the old VC6 days. Enticing as it may be, I'll stop here rather than lurch into sailor-speak rage rant. There are other benefits, but (1a) and (1b) are very important for many real-world usage scenarios on Windows. For example, easily building mercurial with mingw on Windows that work with the official MSVC-built Python. Extend this to other widely used Python and non-Python libraries. Extend this to those building binary libraries on Linux/OSX via mingw for their Windows users. Also, there are other build automation awesomeness things you can do with mingw toolchains. For example, wouldn't it be great if you could download your source software, a complete mingw toolchain, and build the software on Windows in a sandbox using the mingw toolchain? You can with mingw and mingw-w64 toolchains. Nothing to pre-install and highly automated with minimal user intervention.

There's an additional aspect. Even though I don't use Windows as a development platform, I care about being able to propose Python to clients as a cross-platform technology (instead of, say, Java). Having an essential piece of Python infrastructure fail miserably on Windows impacts Python's appeal / credentials. To clarify another point, I think that Python should pick and support an "official" package manager (be it distutils, pip or whatever) not eventually, but ASAP. For the time being it looks like distutils is a keystone, therefore it needs to work.

Eric, I'm assuming you are the key decision maker for any fix. As I want to see this issue stay razor focused, where specifically do you need community help? Depending upon your needs, I can test on my win7 32bit notebook with official python 2.7.3. It also has arch 3.7.9 and ubuntu server 12.10 virtualbox vm's. I don't build python from source with mingw and my win8 64bit box isn't ready yet. The win7 box has mingw gcc 4.6.2 32bit and mingw-w64 32bit toolchains, both of which have msys and autohell goodies. I'm swamped these days but will carve out time from somewhere.

msg189748 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-05-21 13:08

I'd really like to get a resolution on this issue so I've tried to gather some more information about this problem by asking some questions in the mingw-users mailing list. The resulting thread can be found here: http://comments.gmane.org/gmane.comp.gnu.mingw.user/42092 This issue concerns users of distutils --compiler=mingw32 mode. Normally this is used to build with mingw gcc in which case it is currently broken because of -mno-cygwin. It has been suggested above that some may be using it to build with cygwin's gcc. The -mno-cygwin option is there for those using cygwin's gcc. To summarise the points (see the thread on mingw-users for more info): 1) -mno-cygwin has never had a meaningful effect in mingw. 2) -mno-cygwin produces an error in recent (~2011 onwards) mingw releases and will do for all future releases. This prevents distutils from building extensions with mingw. 3) -mno-cygwin only ever had a meaningful effect for cygwin's gcc 3.x where it could be used to build binaries that did not depend in cygwin.dll. 4) -mno-cygwin was always considered an experimental feature and its use is discouraged. 5) -mno-cygwin was removed from cygwin's gcc in the transition from 3.x to 4.x without any deprecation period (as it was an experimental feature). In gcc 4.x it produces an error preventing build. 6) The recommended way to replace the -mno-cygwin option is either to use mingw or to use cygwin's cross-compilers. So there are two types of breakage affected by -mno-cygwin: A: Anyone trying to use recent and future mingw versions to build extensions with distutils in the way that is described in the distutils docs. For this group distutils has been broken for 2 years and will continue to be until the -mno-cygwin option is removed. B: Anyone who is using distutils with --compiler=mingw32 but using cygwin's gcc 3.x instead of mingw's gcc to build an extension for a non-cygwin Python. For this group removing the -mno-cygwin option would result in unusable extension modules. (the resulting .pyd requires cygwin.dll but is to be used in a non-cygwin Python). Firstly, note that users in group B must surely be a group that diminishes with time since they are using the legacy gcc 3.x cygwin compiler. Similarly, since neither of mingw or cygwin will ever bring back -mno-cygwin users in group A will only increase with time. (I am such a user and have been manually removing all reference to -mno-cygwin from distutils for 2 years now.) This means that the balance of breakage will only move towards group A over time. Secondly, any users in group B will suffer the same problem as users in group A if they try to use gcc 4.x. However this has not been reported on the tracker (I read through all matches in a search for '-mno-cygwin'). I think this serves as an indication of how many people are actually using this setup. Thirdly, the -mno-cygwin option is a now-abandoned, experimental feature of a legacy compiler. Its creators did not feel the need to give it a deprecation period and its use is discouraged by both mingw and cygwin. Bringing these points together: not removing -mno-cygwin from distutils trades the possible breakage for possibly non-existent users of the obscure, legacy, and generally considered broken setup B against the definite, known breakage for users of the appropriate documented setup A. I think this should be enough to say that the fix for the next Python version should simply remove all reference to '-mno-cygwin' as I have been doing for 2 years now without problem. The only users who can be adversely affected by this are those in group B who decide to upgrade to non-cygwin Python 3.4 (while still using an ancient cygwin gcc to build extensions). The suggested fix can be either to use mingw or to setup the cross-compilers in their cygwin installation. For Python 2.7, 3.2 and 3.3 I think that this should be considered a bug that can be fixed in a bugfix release. However in that case it may be considered inappropriate to risk the small possibility of users in group B experiencing breakage. Since such users must be using cygwin's gcc 3.x, I propose that distutils check the gcc version and only add '-mno-cygwin' if the major version is 3. This will not adversely affect users in group B and will fix the problem for users in group A. Users in group B who attempt to use gcc 4.x will find that they get a different error message (at import time instead of build time) but that their setup will still be just as broken as it was before this change. Thanks, Oscar

msg189756 - (view) Author: Paul Moore (paul.moore) * Date: 2013-05-21 14:04

+1 for Oscar's proposed fix. It sounds like a sensible approach.

msg189769 - (view) Author: Pete Forman (Pete.Forman) Date: 2013-05-21 16:18

Another +1 for Oscar. I've just done an install of Python 2.7.5 and had to hack cygwinccompiler.py again. I'm using mingw with gcc 4.6.2 on Windows 7.

msg189770 - (view) Author: Martin v. Löwis (loewis) * Date: 2013-05-21 16:21

Oscar: I agree with your analysis, but it is incomplete. There is a group C: Users who have only cygwin gcc 4.x installed For those, the current setup will produce an error message, essentially telling them that the need to fix something (specifically: edit distutils, install mingw). With the proposed change, --compiler=mingw32 will produce a binary, but the binary will incorrectly depend on cygwin. They may not notice on their local system (since cygwin.dll is available), but only on customer systems. That said: which of Roumen's patches (if any) would you recommend for inclusion?

msg189786 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-05-21 21:14

On 21 May 2013 17:21, Martin v. Löwis <report@bugs.python.org> wrote: > > C: Users who have only cygwin gcc 4.x installed > > For those, the current setup will produce an error message, essentially telling them that the need to fix something (specifically: edit distutils, install mingw). With the proposed change, --compiler=mingw32 will produce a binary, but the binary will incorrectly depend on cygwin. They may not notice on their local system (since cygwin.dll is available), but only on customer systems. Well there cannot be anyone in group C who currently has a functioning setup. But I agree that it's better to have a good error message. It may be possible to check in some way that the gcc used is from cygwin and add an error message specifically for this case. I'll have a look at this when I'm next on Windows. More generally I think that compiling non-cygwin extensions with cygwin gcc should be altogether deprecated (for Python 3.4 at least). It should be discouraged in the docs and unsupported in the future. It can only work with -mno-cygwin which in turn only works with gcc 3.x, has never documented as being a stable gcc feature, is now abandoned and is referred to disparagingly on both the mingw and cygwin mailing lists: To quote Dave Korn from cygwin http://cygwin.com/ml/cygwin/2009-03/msg00802.html ''' gcc-3 -mno-cygwin still works just as well (or badly!) as it ever has done, and will be retained for ever. gcc-4 series releases will not support it at all. As the whole thing is (still) experimental and explicitly warned to be unstable I don't see the need to go for a deprecation period. ''' Or Earnie Boyd from mingw-users http://permalink.gmane.org/gmane.comp.gnu.mingw.user/42111 ''' On Mon, May 20, 2013 at 9:13 AM, Paul Moore wrote: > So building an extension using --compiler=mingw in Python could pick up a > cygwin gcc if that was on PATH, and this will work as long as -mno-cygwin is > passed on the command line. But it won't work (it will build a DLL with a > dependency on the cygwin DLL) if -mno-cygwin is omitted. I'd argue that > people should just install and use mingw rather than cygwin, but that may > not be what everyone does in practice. No!!! The -mno-cygwin abomination is dead. If you want to build a native Python using Cygwin you would do it the cross compiler way and state the --host you're configuring for. Python's distutil needs to remove the -mno-cygwin option. ''' However no-cygwin mode is currently a documented feature: http://docs.python.org/3.4/install/index.html#gnu-c-cygwin-mingw So it can't simply be deprecated in already released Pythons but I do want to fix the mingw bug there if possible. The suggestion to make -mno-cygwin conditional on gcc major version may lead to some users who attempt to use a setup that did not previously work not seeing the appropriate error message. However it does, I believe, come with the following two guarantees: 1) Mingw setups that are used, wanted and currently broken will be fixed. 2) No currently functional setups will be broken. That may be the best that is possible given the tight constraints on changes to distutils. > That said: which of Roumen's patches (if any) would you recommend for inclusion? None. I may have misread them but my impression is that they are not particularly intended to be used as individual patches. I can't see one that just makes the relevant changes and collectively they make up a more pervasive change than I was proposing. The patch that I was proposing for 3.4 would simply remove -mno-cygwin on these 5 lines: http://hg.python.org/cpython/file/7fce9186accb/Lib/distutils/cygwinccompiler.py#l322 For 2.7, 3.2 and 3.3 I would do the same but conditional on self.gcc_version. I think Roumen has identified many different issues but I would try and keep it focussed on the one -mno-cygwin issue. Oscar

msg189805 - (view) Author: Martin v. Löwis (loewis) * Date: 2013-05-22 11:43

Am 21.05.13 23:14, schrieb Oscar Benjamin: > More generally I think that compiling non-cygwin extensions with > cygwin gcc should be altogether deprecated (for Python 3.4 at least). > It should be discouraged in the docs and unsupported in the future. I agree with that, although I find it sad that the Cygwin project apparently abandoned support for building Mingw binaries. > It can only work with -mno-cygwin This is factually incorrect. It also works with the i686-pc-mingw32-gcc executable, which (IIUC) is still available for Cygwin.

msg189813 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-05-22 12:40

On 22 May 2013 12:43, Martin v. Löwis <report@bugs.python.org> wrote: > Am 21.05.13 23:14, schrieb Oscar Benjamin: >> More generally I think that compiling non-cygwin extensions with >> cygwin gcc should be altogether deprecated (for Python 3.4 at least). >> It should be discouraged in the docs and unsupported in the future. > > I agree with that, Excellent. > although I find it sad that the Cygwin project > apparently abandoned support for building Mingw binaries. I don't understand their reasoning but given the scorn poured on to -mno-cygwin from at least some people I trust that they had some good reason :) Also they have replaced it with something that they consider more appropriate (the cross-compilers). >> It can only work with -mno-cygwin > > This is factually incorrect. It also works with the i686-pc-mingw32-gcc > executable, which (IIUC) is still available for Cygwin. I should have been slightly clearer. It can only currently work in distutils with -mno-cygwin. The executable you refer to is part of cygwin gcc's cross-compiler toolchain. This is their recommended replacement for -mno-cygwin (if not mingw) but is AFAICT unsupported by distutils. I think there's a case for saying that distutils should support these but it should only be done with a new UnixCCompiler subclass and a new --compiler entry point. It should also perhaps provide a way to specify the --host since I think that facility is part of the purpose of the new toolchain. In any case cygwin cross-compiler support should not be conflated in the codebase with distutils' mingw support and if it is to be added that should be discussed in a separate issue. I personally don't think I would use it and would not push for the support to be added. Going back to the group C users: I think that it should be possible to create an is_cygwingcc() function that would parse the output of 'gcc --version'. Then Mingw32CCompiler.__init__ could do: if is_cygwingcc() and self.gcc_version >= '4': raise RuntimeError('No cygwin mode only works with gcc-3. Use gcc-3 or mingw') The is_cygwingcc() function can be conservative since false positives or more of a problem than false negatives. I think this should address your concern. However on further reflection I'm a little reluctant to force an error if I can't *prove* that the setup is broken. I'm no stranger to monkey-patching distutils and it's possible that someone has already monkey-patched it to make some bizarre setup just about work. I would be a little peeved if my setup broke in a bugfix release simply because someone else who didn't understand it decided that it wasn't viable. (The same monkey-patching concerns apply to the other changes but I think that fixing the non-monkey-patched setup for mingw trumps in that case.) So perhaps the best place to deal with the gcc-4/no-cygwin issue is in the distutils docs. My updated proposal is (I'll write patches if this is acceptable): Python 3.4: Remove '-mno-cygwin'. This breaks the no-cygwin mode and fixes the mingw mode. The distutils docs are updated with something like: ''' Note: Previous Python versions supported another 'no-cygwin' mode that could use cygwin gcc to build extensions without a dependency on cygwin.dll. This is no longer supported. New in Python 3.4: No-cygwin mode is no longer supported. ''' Python 2.7, 3.2 and 3.3: Only use '-mno-cygwin' if self.gcc_version < '4'. This should not break any currently functioning setups (barring serious monkey-patching). The distutils docs are updated with something like: ''' Note: The no-cygwin mode only works with cygwin's gcc-3. For gcc-4 it may produce .pyd files with dependencies on cygwin.dll that are not fully redistributable. The use of no-cygwin mode is deprecated by cygwin and support for it is removed in Python 3.4. ''' If you would rather have the is_cygwingcc() check I'm happy to put that in also if it gets this issue moving but I'm personally cautious about it. Thanks, Oscar

msg189814 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-05-22 13:23

On 22 May 2013 13:40, Oscar Benjamin <report@bugs.python.org> wrote: > > However on further reflection I'm a little reluctant to force an error > if I can't *prove* that the setup is broken. After a little more reflection I realise that we could just do: if self.gcc_version < '4' or is_cygwingcc(): # use -mno-cygwin This way the cygwin/gcc-4 error is still emitted only if gcc emits it. If the is_cygwingcc() function is conservative then there could be cases where it mistakenly does not use -mno-cygwin but that would have to be a broken cygwin/gcc-4 setup anyway.

msg189871 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-05-23 17:57

I have written a function that can be used to determine if the gcc that distutils will use is from Cygwin or MinGW: def is_cygwingcc(): '''Try to determine if the gcc that would be used is from cygwin.''' out = Popen(['gcc', '-dumpmachine'], shell=True, stdout=PIPE).stdout try: out_string = out.read() finally: out.close() # out_string is the target triplet cpu-vendor-os # Cygwin's gcc sets the os to 'cygwin' return out_string.strip().endswith('cygwin') The idea is that 'gcc -dumpmachine' emits a string that always ends in 'cygwin' for the Cygwin gcc (please let me know if I'm wrong about that). Earnie Boyd at mingw-users described this method for distinguishing MinGW and Cygwin gcc as not being a bad idea: http://permalink.gmane.org/gmane.comp.gnu.mingw.user/42137 With this the Mingw32CCompiler.__init__ method can be modified to do: if self.gcc_version < '4' or is_cygwingcc(): no_cygwin = ' -mno-cygwin' else: no_cygwin = '' self.set_executables(compiler='gcc%s -O -Wall' % no_cygwin, compiler_so='gcc%s -mdll -O -Wall' % no_cygwin, compiler_cxx='g++%s -O -Wall' % no_cygwin, linker_exe='gcc%s' % no_cygwin, linker_so='%s%s %s %s' % (self.linker_dll, no_cygwin, shared_option, entry_point)) This will fix the problem for MinGW, should not break existing no-cygwin/gcc 3.x setups and preserves the error message currently seen for no-cygwin with gcc 4.x. In other words it should satisfy users in all three groups A, B and C referred to above. In particular the is_cygwingcc() function hopefully addresses Martin's concern for users in group C. Is this approach acceptable? Thanks, Oscar

msg189874 - (view) Author: Roumen Petrov (rpetrov) * Date: 2013-05-23 19:11

Oscar Benjamin wrote: > Oscar Benjamin added the comment: > > I have written a function that can be used to determine if the gcc > that distutils will use is from Cygwin or MinGW: > > def is_cygwingcc(): > '''Try to determine if the gcc that would be used is from cygwin.''' > out = Popen(['gcc', '-dumpmachine'], shell=True, stdout=PIPE).stdout > try: > out_string = out.read() > finally: > out.close() > # out_string is the target triplet cpu-vendor-os > # Cygwin's gcc sets the os to 'cygwin' > return out_string.strip().endswith('cygwin') > > The idea is that 'gcc -dumpmachine' emits a string that always ends in > 'cygwin' for the Cygwin gcc (please let me know if I'm wrong about > that). Earnie Boyd at mingw-users described this method for > distinguishing MinGW and Cygwin gcc as not being a bad idea: > http://permalink.gmane.org/gmane.comp.gnu.mingw.user/42137 > > With this the Mingw32CCompiler.__init__ method can be modified to do: > > if self.gcc_version < '4' or is_cygwingcc(): It seems to me you try to find another method to detect support of some options. Where is written that compiler is gcc ? Yes this is current distutils code but please review my set of patches > no_cygwin = ' -mno-cygwin' > else: > no_cygwin = '' > > self.set_executables(compiler='gcc%s -O -Wall' % no_cygwin, > compiler_so='gcc%s -mdll -O -Wall' % no_cygwin, > compiler_cxx='g++%s -O -Wall' % no_cygwin, > linker_exe='gcc%s' % no_cygwin, > linker_so='%s%s %s %s' > % (self.linker_dll, no_cygwin, > shared_option, entry_point)) This will not work in new cygwin (1.7) environment with true cross-compilers. Reason is simple - executable is not gcc. > This will fix the problem for MinGW, should not break existing > no-cygwin/gcc 3.x setups and preserves the error message currently > seen for no-cygwin with gcc 4.x. In other words it should satisfy > users in all three groups A, B and C referred to above. In particular > the is_cygwingcc() function hopefully addresses Martin's concern for > users in group C. > > Is this approach acceptable? It is not enough. > Thanks, > Oscar > > ---------- > Roumen

msg189878 - (view) Author: Paul Moore (paul.moore) * Date: 2013-05-23 21:06

On 23 May 2013 20:11, Roumen Petrov <report@bugs.python.org> wrote: > > Is this approach acceptable? > It is not enough. > Support for compilers other than gcc (including cross-compilers) is a separate issue, and one which is much less likely to get accepted.

msg189887 - (view) Author: Renato Silva (renatosilva) Date: 2013-05-24 00:27

I must note that GCC 4.x *does* support -mno-cygwin, at least until 4.4, and at least the MinGW version. I have used it myself for building Pidgin under Windows, which requires that option. See [1] where a Pidgin developer confirms that. [1] http://pidgin.im/pipermail/support/2011-December/011159.html

msg189906 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-05-24 10:52

> Renato Silva added the comment: > > I must note that GCC 4.x *does* support -mno-cygwin, at least until 4.4, and at least the MinGW version. MinGW has never "supported" the -mno-cygwin option. It has simply tolerated it. The option never did anything useful and at some point it became an error to even supply it. I'm not sure exactly when but some time after 4.4 sounds reasonable to me. The option was only ever meaningful in cygwin's gcc 3.x and was always an error in 4.x. > I have used it myself for building Pidgin under Windows, which requires that option. See [1] where a Pidgin developer confirms that. > No the developer does not confirm that the -mno-cygin option is required for MinGW. Also from what I've seen I would say that the error message that the OP shows there comes from Cygwin's gcc not MinGW.

msg189941 - (view) Author: Renato Silva (renatosilva) Date: 2013-05-25 03:43

> MinGW has never "supported" the -mno-cygwin option. It has simply > tolerated it. The option never did anything useful and at some point > it became an error to even supply it. I'm not sure exactly when but > some time after 4.4 sounds reasonable to me. Hi Oscar! Sorry, I just meant to correct this information: "in gcc 4.x it produces an error preventing build". Even if it doesn't do anything useful, still GCC 4.4 does accept that option normally. If MinGW didn't touch anything relevant, then what Cygwin folks said about 4.x [1] clearly did not come to reality. > No the developer does not confirm that the -mno-cygin option is > required for MinGW. Not for MinGW, but for building Pidgin. I have just checked it, and -mno-cygwin actually is no longer necessary since 2.10.7 [1], but it was at the time of that message. Even though it didn't do anything meaningful, a GCC like 4.6 would cause build to fail. > Also from what I've seen I would say that the error message that > the OP shows there comes from Cygwin's gcc not MinGW. No, you can use either Cygwin or MinGW MSYS as environment, but the compiler must be MinGW [2]. [1] https://hg.pidgin.im/pidgin/main/rev/c959cde2a7bd [2] https://developer.pidgin.im/wiki/BuildingWinPidgin#Setupyourbuildenvironment

msg189943 - (view) Author: Renato Silva (renatosilva) Date: 2013-05-25 03:50

ERRATA. Where you see: "[1] clearly did not come to reality." Please read: "[0] clearly did not come to reality." [0] http://cygwin.com/ml/cygwin/2009-03/msg00802.html

msg189946 - (view) Author: Roumen Petrov (rpetrov) * Date: 2013-05-25 08:53

Oscar Benjamin wrote: > [SNIP]The option was only ever meaningful in cygwin's gcc 3.x and was > always an error in 4.x. May be . It seems to me flag was removed in GCC 4.5 .

Since Renata's ERRATA was unclear to whether or not this was *actually* removed (please point to a changeset if the option was removed): If the option is no longer required by Pidgin and that was the original reason to have it in the first, there's really no reason *not* to remove this, no? Eg this code > int main() { return 0; } Fails to compile on MingW gcc 4.6 and 4.7 when passing -mno-cygwin to gcc. The solution that distutils does not work with current versions of MingW (and GCC) for good is unacceptable so the only working solution is removal of no-cygwin. Just that GCC 4.4 would support the flag does not change this bug one way or the other.

ERRATA Misunderstood the meaning of ERRATA, please ignore my last post. Roumen Petrov wrote: > Where is written that compiler is gcc ? Yes this is current distutils > code but please review my set of patches I kinda like the using --target-help for finding out if -mno-cygwin is supported assuming --target-help is supported by all GCC's. I disliked the version-based approach since back from 2011 since it seems fragile to me. I'm noo sure if it falls into the category of lightweight checks Eric asked for in #msg146499 but at least more lightweight than my proposal of trying to compile a sample program with the compiler.

msg189958 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-05-25 12:54

On 25 May 2013 04:43, Renato Silva <report@bugs.python.org> wrote: > > Renato Silva added the comment: > > Hi Oscar! Sorry, I just meant to correct this information: "in gcc 4.x it produces an error preventing build". Even if it doesn't do anything useful, still GCC 4.4 does accept that option normally. If MinGW didn't touch anything relevant, then what Cygwin folks said about 4.x [1] clearly did not come to reality. In context it should be clear that the statement "in gcc 4.x it produces an error preventing build" refers to Cygwin's gcc and not MinGW's. Which gcc are you referring to? >> No the developer does not confirm that the -mno-cygin option is >> required for MinGW. > > Not for MinGW, but for building Pidgin. I have just checked it, and -mno-cygwin actually is no longer necessary since 2.10.7 [1], but it was at the time of that message. Even though it didn't do anything meaningful, a GCC like 4.6 would cause build to fail. Yes gcc 4.6 would fail because it won't accept the -mno-cygwin option. That does not mean that any other MinGW gcc ever *required* the -mno-cygwin option for anything. The MinGW devs have repeatedly and explicitly stated that the -mno-cygwin option never did anything useful when used with MinGW: http://permalink.gmane.org/gmane.comp.gnu.mingw.user/42097 http://permalink.gmane.org/gmane.comp.gnu.mingw.user/42101 http://permalink.gmane.org/gmane.comp.gnu.mingw.user/42104 >> Also from what I've seen I would say that the error message that >> the OP shows there comes from Cygwin's gcc not MinGW. > > No, you can use either Cygwin or MinGW MSYS as environment, but the compiler must be MinGW [2]. Yes but that particular error message is coming from Cygwin's gcc not MinGW. As stated by the Pidgin dev in that message the OP does not know which compiler they are using: http://pidgin.im/pipermail/support/2011-December/011159.html

msg189962 - (view) Author: Roumen Petrov (rpetrov) * Date: 2013-05-25 14:30

Oscar, 10x for info I know how to find information for this particular case . So you last post just confrim what I wrote before two years ( 2011-08-03 http://bugs.python.org/issue12641#msg141614 ) Go ahead and just remove flag. Roumen

msg190060 - (view) Author: Renato Silva (renatosilva) Date: 2013-05-26 04:06

> In context it should be clear that the statement "in gcc 4.x it > produces an error preventing build" refers to Cygwin's gcc and not > MinGW's. Which gcc are you referring to? If it refers to Cygwin only, sorry then. However, didn't folks said in MinGW thread that they didn't touch anything about such flag there? If so, then how can it have been removed later in 4.5 or 4.6 instead of 4.0 like in Cygwin? > Yes gcc 4.6 would fail because it won't accept the -mno-cygwin option. > That does not mean that any other MinGW gcc ever *required* the > -mno-cygwin option for anything. Again, I was not saying it was required *for MinGW*, but *for building Pidgin*. Note that I'm not saying either that their use of such option was ever meaningful in the build process (in fact, they have removed such flag while still using 4.4, indicating that it was really useless). > Yes but that particular error message is coming from Cygwin's gcc not > MinGW. As stated by the Pidgin dev in that message the OP does not > know which compiler they are using: > http://pidgin.im/pipermail/support/2011-December/011159.html We actually cannot confirm whether the GCC was from Cygwin, MSYS or Pidgin build box (the correct), and which version. It could be from Cygwin, but still my point stands, that the Pidgin developer does confirm that GCC 4.4 from MinGW *does* accept that flag.

msg191714 - (view) Author: Nicolas Frattaroli (fratti) Date: 2013-06-23 18:09

It's cool that you guys are discussing semantics of who said what and how, but that still doesn't fix this very simple issue that breaks compiling for everyone on Windows who uses MinGW. -mno-cygwin, was, as far as I know, only ever required to build from cygwin as host to mingw as target. It is very unlikely that anyone who'd upgrade python still needs this option and is running a gcc cygwin 3.x version. And if it breaks for them it's still better than the current state, which is "broken for everyone else". Sorry if this sounds snarky, but the fact that this issue has been open for almost 2 years is quite ridiculous.

msg191735 - (view) Author: Jeffrey Armstrong (Jeffrey.Armstrong) * Date: 2013-06-23 22:00

> ...the fact that this issue has been open for almost 2 years is quite ridiculous. I thought that I'd add a little statistic for everyone that might put this bug into perspective. Since this bug was opened, the MinGW installer has been downloaded about 32,000,000 (32 million) times per sf.net: http://sourceforge.net/projects/mingw/files/Installer/stats/timeline?dates=2011-07-26+to+2013-06-23 If we take a naive approach, that's 32 million compiler installations that can't build Python extensions without manually modifying distutils first. That statistic doesn't include the multitude of people installing other builds of GCC for Windows (including MinGW-W64, a whole other unsupported version of GCC, but that's a different bug).

On 24.06.2013 00:00, Jeffrey Armstrong wrote: > > Jeffrey Armstrong added the comment: > >> ...the fact that this issue has been open for almost 2 years is quite ridiculous. > > I thought that I'd add a little statistic for everyone that might put this bug into perspective. Since this bug was opened, the MinGW installer has been downloaded about 32,000,000 (32 million) times per sf.net: > > http://sourceforge.net/projects/mingw/files/Installer/stats/timeline?dates=2011-07-26+to+2013-06-23 > > If we take a naive approach, that's 32 million compiler installations that can't build Python extensions without manually modifying distutils first. > > That statistic doesn't include the multitude of people installing other builds of GCC for Windows (including MinGW-W64, a whole other unsupported version of GCC, but that's a different bug). Could someone perhaps produce a single final patch file which can be applied to Python 2.7 and 3.2+ ? It is not clear at the moment which of all those patches on the ticket should be applied.

msg191753 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-06-24 11:53

On 24 June 2013 09:07, Marc-Andre Lemburg <report@bugs.python.org> wrote: > > Could someone perhaps produce a single final patch file which can > be applied to Python 2.7 and 3.2+ ? I've attached two patches "check_mno_cywin_py27.patch" for Python 2.7 and "check_mno_cywin_py3.patch" for Python 3.2 and 3.3. The changes are identical but the 2.7 patch didn't apply cleanly against 3.x. I'll upload the files used to test the patches in "test_mno_cygwin.tar.gz". The patches are as I described previously and check the output of 'gcc -dumpmachine' to see if the gcc on PATH is from cygwin. With the patch '-mno-cygwin' will be passed if gcc version < 4 or the gcc is from cygwin. Otherwise it will not be passed. I've tested with versions: Python 2.7.5, 3.2.5 and 3.3.2 MinGW gcc 4.7.2 Cygwin gcc 3.4.4 and 4.5.3 The results of the patch are the same for all versions of Python tested: Cygwin gcc 3.x - still works Cygwin gcc 4.x - still doesn't work (same error message) MinGW gcc 4.7 - fixed after the patch This patch does not attempt to add support for the newer (gcc 4.x) Cygwin cross-compilers. I have experimented with what it would take to have those work and it is something like: if is_cygwingcc() and version >= 4: platform = platform_map[get_platform()] use platform + '-pc-cygwin-gcc' as gcc use platform + '-pc-cygwin-g++' as g++ etc. Then there would also need to modifications to the linker settings to fix the problem that Martin mentioned (a long way above) that it would link against the wrong MSVC runtime. I started writing the patch to do these things as well as fix MinGW support and it became more and more of a mess. I don't think that distutils should be trying to guess whether or not people intended to use the Cygwin cross-compilers. If these are to be supported then they should have a new --compiler=cygwin-cross and a separate subclass of CygwinCCompiler to avoid more issues like this one arising in the future. Oscar

msg191755 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-06-24 12:15

On 24 June 2013 12:53, Oscar Benjamin <report@bugs.python.org> wrote: > The changes > are identical but the 2.7 patch didn't apply cleanly against 3.x. I'll > upload the files used to test the patches in "test_mno_cygwin.tar.gz". Correction: the patches are not quite identical as the py3 patch decodes the output of the subprocess as ascii.

msg191845 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-06-25 10:47

I'm attaching one more patch "check_mno_cywin_py34.patch". This is my preferred patch for Python 3.4 (default). It fixes building with MinGW and removes all support for using Cygwin gcc with --compiler=mingw32. The user would see the following error message: ''' Q:\current\testing\hello>testbuild q:\tools\cygwin\bin -3.3 running build_ext error: Cygwin gcc cannot be used with --compiler=mingw32 ''' I think that this is reasonable as '-mno-cygwin' is a previously experimental and now long deprecated, discouraged and discontinued feature of Cygwin's gcc. Removing support for it in future Pythons would make problems involving MinGW build (like this one) much easier to solve in future: there would be no need to consider anything other than the behaviour of MinGW's gcc.

msg192758 - (view) Author: Christian Heimes (christian.heimes) * Date: 2013-07-09 15:25

The is_cygwingcc() function can be simplified a lot with subprocess.check_output().

msg192763 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-07-09 16:31

On 9 July 2013 16:25, Christian Heimes <report@bugs.python.org> wrote: > > The is_cygwingcc() function can be simplified a lot with subprocess.check_output(). My initial thought was to do that but then I based it on _find_exe_version which for whatever reason uses Popen directly [1]. I'm happy to make that change and retest the patches although I can't do it right now. Can someone first accept or reject the general idea of the patches though? I'm happy to answer any questions about them but it takes time to get the diffs right and test against all compilers and Python versions and I don't really want to do it if the patches will just be rejected. Also I may soon lose access to the machine that I used to write and test these patches. If it is desired for me to change and retest them it may not be possible after two weeks or so. [1] http://hg.python.org/cpython/file/3f3cbfd52f94/Lib/distutils/cygwinccompiler.py#l368

msg192764 - (view) Author: Éric Araujo (eric.araujo) * Date: 2013-07-09 16:36

Don’t forget that distutils is used during CPython’s build process to compile extension modules: subprocess may not be importable then.

msg192766 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-07-09 16:43

On 9 July 2013 17:36, Éric Araujo <report@bugs.python.org> wrote: > > Don’t forget that distutils is used during CPython’s build process to compile extension modules: subprocess may not be importable then. Subprocess is imported at at the top of the module in 3.x [1]. The whole distutils.cygwinccompiler module is an ImportError if subprocess is not importable. Or did you mean for 2.7 only (where get_versions() uses os.popen)? [1] http://hg.python.org/cpython/file/3f3cbfd52f94/Lib/distutils/cygwinccompiler.py#l51

msg192877 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-07-11 16:21

I'm attaching three new patches following on from Eric and Christian's suggestions: check_mno_cywin_py27_1.patch (for Python 2.7) check_mno_cywin_py3_1.patch (for Python 3.2 and 3.3) check_mno_cywin_py34_1.patch (for Python 3.4) The py27 patch now uses os.popen to avoid importing subprocess as suggested by Eric. The other two patches are changed to use check_output as suggested by Christian (subprocess is already imported in 3.x). I've retested the patches using the same setup as before and the results are unchanged for all gcc and Python versions tested.

msg195576 - (view) Author: Geert Jansen (geertj) * Date: 2013-08-18 19:28

*bump*. This is a critical bugfix that prevents I bet 90%+ of Python users on Windows compiling C extensions. It has been open for 2 years and it's a great disservice to people having to compile stuff on Windows. Oscar has been doing a terrific job of providing man patches. May I ask that a core dev finally takes some responsibility here, signs of on the patch, and get it applied?

msg195809 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-08-21 17:14

I just noticed today that the fix that implemented by these patches (only providing -mno-cygwin if gcc_ver < 4) is also used by numpy's distutils. You can see the relevant code here: https://github.com/numpy/numpy/blob/master/numpy/distutils/mingw32ccompiler.py#L117 The relevant commit was three years ago: https://github.com/numpy/numpy/commit/9dd7c7b8ad826beefbbc0c10ff457c62f1be223d They haven't bothered with checking for cygwin gcc (my patches only do this to try and show a helpful error message).

msg198650 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-09-29 21:02

Oscar, thanks for the patches. Two things: - in the 2.7 patch, could you explicitly close the popen() file object instead of relying on deallocation to do it? - have you signed a contributor's agreement? http://www.python.org/psf/contrib/

msg198690 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-09-30 11:08

Thanks for looking at this Antoine. I've attached an updated patch for Python 2.7 called check_mno_cywin_py27_2.patch. This explicitly closes the popen object in the same way as the get_versions() function immediately above. I've just signed an electronic contributor's agreement.

msg198691 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-09-30 11:13

On 30 September 2013 12:08, Oscar Benjamin <report@bugs.python.org> wrote: > I've attached an updated patch for Python 2.7 called > check_mno_cywin_py27_2.patch. To be clear: I retested this patch (using the setup described above) and the results are unchanged.

msg198732 - (view) Author: Roundup Robot (python-dev) Date: 2013-09-30 20:25

New changeset 7d9a1aa8d95e by Antoine Pitrou in branch '2.7': Issue #12641: Avoid passing "-mno-cygwin" to the mingw32 compiler, except when necessary. http://hg.python.org/cpython/rev/7d9a1aa8d95e

msg198733 - (view) Author: Roundup Robot (python-dev) Date: 2013-09-30 20:29

New changeset 6b89176f1be5 by Antoine Pitrou in branch '3.3': Issue #12641: Avoid passing "-mno-cygwin" to the mingw32 compiler, except when necessary. http://hg.python.org/cpython/rev/6b89176f1be5 New changeset 8e180b2067e4 by Antoine Pitrou in branch 'default': Issue #12641: Avoid passing "-mno-cygwin" to the mingw32 compiler, except when necessary. http://hg.python.org/cpython/rev/8e180b2067e4

msg198734 - (view) Author: Antoine Pitrou (pitrou) * Date: 2013-09-30 20:30

Thank you Oscar! This issue can endly be fixed.

msg198763 - (view) Author: Oscar Benjamin (oscarbenjamin) * Date: 2013-10-01 10:21

Thanks Antoine!

msg207498 - (view) Author: Lewis Levin (lewisl) Date: 2014-01-07 00:56

This is broken for Python 2.7.4. Is it fixed in 2.7.6? gcc deprecated -mno-cygwin. It's not there any more. There shouldn't be a discussion. This is an incredibly poor example of how fragmentation and poor process result in poor quality open source software--in the corners, even when the overall code is excellent. Over 2 years on something this trivial. Hundreds of lines of discussion. There wasn't really any reason for this to have persisted.

msg207499 - (view) Author: Antoine Pitrou (pitrou) * Date: 2014-01-07 00:59

> This is broken for Python 2.7.4. Is it fixed in 2.7.6? The fix is in 2.7.6, yes.

msg238062 - (view) Author: Michael Clerx (Michael.Clerx) Date: 2015-03-14 00:02

I'm seeing this bug in 2.7.9. The reason seems to be that the version detection doesn't work... This snippet: out = os.popen(gcc_exe + ' -dumpversion', 'r') out_string = out.read() returns an empty out_string, causing gcc_version = None < '4' Maybe the < '4' check could be restructured to see None as "probably modern" instead of "probably very out of date" ?

msg238067 - (view) Author: Ned Deily (ned.deily) * Date: 2015-03-14 01:46