Le jeu. 23 janv. 2020 à 20:45, Brett Cannon a écrit :

Two pieces of feedback on this. One, nose is a bad example because that project has been telling people for years to switch to nose2 so the fact that people have still not switched something that should mostly be a direct swap after years of being asked to does not motivate in wanting to postpone one more year for that project's benefit. ;) (And where did you get the download stats on libraries.io? https://libraries.io/pypi/nose doesn't show any download counts.)

Oh right, download stats can be found at: https://pypistats.org/packages/nose Downloads last day: 150,272 Downloads last week: 824,557 Downloads last month: 3,077,860 150k downloads yesterday.

Two, you buried the list of things you would like to revert and postpone to Python 3.10. :)

I'm not sure of the meaning of "buried" here. What do you mean? We propose to revert 5 changes: * Removed tostring/fromstring methods in array.array and base64 modules * Removed collections aliases to ABC classes * Removed fractions.gcd() function (which is similar to math.gcd()) * Remove "U" mode of open(): having to use io.open() just for Python 2 makes the code uglier * Removed old plistlib API: 2.7 doesn't have the new API

Was this copied-and-pasted from somewhere, hence why you explain to us how to use `-X dev`?

Well, use whatever you want to see DeprecationWarning, as soon as you fix all of them in your project :-) PYTHONWARNINGS=default should be fine as well. IMO it's simpler to remind "-X dev" than using PYTHONWARNINGS=default. By the way, -X dev and PYTHONWARNINGS=default shows more warnings than just DeprecationWarning: they also show ImportWarning, PendingDeprecationWarning and ResourceWarning. -X dev helps to make your project ready for the next Python version.

Basically it isn't easy to discuss the things you want to revert here, especially since you grouped them all together. Are you planning to start separate conversations on the (I think) 5 things you want to change?

I'm not sure of what you mean. Would you prefer one mail thread per revert? It's not only about specific changes, but more a discussion about a general policy to decide if a deprecated feature should stay until 3.10, or if it's ok to remove it in 3.9. For example, another feature has been removed since we drafted this email: "The encoding parameter of json.loads() has been removed." If we follow the proposed policy, it should also be reverted. It seems like the incompatible change causing most issues is the collections change. We can try to produce statistics per change if you prefer. I understand that you disagree to revert changes, but you would prefer to decide on a case by case basis. Victor -- Night gathers, and now my watch begins. It shall not end until my death.

On 01/23/2020 03:36 PM, Barry Warsaw wrote:

On Jan 23, 2020, at 14:03, Victor Stinner wrote:

...

It's not only about specific changes, but more a discussion about a general policy to decide if a deprecated feature should stay until 3.10, or if it's ok to remove it in 3.9.

Given that we’ve changed the release cadence to one major release per year, it doesn’t seem that much of a burden to revert and carry these changes forward into Python 3.10. And if it helps with the migration off of Python 2.7, then +1 from me.

I suspect anyone who's waited this long to switch to Python 3 will be okay with staying on 3.8 for a few years. -- ~Ethan~

On 01/23/2020 07:02 PM, Robert Collins wrote:

On Fri, 24 Jan 2020 at 14:46, Ethan Furman mailto:ethan@stoneleaf.us> wrote:

On 01/23/2020 03:36 PM, Barry Warsaw wrote: > On Jan 23, 2020, at 14:03, Victor Stinner wrote:

>> It's not only about specific changes, but more a discussion about a >> general policy to decide if a deprecated feature should stay until >> 3.10, or if it's ok to remove it in 3.9. > > Given that we’ve changed the release cadence to one major release per year, it doesn’t seem that much of a burden to revert and carry these changes forward into Python 3.10.  And if it helps with the migration off of Python 2.7, then +1 from me.

I suspect anyone who's waited this long to switch to Python 3 will be okay with staying on 3.8 for a few years.

The issue isn't end users, it is projects who will then be incompatible with 3.10 for "a few years."

I'm sure you meant 3.9, since 3.10 would still have the removals. My understanding is that this postponement of removals is aimed at those who have just migrated to Python 3, not those who have already done it nor those who have 2/3 straddling code bases. For those who have been on 3 for a while, updating to use the newer APIs for 3.9 vs 3.10 shouldn't make a difference. Like-wise for those with 2/3 straddling code bases (we'll just need to add a few more things to our shims). Anyone who hasn't supported/used Python 3 until now shouldn't have a problem with sticking with 3.8 until they are ready to make more adjustments, and those who have had to keep Python 2 around to run those applications/frameworks/whatevers will, I should think, be thrilled to use any Python 3 instead. :-) -- ~Ethan~

Le ven. 24 janv. 2020 à 08:37, Miro Hrončok a écrit :

No, the motivation to pospone the changes to 3.10 are projects that alrady support both 2 and 3 at the same time, with or without compatibility libraries like six. Before they had anough time to make the necessary actions to abandon Python 2.7, we ask them to support 3.9 from the same code base.

Let me give you an example. The python-ipmi project works on Python 2.7-3.8, but it is not longer compatible with Python 3.9 because tostring/fromstring methods have been removed from array.array. IMO this change is a good example to illustrate the issue: https://github.com/kontron/python-ipmi/pull/73/files I chose to add py3_array_tobytes() and py3_array_frombytes() functions rather than adding "if _PY3: ... else: ...". Example: - self.minor = int(data[1:2].tostring().decode('bcd+')) + self.minor = int(py3_array_tobytes(data[1:2]).decode('bcd+')) This change reminds me when I added "Python 3" support to dozens of Python projects. You can see in this PR that Python 3.9 currently requires to modify 9 files in a small project. It is likely to be worse in a larger project. With Python 3.9, we cannot simply group all Python 3.x versions under "Python 3" anymore. Currently, a code compatible with "Python 3" means more likely compatible with "Python 3.5-3.8" ("Python 3.3-3.8" for the most conservative projects?). We kept a compatibility layer with Python 2 on purpose, PEP 4 says: "In order to facilitate writing code that works in both Python 2 & 3 simultaneously, any module that exists in both Python 3.5 and Python 2.7 will not be removed from the standard library until Python 2.7 is no longer supported as specified by PEP 373." The rule was used since Python 3.0 until Python 3.8, but it changed in Python 3.9 which includes many incompatible changes for the first time in the Python 3 major version. Victor

from collections import Sequence

With either:

try: from collections.abc import Sequence except ImprotError: # Python 2.7 doesn't have collections.abc from collections import Sequence

Or:

from compatlib.moves.collections_abc import Sequence

In both cases, we move the burden of having a compatibility layer of some sort from one central location (Python 3.9 stdlib) to dozens (hundreds?) locations.

...

For those who have been on 3 for a while, updating to use the newer APIs for 3.9 vs 3.10 shouldn't make a difference.

For Python 3 only projects? No, no difference.

...

Like-wise for those with 2/3 straddling code bases (we'll just need to add a few more things to our shims).

And our point is that because it's too early in 2020 to drop Python 2 code, it is better to encourage projects to update the code as Python 3 only in a year than to let them add more shims now just to hopefully remove them in couple months.

...

Anyone who hasn't supported/used Python 3 until now shouldn't have a problem with sticking with 3.8 until they are ready to make more adjustments, and those who have had to keep Python 2 around to run those applications/frameworks/whatevers will, I should think, be thrilled to use any Python 3 instead. :-)

This does not affect projects who don't support Python 3 yet.

-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TFCAMIAJ... Code of Conduct: http://python.org/psf/codeofconduct/

-- Night gathers, and now my watch begins. It shall not end until my death.

The change is that Python 2.7 is no longer supported (since 2020-01-01). Victor Le ven. 24 janv. 2020 à 09:19, Chris Angelico a écrit :

On Fri, Jan 24, 2020 at 7:05 PM Victor Stinner wrote:

...

We kept a compatibility layer with Python 2 on purpose, PEP 4 says:

"In order to facilitate writing code that works in both Python 2 & 3 simultaneously, any module that exists in both Python 3.5 and Python 2.7 will not be removed from the standard library until Python 2.7 is no longer supported as specified by PEP 373."

The rule was used since Python 3.0 until Python 3.8, but it changed in Python 3.9 which includes many incompatible changes for the first time in the Python 3 major version.

I'm sorry, I don't understand what 'changed'. Isn't that rule exactly WHY 3.9 is the removal point? Python 2.7 is no longer supported, and its final post-support release is scheduled earlier than 3.9's first beta and feature freeze. Doesn't that mean that PEP 4 is being followed precisely? What have I misunderstood?

ChrisA _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GCCLU2O2... Code of Conduct: http://python.org/psf/codeofconduct/

-- Night gathers, and now my watch begins. It shall not end until my death.

Adding Python 3.9 and dropping Python 2.7 support are two different things, but Python 3.9 somehow "enforces" to do both at the same time. The proposal is to give one year to project maintainers to drop Python 2.7 support, since Python 2.7 end of support just happened a few weeks ago (2020-01-01). Big projects have more resources to schedule dropping Python 2 support, coordinated with other projects: https://python3statement.org/ But for smaller projects with less resources, it's more complicated. Many very popular Python modules are maintained by a single person with limited resources. Even if we propose pull requests, it still takes time to review it, prepare a new release, etc. Victor Le ven. 24 janv. 2020 à 09:42, Chris Angelico a écrit :

On Fri, Jan 24, 2020 at 7:40 PM Victor Stinner wrote:

...

The change is that Python 2.7 is no longer supported (since 2020-01-01).

Which means that it's now okay to remove things, correct? Starting with 3.9, it's no longer necessary to maintain compatibility with 2.7?

ChrisA

-- Night gathers, and now my watch begins. It shall not end until my death.

On 1/24/2020 5:50 AM, Ivan Levkivskyi wrote:

On Fri, 24 Jan 2020 at 10:05, Victor Stinner mailto:vstinner@python.org> wrote:

The proposal is to give one year to project maintainers to drop Python 2.7 support, since Python 2.7 end of support just happened a few weeks ago (2020-01-01).

IMO creating this kind of "gray areas" in support and deprecation issues is bad. What this will create is just more sources for arguing/debates. Once deprecation or EoL schedule is set, it is best to align to it. Discussions about the schedules should happen when setting them, not when the deadline is coming.

Also I am not sure it is really worth it. For example, importing ABCs directly from collections was deprecated 8 years ago, what would 1 extra year change?

I think the concern is that with removing so many deprecated features, we're effectively telling libraries that if they want to support 3.9, they'll have stop supporting 2.7. And many library authors aren't willing to do that yet. Will they be willing to in another year? I can't say. Eric

On 1/24/2020 9:14 AM, Miro Hrončok wrote:

On 24. 01. 20 14:02, Eric V. Smith wrote:

...

I think the concern is that with removing so many deprecated features, we're effectively telling libraries that if they want to support 3.9, they'll have stop supporting 2.7. And many library authors aren't willing to do that yet. Will they be willing to in another year? I can't say.

The concern is not that they don't want to drop 2.7 support, but that is is a nontrivail task to actaually do and we cannot expect them to do it within the first couple weeks of 2020. While at the same time, we want them to support 3.9 since the early development versisons in order to eb able to detect regressions early in the dev cycle.

Ah. So in 3.8, they kept code that had deprecation warnings so that they could be compatible with 2.7. They'd like to now drop that code and be 3.9-only compatible, but they don't have enough time to do that because they couldn't start that work as long as they were supporting 2.7. Do I have that right? If so, I'd be okay with postponing the removal of the deprecated code until 3.10. But I don't think we should postpone it if the driver is so that libraries can remain 2.7 compatible. That could go on forever. This postponement would be a one-time thing. Eric

Le ven. 24 janv. 2020 à 19:35, Brett Cannon a écrit :

I'm also okay with a one-time delay in removals that are problematic for code trying to get off of Python 2.7 this year and might not quite cut it before 2021 hits. I'm sure some people will be caught off-guard once 3.9b1 comes out and they realize that they now have to start caring about deprecation warnings again. So I'm okay letting 3.9 be the release where we loudly declare, "deprecation warnings matter again! Keep your code warnings-free going forward as things will start being removed in 3.10".

Maybe we can put a strong statement at the What's New In Python 3.9 document? Victor -- Night gathers, and now my watch begins. It shall not end until my death.

On Fri, 24 Jan 2020 at 14:14, Miro Hrončok wrote:

On 24. 01. 20 14:02, Eric V. Smith wrote:

...

I think the concern is that with removing so many deprecated features, we're effectively telling libraries that if they want to support 3.9, they'll have stop supporting 2.7. And many library authors aren't willing to do that yet. Will they be willing to in another year? I can't say.

The concern is not that they don't want to drop 2.7 support, but that is is a nontrivail task to actaually do and we cannot expect them to do it within the first couple weeks of 2020. While at the same time, we want them to support 3.9 since the early development versisons in order to eb able to detect regressions early in the dev cycle.

But couldn't they have done that by adding the compatibility shims Victor described. Sure, they are messy, and sure, it would be more convenient not to have to. But the various dates have been known for some time now. Unless you're saying that the change to a yearly schedule for 3.9 has suddenly compressed timescales to an extent that projects can't cope with. So maybe there is a reason why people might legitimately have an issue here. Having said that, I think that far more people will see this as yet another delay before 2.7 dies, and treat it as one more reason to do nothing. So I still have reservations. I have already said that the 5 deprecations Victor described seem minor enough that one release's delay is not a big deal. If that's enough for the people hit by the shorter 3.9 cycle, then great. But it should *not* set a precedent that we're willing to repeatedly have this sort of discussion. It should be a one-off, and definitely not set the scene for a series of "and can this one be added as well" requests. I assume it's obvious that 3.10 would be a hard deadline, as well. There's no justification for deferring deprecations past then. Paul PS As a pip developer, I fully expect to be supporting Python 2.7 for a while yet. So I'm not talking from a perspective of someone who has happily dropped Python 2 support and doesn't understand the issues. But I don't expect *Python* to maintain my compatibility for me, and I don't see why others should, either. And yes, that does include these 5.

On Fri, 24 Jan 2020 at 08:54, Victor Stinner wrote:

The change is that Python 2.7 is no longer supported (since 2020-01-01).

However the assertion here seems to be that some people are unprepared for this (which seems to me like it's their problem, not ours). Features getting deprecated in newer Python versions is routine. Anyone maintaining Python 2.7 compatibility should already be painfully aware that we're deprecating things in 3.x and they have to maintain compatibity code for that. We've been minimising the impact for such people as much as we can for years. When *should* we stop doing so, if not at the point where we finally declared 2.7 unsupported? Having said that, the 5 specific changes noted seem relatively minor and deferring them an extra release doesn't seem like a massive deal. My objection is only if we try to extrapolate a general principle from those specific examples. Paul P.S.

I'm not sure of the meaning of "buried" here. What do you mean? We propose to revert 5 changes: [...] It's not only about specific changes, but more a discussion about a general policy to decide if a deprecated feature should stay until 3.10, or if it's ok to remove it in 3.9.

I agree with Brett, it was hard to understand your message. And I think the problem was that you tried to discuss both the *specific* proposal to delay 5 deprecations, and the *general* principle of when we stop preserving 2.7 compatibility layers. These two are very different questions and (as I say above) don't necessarily have the same answer. I think you should have posted *two* messages. The first very focused on the 5 deprecations, and the second, following on, saying something along the lines of "the previous message raises a broader question of how long we should carry 2.7 compatibility code before removing it - what do people think?". That would separate the general from the specific, while not flooding the list with separate threads for each deprecation (which I assume wasn't a serious suggestion on your part).

See pyfound.blogspot.com/2019/12/python-2-sunset.html for the details. January or April doesn't matter much for this thread, since Python 3.9.0 final is scheduled for October. Victor Le lun. 27 janv. 2020 à 23:13, Barry Scott a écrit :

...

On 24 Jan 2020, at 08:40, Victor Stinner wrote:

The change is that Python 2.7 is no longer supported (since 2020-01-01).

Isn't there going to be a final Python 2.7 release in April 2020?

I know that Twisted has moved its python 2 removal to match the April 2020 date. Other projects may have made the similar decisions.

Barry

...

Victor

Le ven. 24 janv. 2020 à 09:19, Chris Angelico a écrit :

...

On Fri, Jan 24, 2020 at 7:05 PM Victor Stinner wrote:

...

We kept a compatibility layer with Python 2 on purpose, PEP 4 says:

"In order to facilitate writing code that works in both Python 2 & 3 simultaneously, any module that exists in both Python 3.5 and Python 2.7 will not be removed from the standard library until Python 2.7 is no longer supported as specified by PEP 373."

The rule was used since Python 3.0 until Python 3.8, but it changed in Python 3.9 which includes many incompatible changes for the first time in the Python 3 major version.

I'm sorry, I don't understand what 'changed'. Isn't that rule exactly WHY 3.9 is the removal point? Python 2.7 is no longer supported, and its final post-support release is scheduled earlier than 3.9's first beta and feature freeze. Doesn't that mean that PEP 4 is being followed precisely? What have I misunderstood?

ChrisA _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GCCLU2O2... Code of Conduct: http://python.org/psf/codeofconduct/

-- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6YLZXTOJ... Code of Conduct: http://python.org/psf/codeofconduct/

-- Night gathers, and now my watch begins. It shall not end until my death.

On Jan 27, 2020, at 13:56, Barry Scott wrote:

Isn't there going to be a final Python 2.7 release in April 2020?

Yes, but I’ve been calling it a “celebratory release”. We’re gonna party at Pycon like it’s 2000. -Barry

Victor Stinner wrote:

Le jeu. 23 janv. 2020 à 20:45, Brett Cannon brett@python.org a écrit :

...

Two pieces of feedback on this. One, nose is a bad example because that project has been telling people for years to switch to nose2 so the fact that people have still not switched something that should mostly be a direct swap after years of being asked to does not motivate in wanting to postpone one more year for that project's benefit. ;) (And where did you get the download stats on libraries.io? https://libraries.io/pypi/nose doesn't show any download counts.) Oh right, download stats can be found at: https://pypistats.org/packages/nose Downloads last day: 150,272 Downloads last week: 824,557 Downloads last month: 3,077,860 150k downloads yesterday.

Ah, thanks! To put this in perspective, pytest had 631,008 downloads yesterday.

...

Two, you buried the list of things you would like to revert and postpone to Python 3.10. :) I'm not sure of the meaning of "buried" here. What do you mean?

Sorry, North American phrase. It means you put the important part way down towards the bottom. Another North American phrase that might apply is "burying the lead". Basically you put the interesting, important stuff so far down it was easy to overlook. :) My suggestion would have been to structure the email as "Fedora is seeing some transition problems for projects ATM. Reverting these 5 changes and postponing them to 3.10 would help a lot, and here is a bunch of detail about why we are making this proposal." IOW putting the list towards the top makes it easier to set expectations of what you're asking for while you try to justify the request, otherwise I could be reading most of that email as if you're suggesting we bring back the print statement or something. ;)

We propose to revert 5 changes:

Removed tostring/fromstring methods in array.array and base64 modules Removed collections aliases to ABC classes Removed fractions.gcd() function (which is similar to math.gcd()) Remove "U" mode of open(): having to use io.open() just for Python 2 makes the code uglier Removed old plistlib API: 2.7 doesn't have the new API

...

Was this copied-and-pasted from somewhere, hence why you explain to us how to use -X dev? Well, use whatever you want to see DeprecationWarning, as soon as you fix all of them in your project :-) PYTHONWARNINGS=default should be fine as well. IMO it's simpler to remind "-X dev" than using PYTHONWARNINGS=default. By the way, -X dev and PYTHONWARNINGS=default shows more warnings than just DeprecationWarning: they also show ImportWarning, PendingDeprecationWarning and ResourceWarning. -X dev helps to make your project ready for the next Python version. Basically it isn't easy to discuss the things you want to revert here, especially since you grouped them all together. Are you planning to start separate conversations on the (I think) 5 things you want to change? I'm not sure of what you mean. Would you prefer one mail thread per revert? It's not only about specific changes, but more a discussion about a general policy to decide if a deprecated feature should stay until 3.10, or if it's ok to remove it in 3.9. For example, another feature has been removed since we drafted this email: "The encoding parameter of json.loads() has been removed." If we follow the proposed policy, it should also be reverted. It seems like the incompatible change causing most issues is the

collections change. We can try to produce statistics per change if you prefer. I understand that you disagree to revert changes, but you would prefer to decide on a case by case basis. Victor Night gathers, and now my watch begins. It shall not end until my death.

On Fri, Jan 24, 2020 at 12:12 AM Serhiy Storchaka wrote:

I consider breaking unmaintained code is an additional benefit of removing deprecated features.

I'd like to warn against this attitude (even though in the past I've occasionally said such things). I now think core Python should not be so judgmental. We've broken enough code for a lifetime with the Python 2 transition. Let's be *much* more conservative when we remove things from Python 3. Deprecation is fine, and we should look for other ways to handle the problem of unmaintained code. But we should not rush language or stdlib changes for this purpose. -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-c...

On 2020-01-30 07:12, Dima Tisnek wrote:

On Fri, 24 Jan 2020 at 00:27, Victor Stinner wrote:

...

Miro and me consider that Python 3.9 is pushing too much pressure on projects maintainers to either abandon Python 2.7 right now...

Let's not conflate py2 EOL with what-should-go-into-py39.

...

https://copr.fedorainfracloud.org/coprs/g/python/python3.9/monitor/ - Has package failures. Some packages fail because of broken dependencies.

Please provide a better summary. I've looked at first 3 failed packages, and:

* `afflib` only has these error lines: ``` warning: extra tokens at the end of %endif directive in line 186: %endif # with_python2 warning: extra tokens at the end of %endif directive in line 195: %endif # with_python3 ``` I don't know if that's why the build is considered failed, but that issue seems to be on the build script side.

Those are just warnings; ignore them. afflib didn't build because it's missing a dependency (libtermcap-devel). It wasn't counted in the ones with Python-related problems.

* `python-django-post_office` doesn't have any stderr output or other indication why the build failed.

Same situation (this time missing python3-django-jsonfield).

* `ansible-lint` same, does this package even use Python?

And that one is missing python3-passlib, which fails several tests; one of them being ImportError: cannot import name 'MutableMapping' from 'collections'.

...

https://bugzilla.redhat.com/showdependencytree.cgi?id=PYTHON39 - Has open Python 3.9 bug reports for Fedora packages. Some problems

I feel it's still early days for py39 and well-maintained packages are expected to catch up.

...

I, for one, just started testing our crud against py39a3 yesterday, and yes some tools and libraries are broken, so I've opened issues and might post a PR, depending.

I get the gist from your email that you are (or fedora folks are) doing the same.

Definitely! That's exactly what we're doing: helping the well-maintained packages catch up, via bug reports and pull requests as capacity allows. And now we're reporting back that the most common change these packages have to make is the collections.abc removal, and that since many projects are planning to drop py2 compat later in 2020, the pull request we need to send is ugly -- and it could be much nicer one-liner in 2021 if the collections.abc removal was postponed.

So, what's the rationale to change py39 change set?

...

try: from collections.abc import Sequence except ImprotError: # Python 2.7 doesn't have collections.abc from collections import Sequence

I thought that collections compatibility was kept up to 3.8 specifically because py2 was alive. No that that requirement is gone, so should the shim, right? Ofc., my assumption could be just wrong...

...

While if we remove collections.Sequence in 3.10, they will face this decision early in 2021 and they will simply fix their code by adding ".abc" at the proper places, not needing any more compatibility layers. Of course, there will be projects that will still have declared Python 2 support in 2021, but it will arguably not be that many.

I see some semi-abandoned projects, as well as actively maintained projects not addressing DeprecationWarnings. I don't know if some projects just need a harder push (will break, not just deprecated). Either way, my gut tells me, if py39 was changed and became very compatible, we'd see the same discussion in 2021.

Not really. For projects that want to stay be py2/py3-compatible for the long term, the long "try/except ImportError" dance makes sense.

...

* Removed tostring/fromstring methods in array.array and base64 modules * Removed collections aliases to ABC classes * Removed fractions.gcd() function (which is similar to math.gcd()) * Remove "U" mode of open(): having to use io.open() just for Python 2 makes the code uglier * Removed old plistlib API: 2.7 doesn't have the new API

my 2c: all of these are truly minor.

I have a feeling that the discussion is just wrong altogether. It should not be "let's remove" or "let's keep". It should not be "let's keep x but not y".

It should be something along the lines of: * here's a great tool that fedora has that shows which libraries need changes (your links + better output) * here's a build server, where existing packages (maybe not fedora) can be tested and built on py39, new ABI, etc. * [optional] it would be awesome if anyone could submit their repo url and the server would automatically build wheels for all arch,abi and publish to pypi * let's notify package maintainers (presumably done?) and if that doesn't yield the desired result, * let's organise the open-source community (via fame or bounties) and fix (or fork) those that are important (common [transitive] deps to many fedora packages)

Yes! Except a few details, that's just what I'd like to do in the long term. Do you want to help?

I think this particular mess was caused by the hiding of “DeprecationWarning”s by default: If you don’t see any warnings cropping up in your production code, you don’t know you have to fix something. Some languages handle it like this: 1. Silent deprecation warning (deprecated in docs and/or hidden like in Python, the most eagle-eyed maintainers will see it and fix it) 2. Visible deprecation warning (people start seeing it and bug you in issues) 3. Hide things behind a flag: You’ll have breakage but you get a grace perior by enabling some scary looking environment variable or so (nobody can deny that things are getting serious) 4. Removal Going from 1 to 4 is of course pretty harsh, and this is what we’re doing. I propose that at least in the future, we have a schedule for every single deprecation to go to a gradual process like above. Best, Phil Am Fr., 31. Jan. 2020 um 05:21 Uhr schrieb Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp>:

Dima Tisnek writes:

...

I thought that collections compatibility was kept up to 3.8 specifically because py2 was alive. No that that requirement is gone, so should the shim, right?

Python 2 is still very much alive (even in a Python 3 venv :-þ):

(analysis.venv) 01 16:56$ /usr/bin/python Python 2.7.10 (default, Feb 22 2019, 21:55:15) [GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> ^D

We have stopped supporting Python 2 in the sense that we are no longer going to release source updates to Python 2. That doesn't necessarily mean we don't consider the needs of users who continue to depend on Python 2 for one reason or another, excellent or totally (in our oh-so-ever-'umble opinion) mistaken though that reason may be.

...

Ofc., my assumption could be just wrong...

It might be, it might not be, it might be situation-dependent (ie, decisions should be made case by case on a cost to Python core developers versus benefits to Python 2 users and those who support them in their libraries etc basis).

I think it would certainly be reasonable to make the decision that we're going to go full speed ahead with Python 3 and start stripping out Python 2-only features in the stdlib code. I don't support that, but I'm not a core developer, so that's a +1.0e-17 for keeping the code. I get the feeling that Guido is also not in favor of a Procrustean[1] approach to Python 2 support in the Python 3 stdlib, but I could be wrong. Since he's not BDFL, that's merely indicative, but IMO it's a pretty strong indicator.

What would really distress me however, is if we forget that we're not supporting *code*, we're supporting *users* by creating and maintaining code. Of course we can choose that user base, and may have to make painful decisions. But this community is about people supporting people, just like any community.

Footnotes: [1] Look up "Procrustes" in Wikipedia. It's not a story to tell in this family setting. ;-) _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WX4LA55V... Code of Conduct: http://python.org/psf/codeofconduct/

On 01/31/2020 10:47 AM, Mike Miller wrote:

On 2020-01-23 07:20, Victor Stinner wrote:

...

Python 3.9 introduces many small incompatible changes which broke tons

There's a well-known and established way of signaling breaking changes in software platforms—it is to increment the major version number.

Rather than debating the merits of breaking code on 3.9 or 3.10, wouldn't it make more sense to do it in a Python 4.0 instead?

I've gotta say, I like that plan. Instead of going to x.10, go to x+1.0. Every ten years we bump the major version and get rid of all the deprecations. -- ~Ethan~

On 2020-01-31 19:47, Mike Miller wrote:

On 2020-01-23 07:20, Victor Stinner wrote:

...

Python 3.9 introduces many small incompatible changes which broke tons

There's a well-known and established way of signaling breaking changes in software platforms—it is to increment the major version number.

Rather than debating the merits of breaking code on 3.9 or 3.10, wouldn't it make more sense to do it in a Python 4.0 instead?  Well, either of these strategies sound logical to me:

- Python 4.0 with removal of all of the Python 3-era deprecations - Continuing Python 3.1X with no breaks

In other words, we should keep compatibility, or not.  In any case, from the looks of it these will be tiny breaks compared to the Unicode transition.

The Unicode transition also looked very small back when 3.0 was planned. It only takes one such not-so-little thing to make a big breaking release like 3.0. And even if all the changes were little, I wouldn't want to inflict 10 years of papercuts at once. When the changes are rolled out gradually across minor releases, those that cause unforeseen trouble in real-world code can be identified in the alphas/betas, and rethought/reverted if necessary. Ethan Furman wrote:

I've gotta say, I like that plan. Instead of going to x.10, go to x+1.0. Every ten years we bump the major version and get rid of all the deprecations.

I don't. I hope the 10-year (and counting) transition from Python 2 to Python 3 will not become a tradition. I'd rather iterate on making removals less drastic (e.g. by making the DeprecationWarnings more visible). Iterate with a feedback loop, rather than do a one-time change and hope that everything goes well.

On 2020-02-03 12:55, Thomas Wouters wrote:

On Mon, Feb 3, 2020 at 10:53 AM Petr Viktorin mailto:encukou@gmail.com> wrote:

On 2020-01-31 19:47, Mike Miller wrote: > > On 2020-01-23 07:20, Victor Stinner wrote: >  > Python 3.9 introduces many small incompatible changes which broke tons > > > There's a well-known and established way of signaling breaking changes > in software platforms—it is to increment the major version number. > > Rather than debating the merits of breaking code on 3.9 or 3.10, > wouldn't it make more sense to do it in a Python 4.0 instead?  Well, > either of these strategies sound logical to me: > > - Python 4.0 with removal of all of the Python 3-era deprecations > - Continuing Python 3.1X with no breaks > > In other words, we should keep compatibility, or not.  In any case, from > the looks of it these will be tiny breaks compared to the Unicode > transition.

The Unicode transition also looked very small back when 3.0 was planned. It only takes one such not-so-little thing to make a big breaking release like 3.0. And even if all the changes were little, I wouldn't want to inflict 10 years of papercuts at once.

I agree with the sentiment that gradual deprecations are more easily managed, this statement about Python 3.0 is not true. The unicode transition was never thought to be small, and that's *why* 3.0 was such a big change.

Alright, "very small" is an overstatement. But it did seem much smaller than it turned out to be. https://docs.python.org/3.0/whatsnew/3.0.html lists it as the last of the big breaking changes, while most of the porting efforts were spent on it.

We knew it was going to break everything, so we took the opportunity to break more things, like the behaviour of indexing bytestrings. (Bytestrings were even going to be *mutable* for a while.) I think we can all agree that it was a mistake, and that's certainly something we don't want to repeat: even if we defer actual removal of features for a x.0.0 release, it must not become carte blanche for breaking things. >

When the changes are rolled out gradually across minor releases, those that cause unforeseen trouble in real-world code can be identified in the alphas/betas, and rethought/reverted if necessary.

That's also the case if things are (loudly) deprecated *but not removed* until a x.0.0 release. The replacement would already be in use for years before the old way would go away.

I fear that this is only so if no unexpected issues come up. If we do a loud deprecation, how can we be sure we did it right?

Ethan Furman wrote: > I've gotta say, I like that plan.  Instead of going to x.10, go to x+1.0.  Every ten years we bump the major version and get rid of all the deprecations.

I don't. I hope the 10-year (and counting) transition from Python 2 to Python 3 will not become a tradition. I'd rather iterate on making removals less drastic (e.g. by making the DeprecationWarnings more visible). Iterate with a feedback loop, rather than do a one-time change and hope that everything goes well.

On 2020-02-03 16:44, Antoine Pitrou wrote:

On Mon, 3 Feb 2020 13:18:46 +0100 Petr Viktorin wrote:

...

...

I agree with the sentiment that gradual deprecations are more easily managed, this statement about Python 3.0 is not true. The unicode transition was never thought to be small, and that's *why* 3.0 was such a big change.

Alright, "very small" is an overstatement. But it did seem much smaller than it turned out to be. https://docs.python.org/3.0/whatsnew/3.0.html lists it as the last of the big breaking changes, while most of the porting efforts were spent on it.

I don't think the order in this page has any relationship to the difficulty of each change. I'm not sure there was any particular reason to order them in this way, but it seems to me that the easier to understand changes were put first.

AFAIR, we were all acutely aware that the text model overhaul was going to be the hardest change for many users (especially those who have never encountered non-ASCII input). Most other changes in Python 3 have a predetermined context-free repair, while migrating to the new text model requires *thinking* about the context in which each Python 2 bytestring is used. It forces people to think about the problem they're solving in ways many of them didn't think of it before.

I stand corrected. Thank you!

On 03/02/2020 22:06, Mike Miller wrote:

On 2020-02-03 01:50, Petr Viktorin wrote:

...

When the changes are rolled out gradually across minor releases, those that cause unforeseen trouble in real-world code can be identified in the alphas/betas, and rethought/reverted if necessary.

To be clear, my suggestion was to maintain gradual deprecations and warnings, but have a single removal event at the start of a new version major number.  So there will be many years of betas and releases to haggle over.

I think that just enables laziness. "We don't need to worry about the deprecations, nothing is going to happen for years yet," is more or less what happened with the Python2 to Python3 shift. People naturally enjoy adding shiny new features to their projects over maintenance.

As a refinement to the idea above, perhaps a sub rule could be added:

  - No deprecations should appear in a X.9 release to give folks time     to prepare.

That feels like a very bad idea indeed. If you don't tell people things are going away, they definitely won't prepare for that. -- Rhodri James *-* Kynesim Ltd

On 2020-01-23 07:20, Victor Stinner wrote:

Python 3.9 introduces many small incompatible changes which broke tons

On 2020-01-31 19:47, Mike Miller wrote:

There's a well-known and established way of signaling breaking changes in software platforms—it is to increment the major version number.

Rather than debating the merits of breaking code on 3.9 or 3.10, wouldn't it make more sense to do it in a Python 4.0 instead? Well, either of these strategies sound logical to me:

- Python 4.0 with removal of all of the Python 3-era deprecations - Continuing Python 3.1X with no breaks

In other words, we should keep compatibility, or not. In any case, from the looks of it these will be tiny breaks compared to the Unicode transition.

Ethan Furman wrote:

I've gotta say, I like that plan. Instead of going to x.10, go to x+1.0. Every ten years we bump the major version and get rid of all the deprecations.

Petr Viktorin wrote:

I don't. I hope the 10-year (and counting) transition from Python 2 to Python 3 will not become a tradition. I'd rather iterate on making removals less drastic (e.g. by making the DeprecationWarnings more visible). Iterate with a feedback loop, rather than do a one-time change and hope that everything goes well.

As a user I would much rather know that my 3.2 code worked in every version of 3.x, and not have to make changes in 3.5 and 3.7 and 3.11. Talk about death by paper cuts! I'd either be stuck updating already working code to get the benefits of the latest Python 3, or having multiple versions of Python 3 on my system. Both options are galling. Having the latest Python 2, the latest Python 3, and the latest Python 4 is much more palatable. -- ~Ethan~

On 2020-02-03 17:00, Brett Cannon wrote:

Until you're being asked to maintain all of that for a decade. We paid a major price keeping Python 2 alive for over a decade. Now I'm not saying it wasn't the right thing to do considering what we changed, but for the stuff we are talking about removing it doesn't require a massive rewrite on the behalf of users. And we know from experience anything that is left in will get used no matter how loudly we try to broadcast that fact (and we know people still do not have a habit of running their code with warnings turned on).

Potentially up to a decade, often not. We seem to agree that these are most likely minor things like duplicate import paths and such. The maintenance burden is low on these items. Also, due to skittishness, removals have drug out to five plus releases anyway. So, why the aversion to formalizing the process, instead of "winging it" every release? Why force devs/end-users to maintain state about what release is compatible or waste time on investigation? Really, this was solved decades ago.  Everyone knows what to do when a major version number changes. Other language platforms are not afraid to change them, likely because the breaks were typically minor and not break-the-world events. Just like we're talking about here. More broadly, an agile cadence is great for innovative apps, not so great for mature platforms. Platforms are supposed to provide stability and predictability, and the popular ones do. -Mike

On 2020-02-04 12:10, Brett Cannon wrote:

Please be careful making that claim. Over my 16 years of helping manage this project I can tell you that claim is not universally true no matter how small and simple you think something is.

The answer to that concern is to not break compatibility in the first place, and/or revert it when the mistake is discovered. It happens. As mentioned, significant breaks are now discouraged. The code is already written, so I'd argue keeping it deprecated potentially a few years extra in a regular process is not an undue burden.

Python predates semver. Assume every feature/minor release potentially has a breaking change and we have (hopefully) been raising warnings to the user for the past two years about the breaking change coming.

Hmm, was thinking about MS-DOS when I wrote that. Still, I don't think this a good system having breaking changes in the minor version number. It's not traditional and it's not newfangled either, it's relatively unique. On that note, just noticed that Wikipedia has its own section for Python under the "Software Versioning" article: The PSF has published PEP 440 -- Version Identification and Dependency Specification,[16] outlining their own flexible (complicated) scheme, that defines an epoch segment, a release segment, pre-release and post-release segments and a development release segment. https://en.wikipedia.org/wiki/Software_versioning#Python The main argument here seems to be it this uncommon versioning scheme saves core developer time, (unsaid) at the expense of dev or end-user time. As there are probably a million end-users per Python contributor, I argue this is not a good tradeoff to make. -Mike

On 2020-02-04 14:40, Paul Moore wrote:

...

The answer to that concern is to not break compatibility in the first place, and/or revert it when the mistake is discovered. It happens.

That sounds to me like an argument for stagnation. We already take backwards compatibility very seriously. If you're suggesting that we don't even have the option of deprecating things, then I'm not sure how you expect the language to evolve.

No, I chose the wrong phrasing there. Have been arguing for deprecations to occur on schedule but removals to occur later. By "not break compatibility…" I meant not to break it in a large, destructive manner. Small ones, sure.

The point has already been made that "keeping code around but deprecated" isn't the problem, it's bug triage, telling people who report problems with deprecated code that "this is not going to be fixed", educating people who copy/paste old and deprecated code and wonder why they are now getting warnings with it, etc. I think it's probably up to the core devs (who do the work) to judge what is an "undue burden", but if you do want to try to judge it, please take all of those extra tasks into account when reckoning up.

This happens already and won't go away. I'm arguing that a *very* predictable release process helps and doesn't hurt in this department, resulting in fewer questions. Instead of every release is a unique snowflake to be considered. Still have questions? Go to FAQ #42.

Restricting compatibility breaks to the major version isn't that important in my experience. IMO, the version number isn't anywhere near as important as the *process* of handling backward

It matters, when say Ubuntu drops X.4 when X.6 comes out and X.6 isn't backwards compatible. You get forced to update or find one of the various workarounds. I argue this should happen less frequently at predictable times.

No, it's that conserving the *extremely* limited resource that is freely donated, volunteer core developer time is essential if we want to have a viable core development team at all.

Sorry, didn't mean that part as an attack, rather an illustration of the tradeoff being made. Now if PSF needs help to build a more predictable release process, well then, by all means ask. But that can't happen until the decision is made to do it. -Mike

Hi, I proposed a second change to restore the "U" open() mode (universal newlines): https://github.com/python/cpython/pull/18767 Again, it's only kept in Python 3.9 to ease migration from Python 2, but will be removed in Python 3.10. By the way, I also added a new "You should check for DeprecationWarning in your code" section to "What's New In Python 3.9" document, to strongly advice users to check for deprecation warnings in their code: https://docs.python.org/dev/whatsnew/3.9.html#you-should-check-for-deprecati... Victor Le mar. 18 févr. 2020 à 12:37, Victor Stinner a écrit :

Hi,

I created an issue:

"Keep deprecated features in Python 3.9 to ease migration from Python 2.7, but remove in Python 3.10" https://bugs.python.org/issue39674

And proposed a first pull request to add again collections.Mapping:

https://github.com/python/cpython/pull/18545

Victor