From 61cfaae279e1c594cd64d7cb95b23bac9ffc8f0e Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Sun, 4 Aug 2019 16:48:50 -0700 Subject: [PATCH 01/13] Add examples and explanations for the walrus operator --- Doc/whatsnew/3.8.rst | 44 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 42 insertions(+), 2 deletions(-) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 0455688841fbf8..094491d4a1148b 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -76,12 +76,52 @@ New Features Assignment expressions ---------------------- -There is new syntax (the "walrus operator", ``:=``) to assign values -to variables as part of an expression. Example:: +There is new syntax ``:=`` that assigns values to variables as part of a larger +expression. It is affectionately known as "walrus operator" due to +its resemblance to `the eyes and tusks of a walrus +`_. + +In this example, the assignment expression helps avoid calling +:func:`len` twice:: if (n := len(a)) > 10: print(f"List is too long ({n} elements, expected <= 10)") +A similar benefit arises during regular expression matching where +match objects are needed twice, once to test whether a match +occurred and another to extract a subgroup:: + + discount = 0.0 + if (mo := re.search(r'(\d+)% discount', advertisement)): + discount = float(mo.group(1)) / 100.0 + +The operator is also useful with while-loops that compute +a value to test loop termination and then need that same +value again in the body of the loop:: + + # Loop over fixed length blocks + while (block := f.read(256)) != '': + process(block) + +Another motivating use case arises in list comprehensions where +a value computed in a filtering condition is also needed in +the expression body:: + + [clean_name.title() for name in names + if (clean_name := normalize('NFC', name)) in allowed_names] + +Try to limit use of the walrus operator to clean cases that reduce +complexity and improve readability. When combined with other features +like f-strings or conditional expressions, it is easy to go over-board +with code that is challenging to decipher:: + + # Cryptic assignment expression embedded in a conditional expression + binomial_coefficient = [(c := c * (n - k + 1) // k if k else 1) for k in range(n+1)] + + # Visually confusing assignment expression embedded in an f-string + for angle in range(360): + print(f'{angle=}\N{degree sign} {(theta:=radians(angle))=:.3f} {sin(theta)=:.3f}') + See :pep:`572` for a full description. (Contributed by Emily Morehouse in :issue:`35224`.) From fbd8b703973b7cf338095de791f7919625ee2238 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Sun, 4 Aug 2019 17:43:42 -0700 Subject: [PATCH 02/13] Add motivating examples and rationale for positional-only arguments --- Doc/whatsnew/3.8.rst | 71 +++++++++++++++++++++++++++++++++++++------- 1 file changed, 60 insertions(+), 11 deletions(-) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 094491d4a1148b..4394d0c7842d5f 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -132,20 +132,69 @@ See :pep:`572` for a full description. Positional-only parameters -------------------------- -There is new syntax (``/``) to indicate that some function parameters -must be specified positionally (i.e., cannot be used as keyword -arguments). This is the same notation as shown by ``help()`` for -functions implemented in C (produced by Larry Hastings' "Argument -Clinic" tool). Example:: +There is a new function parameter syntax ``/`` to indicate that some +function parameters must be specified positionally and cannot be used as +keyword arguments. This is the same notation shown by ``help()`` for C +functions annotated with Larry Hastings' `Argument Clinic +`_ tool. + +In the following example, parameters *a* and *b* are positional-only, +while *c* or *d* can be positional or keyword, and *e* or *f* are +required to be keywords:: + + def f(a, b, \, c, d, *, e, f): + print(a, b, c, d, e, f) + +The following is a valid call:: + + f(10, 20, 30, d=40, e=50, f=60) + +However, these are invalid calls:: + + f(10, b=20, c=30, d=40, e=50, f=60) # b cannot be a keyword argument + f(10, 20, 30, 40, 50, f=60) # e must be a keyword argument + +One use case for this notation is that it allows pure Python functions +to fully emulate behaviors of existing C coded functions. For example, +the built-in :func:`pow` function does not accept keyword arguments:: def pow(x, y, z=None, /): - r = x**y - if z is not None: - r %= z - return r + "Emulate the built in pow() function" + r = x ** y + return r if z is None else r%z + +Another use case is to preclude keyword arguments when the parameter +name is not helpful. For example, the builtin :func:`len` function has +the signature ``len(obj, /)``. This precludes awkward calls such as:: + + len(obj='hello') # The "obj" keyword argument impairs readability + +A further benefit of marking a parameter as positional-only is that it +allows the parameter name to be changed in the future without risk of +breaking client code. For example, in the :mod:`statistics` module, the +parameter name *dist* may be changed in the future. This was made +possible with the following function specification:: + + def quantiles(dist, /, *, n=4, method='exclusive') + ... + +Since the parameters to the left of ``/`` are not exposed as possible +keywords, the parameters names remain available for use in ``**kwargs``:: + + >>> def f(a, b, /, **kwargs): + ... print(a, b, kwargs) + ... + >>> f(10, 20, a=1, b=2, c=3) # a and b are used in two ways + 10 20 {'a': 1, 'b': 2, 'c': 3} + +This greatly simplifies the implementation of functions and methods +that need to accept arbitrary keyword arguments. For example, here +is an except from code in the :mod:`collections` module:: + + class Counter(dict): -Now ``pow(2, 10)`` and ``pow(2, 10, 17)`` are valid calls, but -``pow(x=2, y=10)`` and ``pow(2, 10, z=17)`` are invalid. + def __init__(self, iterable=None, /, **kwds): + # Note "iterable" is a possible keyword argument See :pep:`570` for a full description. From c6cbeacd12a0eb914f80d11c864ba73a22b0e45c Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Sun, 4 Aug 2019 17:54:04 -0700 Subject: [PATCH 03/13] Add editor. Clean-up heading and intro. --- Doc/whatsnew/3.8.rst | 19 ++++++++----------- 1 file changed, 8 insertions(+), 11 deletions(-) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 4394d0c7842d5f..9604943c702206 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -42,21 +42,18 @@ This saves the maintainer the effort of going through the Mercurial log when researching a change. -This article explains the new features in Python 3.8, compared to 3.7. +:Editor: Raymond Hettinger +This article explains the new features in Python 3.8, compared to 3.7. For full details, see the :ref:`changelog `. -.. note:: - - Prerelease users should be aware that this document is currently in draft - form. It will be updated substantially as Python 3.8 moves towards release, - so it's worth checking back even after reading earlier versions. - - Some notable items not yet covered here: +Prerelease users should be aware that this document is currently in +draft form. It will be updated as Python 3.8 moves towards release, so +it's worth checking back even after reading earlier versions. Some +notable items not yet covered are: - * :pep:`578` - Runtime audit hooks for potentially sensitive operations - * ``python -m asyncio`` runs a natively async REPL - * ... +* :pep:`578` - Runtime audit hooks for potentially sensitive operations +* ``python -m asyncio`` runs a natively async REPL Summary -- Release highlights From 980b4524bb1f21a20c9d1cdae041fa0bb473ad1e Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Sun, 4 Aug 2019 18:27:14 -0700 Subject: [PATCH 04/13] Add missing detail and examples to the f-string '=' specifier --- Doc/whatsnew/3.8.rst | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 9604943c702206..ce125338e11d28 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -260,17 +260,31 @@ Android and Cygwin, whose cases are handled by the script); this change is backward incompatible on purpose. (Contributed by Victor Stinner in :issue:`36721`.) -f-strings now support = for quick and easy debugging ------------------------------------------------------ -Add ``=`` specifier to f-strings. ``f'{expr=}'`` expands -to the text of the expression, an equal sign, then the repr of the -evaluated expression. So:: +f-strings support ``=`` for self-documenting expressions and debugging +---------------------------------------------------------------------- + +Added an ``=`` specifier to :term:`f-string`\s. An f-string such as +``f'{expr=}'`` will expand to the text of the expression, an equal sign, +then the representation of the evaluated expression. For example: + + >>> user = 'eric_idle' + >>> member_since = date(1975, 7, 31) + >>> f'{user=} {member_since=}' + "user='eric_idle' member_since=datetime.date(1975, 7, 31)" + +The usual :ref:`f-string format specifiers ` allow more +control over how the result of the expression is displayed:: + + >>> delta = date.today() - member_since + >>> f'{user=!s} {delta.days=:,d}' + 'user=eric_idle delta.days=16,075' - x = 3 - print(f'{x*9 + 15=}') +The ``=`` specifier will display the whole expression so that +calculations can be shown:: -Would print ``x*9 + 15=42``. + >>> print(f'{theta=} {cos(radians(theta))=:.3f}') + theta=30 cos(radians(theta))=0.866 (Contributed by Eric V. Smith and Larry Hastings in :issue:`36817`.) From 4712127b04222613d1c4f1945eb11b6bc38077d2 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Sun, 4 Aug 2019 19:18:04 -0700 Subject: [PATCH 05/13] Add examples, motivation, and links for the modulo multiplicative inverse --- Doc/whatsnew/3.8.rst | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index ce125338e11d28..7256d3f3dd89de 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -443,10 +443,30 @@ Other Language Changes * Added new ``replace()`` method to the code type (:class:`types.CodeType`). (Contributed by Victor Stinner in :issue:`37032`.) -* For integers, the three-argument form of the :func:`pow` function now permits - the exponent to be negative in the case where the base is relatively prime to - the modulus. It then computes a modular inverse to the base when the exponent - is ``-1``, and a suitable power of that inverse for other negative exponents. +* For integers, the three-argument form of the :func:`pow` function now + permits the exponent to be negative in the case where the base is + relatively prime to the modulus. It then computes a modular inverse to + the base when the exponent is ``-1``, and a suitable power of that + inverse for other negative exponents. For example, to compute the + `modular multiplicative inverse + `_ of 38 + in modulo 137, write:: + + >>> pow(38, -1, 137) + 119 + >>> 119 * 38 % 137 + 1 + + Modular inverses arise in the solution of `linear Diophantine + equations `_. + For example, to find integer solutions for ``4258𝑥 + 147𝑦 = 369``, + first rewrite as ``4258𝑥 ≡ 369 (mod 147)`` then solve: + + >>> x = 369 * pow(4258, -1, 147) % 147 + >>> y = (4258 * x - 369) // -147 + >>> 4258 * x + 147 * y + 369 + (Contributed by Mark Dickinson in :issue:`36027`.) * When dictionary comprehensions are evaluated, the key is now evaluated before From 25fa5b218a4639d2e092a12f3d1da8208690ca79 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Sun, 4 Aug 2019 19:21:06 -0700 Subject: [PATCH 06/13] Add blurb --- .../next/Documentation/2019-08-04-19-20-58.bpo-37759.EHRF4i.rst | 1 + 1 file changed, 1 insertion(+) create mode 100644 Misc/NEWS.d/next/Documentation/2019-08-04-19-20-58.bpo-37759.EHRF4i.rst diff --git a/Misc/NEWS.d/next/Documentation/2019-08-04-19-20-58.bpo-37759.EHRF4i.rst b/Misc/NEWS.d/next/Documentation/2019-08-04-19-20-58.bpo-37759.EHRF4i.rst new file mode 100644 index 00000000000000..90fb7213ebdeb9 --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2019-08-04-19-20-58.bpo-37759.EHRF4i.rst @@ -0,0 +1 @@ +Beginning edits to Whatsnew 3.8 From a42ec6f8de9ef5d0d7e7a45cef4fec8a3e358c59 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Sun, 4 Aug 2019 20:14:10 -0700 Subject: [PATCH 07/13] Add testsetup for doctests --- Doc/whatsnew/3.8.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 7256d3f3dd89de..2fed987a5d51d2 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -55,6 +55,11 @@ notable items not yet covered are: * :pep:`578` - Runtime audit hooks for potentially sensitive operations * ``python -m asyncio`` runs a natively async REPL +.. testsetup:: + + from datetime import date + from math import cos, radians + Summary -- Release highlights ============================= From 69c050c24705dae296f3f2dcfdb5681530df42fb Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Sun, 4 Aug 2019 20:23:13 -0700 Subject: [PATCH 08/13] Convert backslash to forward slash (spotted by Carol) --- Doc/whatsnew/3.8.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 2fed987a5d51d2..638abad26a8530 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -144,7 +144,7 @@ In the following example, parameters *a* and *b* are positional-only, while *c* or *d* can be positional or keyword, and *e* or *f* are required to be keywords:: - def f(a, b, \, c, d, *, e, f): + def f(a, b, /, c, d, *, e, f): print(a, b, c, d, e, f) The following is a valid call:: From cdc6ee6dba99242dced7e69c045453aaf87dceae Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Sun, 4 Aug 2019 20:40:48 -0700 Subject: [PATCH 09/13] Add example for unicode names in regular expressions --- Doc/whatsnew/3.8.rst | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 638abad26a8530..f35faccd7308c1 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -59,6 +59,7 @@ notable items not yet covered are: from datetime import date from math import cos, radians + import re Summary -- Release highlights @@ -400,7 +401,13 @@ Other Language Changes or :meth:`~object.__complex__` is not available. (Contributed by Serhiy Storchaka in :issue:`20092`.) -* Added support of ``\N{name}`` escapes in :mod:`regular expressions `. +* Added support of ``\N{name}`` escapes in :mod:`regular expressions `:: + + >>> notice = 'Copyright © 2019' + >>> copyright_year_pattern = re.compile(r'\N{copyright sign}\s*(\d{4})') + >>> int(copyright_year_pattern.search(notice).group(1)) + 2019 + (Contributed by Jonathan Eunice and Serhiy Storchaka in :issue:`30688`.) * Dict and dictviews are now iterable in reversed insertion order using From d56eb8a83f3c85099da8280d6a891fbf518827dc Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Sun, 4 Aug 2019 21:30:49 -0700 Subject: [PATCH 10/13] Add a motivating example for math.prod() --- Doc/whatsnew/3.8.rst | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index f35faccd7308c1..8b1fbc78aa2d1a 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -60,6 +60,7 @@ notable items not yet covered are: from datetime import date from math import cos, radians import re + import math Summary -- Release highlights @@ -708,7 +709,14 @@ Formerly, it only supported the 2-D case. Added new function, :func:`math.prod`, as analogous function to :func:`sum` that returns the product of a 'start' value (default: 1) times an iterable of -numbers. (Contributed by Pablo Galindo in :issue:`35606`) +numbers:: + + >>> prior = 0.8 + >>> likelihoods = [0.625, 0.84, 0.30] + >>> (link: http://math.prod) math.prod(likelihoods, start=prior) + 0.126 + +(Contributed by Pablo Galindo in :issue:`35606`) Added new function :func:`math.isqrt` for computing integer square roots. (Contributed by Mark Dickinson in :issue:`36887`.) From 74ecd05acd4fb846ae348966eaf97c258fd401bf Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Sun, 4 Aug 2019 21:51:17 -0700 Subject: [PATCH 11/13] Minor grammar fix --- Doc/whatsnew/3.8.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 8b1fbc78aa2d1a..f0d1c134960de9 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -1497,7 +1497,7 @@ Changes in the Python API * :func:`shutil.copyfile` default buffer size on Windows was changed from 16 KiB to 1 MiB. -* ``PyGC_Head`` struct is changed completely. All code touched the +* The ``PyGC_Head`` struct has changed completely. All code that touched the struct member should be rewritten. (See :issue:`33597`) * The ``PyInterpreterState`` struct has been moved into the "internal" From ca613e45ea7101318e8a2520d452dbfc269bc524 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Sun, 4 Aug 2019 22:00:31 -0700 Subject: [PATCH 12/13] Remove the walrus operator anti-pattern examples (suggested by GvR) --- Doc/whatsnew/3.8.rst | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index f0d1c134960de9..3b7f7f99203bd0 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -115,16 +115,7 @@ the expression body:: if (clean_name := normalize('NFC', name)) in allowed_names] Try to limit use of the walrus operator to clean cases that reduce -complexity and improve readability. When combined with other features -like f-strings or conditional expressions, it is easy to go over-board -with code that is challenging to decipher:: - - # Cryptic assignment expression embedded in a conditional expression - binomial_coefficient = [(c := c * (n - k + 1) // k if k else 1) for k in range(n+1)] - - # Visually confusing assignment expression embedded in an f-string - for angle in range(360): - print(f'{angle=}\N{degree sign} {(theta:=radians(angle))=:.3f} {sin(theta)=:.3f}') +complexity and improve readability. See :pep:`572` for a full description. From fcd791b8bc69d7ac8844f081c134ba33c1ee3247 Mon Sep 17 00:00:00 2001 From: Raymond Hettinger Date: Mon, 5 Aug 2019 08:27:00 -0700 Subject: [PATCH 13/13] Dropped an unconventional preposition (suggested by Jeroen) --- Doc/whatsnew/3.8.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/whatsnew/3.8.rst b/Doc/whatsnew/3.8.rst index 3b7f7f99203bd0..d8062e772f6e0c 100644 --- a/Doc/whatsnew/3.8.rst +++ b/Doc/whatsnew/3.8.rst @@ -454,7 +454,7 @@ Other Language Changes inverse for other negative exponents. For example, to compute the `modular multiplicative inverse `_ of 38 - in modulo 137, write:: + modulo 137, write:: >>> pow(38, -1, 137) 119