b'abc\xff'.decode('punycode', 'ignore') raises the same error than b'abc\xff'.decode('punycode', 'replace') or b'abc\xff'.decode('punycode'): it uses the strict error handler, and simply ignores the error handler.

punycodec, as idna, should raise an exception if the error handler is different than strict.

base64_codec.py uses an assertion to check the error handler: it should use an if+raise (assertions are ignored in optimized mode, python -O).

punycode_errors.patch:
 - raise an error if errors="replace": only accept strict and ignore
 - use errors to encode/decode to/from ASCII
 - add unit tests

I don't think that the change should be documented, because punycode has no section in Doc/library/codecs.rst, and I hope that nobody uses something different than strict :-)

@Martin: Can you review my patch?

What's the point of disallowing the replace error handler? That's a slightly incompatible change, isn't it?

Oh, I forgot to give a little bit more details.

b'abc\xff-'.decode('punycode', 'ignore') and b'abc\xff-'.decode('punycode', 'replace') raise a UnicodeDecodeError: the error handler is just useless (ignored) here.

With my patch, b'abc\xff-'.decode('punycode', 'ignore') gives 'abc'. (If I change the code to accept replace) b'abc\xff-'.decode('punycode', 'replace') gives also 'abc', but 'replace' doesn't work correctly in the part after "-" contain illegal byte sequences.

For example, b'a\xff-\xffb\xffga\xff'.decode("punycode", "replace") gives 'a�', whereas I would expect 'a�é' or 'aé�'. b'a-bga\xff'.decode("punycode", "replace") gives 'aé' as b'a-bga'.decode("punycode", "replace"), whereas I would expect 'aé�' or something like that.

> What's the point of disallowing the replace error handler?

It's just that I'm unable to patch punycode decoder to support the replace handler. Do you want to "implement" it?

> That's a slightly incompatible change, isn't it?

I don't think so because I consider that the punycode decoder never supported error handlers (other than strict) in Python 3.

What do you think?

> Oh, I forgot to give a little bit more details.
> 
> b'abc\xff-'.decode('punycode', 'ignore') and
> b'abc\xff-'.decode('punycode', 'replace') raise a UnicodeDecodeError:
> the error handler is just useless (ignored) here.

That's not my point:

b"foo".decode("punycode","replace")

currently succeeds, but raises an UnicodeError under the patch.

I prefer to not touch punycode right now, it works, there is no need to modify it.