bpo-37603: tok_nextc() now also updates multi_line_start on REALLOC()#14789
bpo-37603: tok_nextc() now also updates multi_line_start on REALLOC()#14789vstinner wants to merge 1 commit intopython:masterfrom vstinner:tokenizer_realloc
Conversation
Fix Python parser crash: when the tokenizer grows its internal buffer using REALLOC(), update also tok->multi_line_start.
|
I don't know well this parser/tokenizer code of Python, so I'm not confident in my change. At least, it fix https://bugs.python.org/issue37603 and all tests pass. cc @asottile |
|
I have (what I believe to be) an equivalent PR here: https://github.com/python/cpython/pull/14433/files |
Your change is "tok->multi_line_start = tok->buf + cur_multi_line_start; ". It's different than mine. I have no idea which change is correct :-) |
|
let me check out your branch and see if it fixes both issues :) |
|
yeah this PR doesn't pass the test from the other PR: Notably it ends up with a misplaced syntax error pointer: |
|
My PR is wrong, PR #14433 seems to be the correct fix and includes a test. |
Fix Python parser crash: when the tokenizer grows its internal buffer
using REALLOC(), update also tok->multi_line_start.
https://bugs.python.org/issue37603