There is also another corner case to consider:

class C:
    exec('x: int')

assert C.__annotations__ == {'x': int}
assert __annotations__ == {}

I am not sure this one will be covered correctly.
But the main argument here is speed I think.

Let us see what others think about this.