Skip to content

bpo-43313: support pymalloc for subinterpreters. each subinterpreter ha…#24857

Open
dexterhahaha wants to merge 1 commit intopython:mainfrom
dexterhahaha:fix/pymalloc_subinterpreter
Open

bpo-43313: support pymalloc for subinterpreters. each subinterpreter ha…#24857
dexterhahaha wants to merge 1 commit intopython:mainfrom
dexterhahaha:fix/pymalloc_subinterpreter

Conversation

@dexterhahaha
Copy link
Copy Markdown
Contributor

@dexterhahaha dexterhahaha commented Mar 14, 2021

two changes:

  1. support pymalloc for subinterpreters. each subinterpreter has pymalloc_state

  2. _copy_raw_string api alloc memory use PyMem_RawFree and PyMem_RawMalloc.

I extend _xxsubinterpretermodule.c to support call any function in sub interpreter.
when i need return result from sub interpreter call.

  1. i need create item->name in shared item. will use pymem_malloc api to manage memory. when with_pymalloc macro defined, it will create memory and bound to interpreter(iterp1) pymalloc state.

  2. after switch interpreter state, now in iterp2 state, get return value from shareditem, and i need free shared item. but item->name memory managed by interp1 pymalloc state. if i want to free them, i need switch to interpreter state 1. it's complicated.

so i think, in _sharednsitem_init _copy_raw_string, item->name need malloc by PyMem_RawAPI. easy to management.

static int
_sharednsitem_init(struct _sharednsitem *item, PyObject *key, PyObject *value)
{
    item->name = _copy_raw_string(key);

...

static char *
_copy_raw_string(PyObject *strobj)
{
    const char *str = PyUnicode_AsUTF8(strobj);
    if (str == NULL) {
        return NULL;
    }
    char *copied = PyMem_Malloc(strlen(str)+1);
    if (copied == NULL) {
        PyErr_NoMemory();
        return NULL;
    }
    strcpy(copied, str);
    return copied;
}

_sharedns *result_shread = _sharedns_new(1);


#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS
    // Switch to interpreter.
    PyThreadState *new_tstate = PyInterpreterState_ThreadHead(interp);
    PyThreadState *save1 = PyEval_SaveThread();

    (void)PyThreadState_Swap(new_tstate);
#else
    // Switch to interpreter.
    PyThreadState *save_tstate = NULL;
    if (interp != PyInterpreterState_Get()) {
        // XXX Using the "head" thread isn't strictly correct.
        PyThreadState *tstate = PyInterpreterState_ThreadHead(interp);
        // XXX Possible GILState issues?
        save_tstate = PyThreadState_Swap(tstate);
    }
#endif
    
    PyObject *module = PyImport_ImportModule(PyUnicode_AsUTF8(module_name));
    PyObject *function = PyObject_GetAttr(module, function_name);
    
    result = PyObject_Call(function, args, kwargs);

    if (result == NULL) {
        // exception handler
        ...
    }

    if (result && _sharednsitem_init(&result_shread->items[0], PyUnicode_FromString("result"), result) != 0) {
        PyErr_Format(RunFailedError, "interp_call_function result convert to shared failed");
        return NULL;;
    }
#ifdef EXPERIMENTAL_ISOLATED_SUBINTERPRETERS
    // Switch back.
    PyEval_RestoreThread(save1);
#else
    // Switch back.
    if (save_tstate != NULL) {
        PyThreadState_Swap(save_tstate);
    }
#endif
    // ...

    if (result) {
        result = _PyCrossInterpreterData_NewObject(&result_shread->items[0].data);
        _sharedns_free(result_shread);
    }

https://bugs.python.org/issue43313

@dexterhahaha dexterhahaha changed the title feature: support pymalloc for subinterpreters. each subinterpreter ha… bpo-43313: support pymalloc for subinterpreters. each subinterpreter ha… Mar 14, 2021
@dexterhahaha
Copy link
Copy Markdown
Contributor Author

I found the creation and release of the channel object in different interpreters, I will fix it

@github-actions
Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 15, 2021
@github-actions github-actions Bot removed the stale Stale PR or inactive for long period of time. label Aug 5, 2022
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 9, 2026

This PR is stale because it has been open for 30 days with no activity.

@github-actions github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting review stale Stale PR or inactive for long period of time.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants