`test_init_use_frozen_modules` fails on OpenIndiana

[mtelka]

The test_init_use_frozen_modules test fails on OpenIndiana.

test_init_use_frozen_modules (test.test_embed.InitConfigTests.test_init_use_frozen_modules) ... --- ['/data/builds/ul-workspace/components/python/python-314/build/amd64/Programs/_testembed', 'test_init_use_frozen_modules'] failed ---
stdout:

stderr:
Fatal Python error: memory allocation failed
Python runtime state: core initialized

Current thread 0x0000000000000001 (most recent call first):
  <no Python frame>

------

  test_init_use_frozen_modules (test.test_embed.InitConfigTests.test_init_use_frozen_modules) ['=off'] ... FAIL
--- ['/data/builds/ul-workspace/components/python/python-314/build/amd64/Programs/_testembed', 'test_init_use_frozen_modules'] failed ---
stdout:

stderr:
Fatal Python error: memory allocation failed
Python runtime state: core initialized

Current thread 0x0000000000000001 (most recent call first):
  <no Python frame>

------
  test_init_use_frozen_modules (test.test_embed.InitConfigTests.test_init_use_frozen_modules) ['=on'] ... FAIL
--- ['/data/builds/ul-workspace/components/python/python-314/build/amd64/Programs/_testembed', 'test_init_use_frozen_modules'] failed ---
stdout:

stderr:
Fatal Python error: memory allocation failed
Python runtime state: core initialized

Current thread 0x0000000000000001 (most recent call first):
  <no Python frame>

------
  test_init_use_frozen_modules (test.test_embed.InitConfigTests.test_init_use_frozen_modules) ['='] ... FAIL

======================================================================
FAIL: test_init_use_frozen_modules (test.test_embed.InitConfigTests.test_init_use_frozen_modules) ['=off']
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/builds/ul-workspace/components/python/python-314/Python-3.14.0/Lib/test/test_embed.py", line 1827, in test_init_use_frozen_modules
    self.check_all_configs("test_init_use_frozen_modules", config,
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                           api=API_PYTHON, env=env)
                           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/builds/ul-workspace/components/python/python-314/Python-3.14.0/Lib/test/test_embed.py", line 941, in check_all_configs
    out, err = self.run_embedded_interpreter(testname,
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
                                             env=env, cwd=cwd)
                                             ^^^^^^^^^^^^^^^^^
  File "/data/builds/ul-workspace/components/python/python-314/Python-3.14.0/Lib/test/test_embed.py", line 138, in run_embedded_interpreter
    self.assertEqual(p.returncode, returncode,
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
                     "bad returncode %d, stderr is %r" %
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                     (p.returncode, err))
                     ^^^^^^^^^^^^^^^^^^^^
AssertionError: 1 != 0 : bad returncode 1, stderr is 'Fatal Python error: memory allocation failed\nPython runtime state: core initialized\n\nCurrent thread 0x0000000000000001 (most recent call first):\n  <no Python frame>\n'

======================================================================
FAIL: test_init_use_frozen_modules (test.test_embed.InitConfigTests.test_init_use_frozen_modules) ['=on']
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/builds/ul-workspace/components/python/python-314/Python-3.14.0/Lib/test/test_embed.py", line 1827, in test_init_use_frozen_modules
    self.check_all_configs("test_init_use_frozen_modules", config,
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                           api=API_PYTHON, env=env)
                           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/builds/ul-workspace/components/python/python-314/Python-3.14.0/Lib/test/test_embed.py", line 941, in check_all_configs
    out, err = self.run_embedded_interpreter(testname,
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
                                             env=env, cwd=cwd)
                                             ^^^^^^^^^^^^^^^^^
  File "/data/builds/ul-workspace/components/python/python-314/Python-3.14.0/Lib/test/test_embed.py", line 138, in run_embedded_interpreter
    self.assertEqual(p.returncode, returncode,
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
                     "bad returncode %d, stderr is %r" %
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                     (p.returncode, err))
                     ^^^^^^^^^^^^^^^^^^^^
AssertionError: 1 != 0 : bad returncode 1, stderr is 'Fatal Python error: memory allocation failed\nPython runtime state: core initialized\n\nCurrent thread 0x0000000000000001 (most recent call first):\n  <no Python frame>\n'

======================================================================
FAIL: test_init_use_frozen_modules (test.test_embed.InitConfigTests.test_init_use_frozen_modules) ['=']
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/builds/ul-workspace/components/python/python-314/Python-3.14.0/Lib/test/test_embed.py", line 1827, in test_init_use_frozen_modules
    self.check_all_configs("test_init_use_frozen_modules", config,
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                           api=API_PYTHON, env=env)
                           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/builds/ul-workspace/components/python/python-314/Python-3.14.0/Lib/test/test_embed.py", line 941, in check_all_configs
    out, err = self.run_embedded_interpreter(testname,
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
                                             env=env, cwd=cwd)
                                             ^^^^^^^^^^^^^^^^^
  File "/data/builds/ul-workspace/components/python/python-314/Python-3.14.0/Lib/test/test_embed.py", line 138, in run_embedded_interpreter
    self.assertEqual(p.returncode, returncode,
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
                     "bad returncode %d, stderr is %r" %
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                     (p.returncode, err))
                     ^^^^^^^^^^^^^^^^^^^^
AssertionError: 1 != 0 : bad returncode 1, stderr is 'Fatal Python error: memory allocation failed\nPython runtime state: core initialized\n\nCurrent thread 0x0000000000000001 (most recent call first):\n  <no Python frame>\n'

The immediate cause for the failure is that in the _PyConfig_AsDict() function the config_get() call fails (returns NULL) for xoptions and orig_argv specs.

Linked PRs

• gh-139927: Fix test_embed on OpenIndiana #142514
• [3.14] gh-139927: Fix test_embed on OpenIndiana (GH-142514) #142520
• [3.13] gh-139927: Fix test_embed on OpenIndiana (GH-142514) #142521

[vstinner]

Can you run directly the test command? Example:

TESTFROZEN=on ./Programs/_testembed test_init_use_frozen_modules

It should output JSON to stdout. You can use this command to have a pretty output:

TESTFROZEN=on ./Programs/_testembed test_init_use_frozen_modules | ./python -m json

The immediate cause for the failure is that in the _PyConfig_AsDict() function the config_get() call fails (returns NULL) for xoptions and orig_argv specs.

Is your machine low in memory?

[mtelka]

$ TESTFROZEN=on ./Programs/_testembed test_init_use_frozen_modules
Fatal Python error: memory allocation failed
Python runtime state: core initialized

Current thread 0x0000000000000001 (most recent call first):
  <no Python frame>
$

The machine does have 64 GB of memory with 52 GB of free memory.

Please note the error message is misleading. The problem is that the xoptions and orig_argv are missing.

[vstinner]

The problem is that the xoptions and orig_argv are missing.

Would you mind to elaborate? _PyConfig_AsDict() is causing this error?

[vstinner]

The problem comes from check_use_frozen_modules() in Programs/_testembed.c: swprintf(optval, 100, L"frozen_modules=%s", "on") creates a string which looks like random characters:

(gdb) p optval
$2 = L"\x7a6f7266\x6d5f6e65\x6c75646f\x6f3d7365n", '\000' <repeats 94 times>

[vstinner]

Example in C:

#include <stdio.h>
#include <wchar.h>
#include <string.h>

int main()
{
    wchar_t optval[100];
    memset(optval, 0, sizeof(optval));
    swprintf(optval, 100, L"frozen_modules=%s", "on");
    printf("optval >");
    for (wchar_t *s = optval; *s != 0; s++) {
        printf("\\x%08x", *s);
    }
    printf("<\n");
    return 0;
}

Output:

$ gcc x.c -o x && ./x
optval >\x7a6f7266\x6d5f6e65\x6c75646f\x6f3d7365\x0000006e<