Skip to content

Vary compiler flags in fuzz_pycompile #145236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alex merged 2 commits into from
Mar 9, 2026
Merged

Vary compiler flags in fuzz_pycompile #145236

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
702aa3f
Vary compiler flags in fuzz_pycompile
StanFromIreland Feb 25, 2026
2418cbd
Drop `PyCF_SOURCE_IS_UTF8`
StanFromIreland Mar 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 31 additions & 17 deletions Modules/_xxtestfuzz/fuzzer.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -516,8 +516,8 @@ static int fuzz_pycompile(const char* data, size_t size) {
return 0;
}

// Need 2 bytes for parameter selection
if (size < 2) {
// Need 3 bytes for parameter selection
if (size < 3) {
return 0;
}

Expand All @@ -529,25 +529,39 @@ static int fuzz_pycompile(const char* data, size_t size) {
unsigned char optimize_idx = (unsigned char) data[1];
int optimize = optimize_vals[optimize_idx % NUM_OPTIMIZE_VALS];

// Use third byte to determine compiler flags to use.
unsigned char flags_byte = (unsigned char) data[2];
PyCompilerFlags flags = _PyCompilerFlags_INIT;
if (flags_byte & 0x01) {
Copy link
Copy Markdown
Member

@alex alex Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure all of these are useful/make sense.

I don't think PyCF_SOURCE_IS_UTF8 does anything, for example. So I'd drop that and PyCF_ONLY_AST,

Copy link
Copy Markdown
Member Author

@StanFromIreland StanFromIreland Mar 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can drop PyCF_SOURCE_IS_UTF8, but PyCF_ONLY_AST does change quite a lot.

flags.cf_flags |= PyCF_DONT_IMPLY_DEDENT;
}
if (flags_byte & 0x02) {
flags.cf_flags |= PyCF_ONLY_AST;
}
if (flags_byte & 0x04) {
flags.cf_flags |= PyCF_IGNORE_COOKIE;
}
if (flags_byte & 0x08) {
flags.cf_flags |= PyCF_TYPE_COMMENTS;
}
if (flags_byte & 0x10) {
flags.cf_flags |= PyCF_ALLOW_TOP_LEVEL_AWAIT;
}
if (flags_byte & 0x20) {
flags.cf_flags |= PyCF_ALLOW_INCOMPLETE_INPUT;
}
if (flags_byte & 0x40) {
flags.cf_flags |= PyCF_OPTIMIZED_AST;
}

char pycompile_scratch[MAX_PYCOMPILE_TEST_SIZE];

// Create a NUL-terminated C string from the remaining input
memcpy(pycompile_scratch, data + 2, size - 2);
memcpy(pycompile_scratch, data + 3, size - 3);
// Put a NUL terminator just after the copied data. (Space was reserved already.)
pycompile_scratch[size - 2] = '\0';

// XXX: instead of always using NULL for the `flags` value to
// `Py_CompileStringExFlags`, there are many flags that conditionally
// change parser behavior:
//
// #define PyCF_TYPE_COMMENTS 0x1000
// #define PyCF_ALLOW_TOP_LEVEL_AWAIT 0x2000
// #define PyCF_ONLY_AST 0x0400
//
// It would be good to test various combinations of these, too.
PyCompilerFlags *flags = NULL;

PyObject *result = Py_CompileStringExFlags(pycompile_scratch, "<fuzz input>", start, flags, optimize);
pycompile_scratch[size - 3] = '\0';

PyObject *result = Py_CompileStringExFlags(pycompile_scratch, "<fuzz input>", start, &flags, optimize);
if (result == NULL) {
/* Compilation failed, most likely from a syntax error. If it was a
SystemError we abort. There's no non-bug reason to raise a
Expand Down
Loading