Skip to content

A non-ASCII comment loses its parentheses when a header is folded with an encoded word #162125

Open

Description

@FedeLoch

Bug report

Hello,

While fuzzing the email package of Python 3.14.7 with Termo, a fuzzer I'm developing, I checked that a header value survives as_string() and parsing back with policy.default. Structured headers that contain a comment with non-ASCII text don't survive it. The comment is written as an RFC 2047 encoded word without its parentheses, so on reading back it is no longer a comment. In an address header this changes the addresses.

import email
from email import policy
from email.message import EmailMessage

msg = EmailMessage()
msg["From"] = "[email protected] (José)"
out = msg.as_string()
print(out.splitlines()[0])
# From: [email protected] =?utf-8?q?Jos=C3=A9?=
print(str(email.message_from_string(out, policy=policy.default)["From"]))
# [email protected], <>        <- one sender became two addresses, the second empty

msg = EmailMessage()
msg["To"] = "Ana (ñ) <[email protected]>"
out = msg.as_string()
print(out.splitlines()[0])
# To: Ana =?utf-8?q?=C3=B1?= <[email protected]>
print(email.message_from_string(out, policy=policy.default)["To"].addresses[0].display_name)
# Ana ñ                    <- the comment is now part of the display name

msg = EmailMessage()
msg["Content-Disposition"] = "attachment (é)"
print(msg.as_string().splitlines()[0])
# Content-Disposition: attachment =?utf-8?q?=C3=A9?=

With an ASCII comment the parentheses are kept (From: [email protected] (Jose)), and RFC 2047 §5 (2) allows an encoded word inside a comment. So the expected output is, for example, To: Ana (=?utf-8?q?=C3=B1?=) <[email protected]>.

Versions:

  • 3.14.7: all three as above.
  • 3.15.2: the To and Content-Disposition cases are the same. The From case now raises HeaderWriteError: Non-ASCII domain 'example.com (José)' is invalid under current policy setting (utf8=False), and "Jürgen <[email protected]> (café)" raises Non-ASCII cfws ' (café)' is invalid. The comment is not part of the domain, so this error is also misleading.
  • main (5a22a62): I haven't run it, but the code path below is unchanged.

The cause appears to be in _refold_with_ew (Lib/email/_header_value_parser.py). When a part needs encoding because it has non-ASCII characters, a non-terminal part is unpacked into its children (line 2973, "It's not a Terminal, do each piece individually"). For a Comment, the children don't include the ( and ), which only Comment.__str__ adds. The parenthesis handling added for gh-143935 (line 3036, if part.token_type == 'comment':) is only on the path for parts too long to fit on a line, not on this one. Wrapping the children the same way there (an opening ( terminal, the children, then a closing ), with the encoded words kept inside) should fix it.

I can turn these into test cases for test__header_value_parser / test_headerregistry if that helps.

If you'd like me to keep fuzzing the email package (other headers and policies, or re-checking after a fix), feel free to reach out. I'd be glad to keep working on it with you.

Thanks!

PS: Sorry if the issue looks like a template, it's, I am reporting plenty of issues to different projects and I am a simple human :)

CPython versions tested on:

3.14

Operating systems tested on:

macOS

Activity

  1. added
    stdlibStandard Library Python modules in the Lib/ directory
    on Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    stdlibStandard Library Python modules in the Lib/ directorytopic-emailtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions