Skip to content

Missing content in return value of repr() called on UTF-16-encoded bytes object #154827

Description

@alovicious72

Bug report

Bug description:

When repr() is called on a bytes instance containing UTF-16-encoded bytes, the return value can erroneously omit some of the content of the bytes object. For example, the series of statements provided under "Steps to reproduce" show that a call to repr() omits one of the bytes required to encode the unicode character used in that example when that character is encoded using 'utf-16', 'utf-16-be' or 'utf-16-le'.

I observed this behavior when using Python v3.14.4. Additional information concerning the hardware and Windows environment on the affected system is below:

OS Name Microsoft Windows 11 Home
Version 10.0.26200 Build 26200
Other OS Description Not Available
OS Manufacturer Microsoft Corporation
Processor AMD Ryzen 7 7730U with Radeon Graphics, 2000 Mhz, 8 Core(s), 16 Logical Processor(s)
Installed Physical Memory (RAM) 16.0 GB
Total Physical Memory 15.4 GB
Available Physical Memory 7.66 GB
Total Virtual Memory 16.6 GB
Available Virtual Memory 5.37 GB
Page File Space 1.19 GB

Steps to reproduce:

The example shown below will easily reproduce the bug. The output of all three "print()" statements directly performed on the bytes object (b_utf16) is missing the byte with value 0x3c.

u = '\U0001F3AD'

b_utf16 = u.encode(encoding="utf-16")
b_utf16_be = u.encode(encoding="utf-16-be")
b_utf16_le = u.encode(encoding="utf-16-le")

print(*map(lambda x: hex(x), b_utf16)) # 0xff 0xfe 0x3c 0xd8 0xad 0xdf
print(b_utf16) # 0xd8 0x3c 0xdf 0xad
    
print(*map(lambda x: hex(x), b_utf16_be)) # 0xd8 0x3c 0xdf 0xad
print(b_utf16_be) # b'\xd8<\xdf\xad'

print(*map(lambda x: hex(x), b_utf16_le)) # 0x3c 0xd8 0xad 0xdf
print(b_utf16_le) # b'<\xd8\xad\xdf'

CPython versions tested on:

3.14

Operating systems tested on:

Windows

Metadata

Metadata

Assignees

No one assigned

    Labels

    type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions