When JavaScript and Python Disagree About Text Offsets

Disclosure: This article was generated with AI and was not reviewed by Martin Yovchevski before publication. Martin is a co-founder of Unplag.bg.

Imagine a teacher opening a report and seeing the wrong letter highlighted. No student did anything wrong: a browser and a Python service may simply be counting the same text in different units.

JavaScript string offsets use UTF-16 code units. Python indexes a string by Unicode code points. For many Bulgarian letters those positions coincide, but an emoji outside the Basic Multilingual Plane uses two UTF-16 code units and one Python code point.

For the fictional string А😀Б, JavaScript's indexOf('Б') returns 3; Python's index('Б') returns 2. Applying a JavaScript offset directly as a Python slice index points at the wrong boundary.

A small boundary converter

The function below converts a UTF-16 code-unit boundary into a Python index for the same, unchanged string. It rejects offsets inside a surrogate pair and outside the text.

def utf16_to_py_index(text: str, offset: int) -> int: if isinstance(offset, bool) or not isinstance(offset, int) or offset < 0: raise ValueError("offset must be a non-negative integer") units = 0 for index, char in enumerate(text): if units == offset: return index units += 2 if ord(char) > 0xFFFF else 1 if units > offset: raise ValueError("offset falls inside a surrogate pair") if units == offset: return len(text) raise ValueError("offset is outside the string") sample = "А😀Б" start = utf16_to_py_index(sample, 3) end = utf16_to_py_index(sample, 4) print(sample[start:end]) # Б 

If an upstream offset is 2 in that example, it falls inside the emoji's two-unit representation. The function raises an error rather than silently guessing. A zero offset and the boundary at the end of the text are valid.

Limits that matter

This conversion is for boundaries in the same sequence of Unicode code points. It does not repair offsets after text is edited or normalised. It also does not count user-perceived characters (grapheme clusters), which can contain multiple code points. Keep the original text and offset convention together, and test the entire browser-to-server path before using this in a real application.

Use fictional data in tests. In a classroom review, a highlight is evidence to inspect in context; it is not proof of misconduct. AI-writing indicators can be wrong, too, and the educator remains responsible for the decision.

References: MDN on JavaScript string length and Python's text sequence documentation.

Teachers, lecturers and education developers who want to discuss text-review workflows can write to office@unplag.info.