Validate the output, always

Text that looks like data is still text

You asked for JSON. You got JSON, in your ten manual tests. On the eleventh call you get this:

Sure! Here is the analysis you asked for:

{"sentiment": "positive", "score": 8}

Let me know if you would like me to adjust anything.

json.loads on that raises JSONDecodeError on the very first character. The model did nothing wrong by any definition it was given: it produced a helpful message that contains the object. Your code assumed the string is the object.

This is the same class of problem as parsing a user-uploaded CSV, and it deserves the same treatment. The boundary between "text from outside" and "data my code relies on" needs one function, and everything past that function is trusted because it was checked.

The validation boundary

One function. It takes raw text and returns either the value your code wants, or a clear signal that it did not get one.

import json


def parse_review(raw):
    start = raw.find("{")
    end = raw.rfind("}")
    if start == -1 or end == -1 or end < start:
        return None
    try:
        data = json.loads(raw[start : end + 1])
    except ValueError:
        return None
    if not isinstance(data, dict):
        return None
    sentiment = data.get("sentiment")
    score = data.get("score")
    if not isinstance(sentiment, str) or not sentiment:
        return None
    if isinstance(score, bool) or not isinstance(score, (int, float)):
        return None
    return {"sentiment": sentiment, "score": score}

Every line is answering a question you would otherwise be answering at 3am.

Locate the object. Take everything from the first { to the last }. Crude, and it handles the overwhelmingly common case of an object wrapped in friendly prose or a code fence.

Parse defensively. json.loads raises on bad input, so the try is how failure becomes a return value instead of an escape.

Check the shape. Valid JSON can be a list, a number, or the string "null". data.get(...) on a list raises AttributeError, so the type of the top-level value is checked before anything is read out of it.

Check every field you will use. Presence, type, and range are three separate questions. A key can be present and None. A score can be a string "8". A rating meant to be 0 to 10 can be 47.

Return one normalized shape. Callers get either the dictionary they expect or None. They never get a half-validated dictionary with two good fields and one surprise.

Deciding what a failure means

Returning None is a decision, not a default. The alternatives are real and the right one depends on what the caller does next.

What is not acceptable is silently substituting a plausible value. A missing score that becomes 0 will be averaged into a dashboard and nobody will ever know the difference between "rated zero" and "never rated". That is the same shape as the swallowed exception from earlier in this level, wearing different clothes.

Ask for a shape that is easy to check

There is a design lever here that is worth more than any parsing cleverness. The narrower the shape you ask for, the cheaper validation is. {"score": 8} is a stronger request than "tell me how positive this is", because you can check the former in three lines and the latter needs a judgment call. A short fixed vocabulary beats free text, and one flat object beats a nested structure whose depth you have to walk.

Interview nuance: if you say "the model returns text, so I would validate it at the boundary and decide what a failure means before I wire it into anything", you have described the same architecture you would use for a webhook, a file upload, or a partner API. Interviewers are checking that you treat the model as a normal untrusted input source, because the candidates who do not are the ones who ship a handler that crashes on the first friendly preamble.