This project is archived and is in readonly mode.

#172 ✓resolved
Psycopg website

row_to_json data gets interpreted

Reported by Psycopg website | July 19th, 2013 @ 04:28 PM

Submitted by: eric casteleijn

Recent versions of Postgres allow returning json from queries. Psycop since version 2.5.0 decided to call json.loads on the query results for some mysterious reason. This is not psycops responsibiblity: if I query the database, and ask for json, I expect json back, not json interpreted into python structures. (One of the reasons to use this postgres features is to return json responses directly from the database to a webserver. Any advantages of doing that are lost if you need to call json.dumps again after psycop calls json.loads.)

Comments and changes to this ticket

  • Federico Di Gregorio

    Federico Di Gregorio July 19th, 2013 @ 04:36 PM

    I have to agree. IMHO, the json adapter should exist but its registration should not be automatic.

  • Daniele Varrazzo

    Daniele Varrazzo July 19th, 2013 @ 05:35 PM

    Mmm, i think you are right, the choice of adapting automatically json is questionable.

    Eric, while we decide how to fix the problem you can disable the automatic adaptation using:

    import psycopg2.extensions as ext
    ext.string_types.pop(ext.JSON.values[0], None)
    ext.string_types.pop(ext.JSONARRAY.values[0], None)

    It's enough to do it only once and is idempotent anyway. If you guard it with an if hasattr(ext, 'JSON') it would work with any psycopg release.

  • Daniele Varrazzo

    Daniele Varrazzo July 19th, 2013 @ 06:49 PM

    I've written to the mailing list my ideas about the issue. Feel free to take part to the discussion.

    The automatic cast of json was introduced in a major release (2.5): although the choice can be questioned (not that anybody did) it was done in a legitimate moment. Dropping automatic casting in 2.5.x would be a bad reaction; the right moment to think about a different behaviour is for 2.6.

    You have plenty of options to work around the issue:

    • use psycopg < 2.4.x, which is still being maintained (pip install "psycopg2<2.5" will give you the most up-to-date)
    • cast your json as text in the query
    • use the above snippet to unregister the json typecaster.

    Sorry for the inconvenience.

  • Daniele Varrazzo

    Daniele Varrazzo September 20th, 2013 @ 11:31 AM

    • State changed from “new” to “resolved”

    After a chat on the ML, the idea is to leave the json adapter as it is.

    Easy workarounds are to cast json to text and to register the text/unicode adapter on the JSON oid.

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

<b>WARNING:</b> the informations in this tracker are archived. Please submit new tickets or comments to <a href="">the new tracker</a>.
Psycopg is the most used PostgreSQL adapter for the Python programming language. At the core it fully implements the Python DB API 2.0 specifications. Several extensions allow access to many of the features offered by PostgreSQL.

Shared Ticket Bins