This project is archived and is in readonly mode.
TypeError: can't escape unicode to binary: writing unicode into bytea
Reported by tamagotchi | April 23rd, 2011 @ 01:51 AM
Hi,
I have been using psycopg2 on PostgreSQL 8.3, 8.4 on Linux and
Windows without any issues.
Now I started working on a Mac OSX machine and installed the latest
PostgreSQL 9 + psycopgs2 2.4 (dt dec pq3 ext) + Python 2.7.1
And I noticed that my application (which is using SQLAlchemy) can't
write into bytea fields anymore.
I'm getting:
TypeError: can't escape unicode to binary
I boiled it down to this simple code block:
b = psycopg2.Binary(u"XYZABCTEst")
c = psycopg2.connect("dbname=acttest")
cur = c.cursor()
cur.execute("INSERT INTO users (userid, preferences) VALUES
('userid', %(prefs)s) RETURNING users.id", {'prefs': b})
If I use a regular string
b = psycopg2.Binary("XYZABCTEst")
everything works fine.
Binary is supposed to work with unicode objects, isn't it?
Thanks,
-T
Comments and changes to this ticket
-
tamagotchi April 25th, 2011 @ 07:30 PM
After poking around some more, I figure out that the above code doesn't work with other versions of Postgres and psycopg as well.
The reason why my application broke was the upgrade from cherrypy 3.1.x to 3.2 where the data in the request is now passed as unicode objects rather than strings.
The question for me remains still, is it reasonable to expect that byteas/Binaries should work with unicode?
If not, why not? (in which case this bug report would be invalid)Thanks,
-T
-
Daniele Varrazzo May 3rd, 2011 @ 11:02 PM
- Tag set to “binary, unicode”
- State changed from “new” to “invalid”
No, it's not reasonable for unicode to be interpreted as a binary string. This is the entire point for which there are separate unicode and string object (or str and bytes in Py3).
Internal unicode representation could be 2 or 4 bytes per codepoint. It can be converted to binary in different ways according to different encoding.
If you want to pass unicode to a binary you will have to convert it to byte string specifying your own encoding. For me it's a bug in cherrypy 3.2.
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป
<b>WARNING:</b> the informations in this tracker are archived. Please submit new tickets or comments to <a href="https://github.com/psycopg/psycopg2/issues">the new tracker</a>.
<br/>
Psycopg is the most used PostgreSQL adapter for the Python programming language. At the core it fully implements the Python DB API 2.0 specifications. Several extensions allow access to many of the features offered by PostgreSQL.
Daniele Varrazzo
tamagotchi