This project is archived and is in readonly mode.
Return number of rows for copy operations
As asked by Ryan Kelly on the ML:
I'm wondering if it's possible to get the number of rows copied
using copy_from? I would've thought the .rowcount attribute of the cursor
would have this value, but that doesn't seem to be the case.
Comments and changes to this ticket
If I remember correctly the libpq docs don't say if it is possible to
retrieve the number of inserted columns and we're not sending data
line-by-line but as chunks so counting the lines isn't possible. Maybe
(a big maybe) the final PGresult contains that but I can't find this documented anywhere. We'll need to instrument psycopg to print PGresult
fields and do a bit of guesswork.
I'm no expert, but isn't it described here?:
This function returns a string containing the number of rows affected by the SQL statement that generated the PGresult. This function can only be used following the execution of a SELECT, CREATE TABLE AS, INSERT, UPDATE, DELETE, MOVE, FETCH, or COPY statement, ...
On a related note, I have been using psycopg2 with Amazon Redshift and recently asked a similar question about getting the rowcount for COPY statements executed via cursor.execute. I've attached a PDF of the linked AWS forum page in case you can't access it.
Amazon claims that Redshift correctly reports rowcount to the calling interface and that psycopg2 doesn't seem to pick it up.
How can I validate Amazon's claim?
- State changed from open to resolved
- Milestone set to psycopg2 2.5.3
As Rui pointed out the number of rows is returned by PQcmdTuples. We weren't calling this function on copy results but only after normal opreations.
Fixed in 2.5.3. Nicholas, try checking out https://github.com/psycopg/psycopg2/tree/maint_2_5 and see if it works as expected, thank you.
It appears that we are still seeing the -1 rowcount on COPYs in psycopg 2.5.3 when testing against Redshift 1.0.789. This may well be an issue on Redshift's side, and I think it's beyond psycopg2's mandate to work cleanly with forks of PostgreSQL. (I think Bilal is also testing against Redshift.)
FYI: Redshift is a distant fork of PostgreSQL 8 with a column-oriented storage engine, hosted as a service by Amazon. Its primary use case is for massive analytics workloads. Amazon recommends using PostgreSQL drivers to interact with the database.
Anyway, I am following up with Amazon support here to get to the bottom of this. They claim it should work.
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile »
<b>WARNING:</b> the informations in this tracker are archived. Please submit new tickets or comments to <a href="https://github.com/psycopg/psycopg2/issues">the new tracker</a>.
Psycopg is the most used PostgreSQL adapter for the Python programming language. At the core it fully implements the Python DB API 2.0 specifications. Several extensions allow access to many of the features offered by PostgreSQL.