Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

This project is archived and is in readonly mode.

copy_from ValueError "column list to long"

#68

Hi All,

I am using psycopg2 happyly to import data from csv files from other databases into a postgres database for analysis. Today I ran into a ValueError raised by cursor.copy_from with the message "column list too long".
After digging around some in the source I saw in cursor_type.h in _psyco_curs_copy_columns the if statement
if (offset + collen > DEFAULT_COPYBUFF - 2) {

Py_DECREF(col);
Py_DECREF(coliter);
PyErr_SetString(PyExc_ValueError, "column list too long");
return -1;

}

Why is psycopg2 only allowing a maximum number of characters in the column list? This doesn't seem to make much sense to me unless I am missing something. I was told by the people in the #postgresql channel on irc.freenode.net that this is not due to a limitation in postgresql itself. Is that correct?
Is there any way to allow a longer column length somehow?
Cheers,

Dolf.

Reported by Dolf Andringa · August 30th, 2011 @ 04:04 PM

State: resolved
Milestone: none
Assigned to: nobody

Activity

  1. Daniele Varrazzo
    Daniele Varrazzo

    Does it mean the sum of the length of the column names is more than 8k chars? How long is that?

    Probably the author thought it would have been a safe limit... We should use a dynamically allocated buffer.

    September 8th, 2011 @ 01:51 PM

  2. Daniele Varrazzo
    Daniele Varrazzo
    • State changed from new to resolved
    • Tag set to copy

    Fixed in my devel. To be released in 2.4.3.

    September 12th, 2011 @ 01:22 AM