Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

There was a problem

You must be a member of this account.

This project is archived and is in readonly mode.

Memory not released immediately with named cursors

#117

Submitted by: flox

When fetching a named cursor with large records > 16340 bytes, the memory is not released.

Reported by Psycopg website · August 10th, 2012 @ 11:18 PM

State: invalid
Milestone: none
Assigned to: nobody

Activity

  1. flox
    flox
    • Assigned user set to Daniele Varrazzo

    This is the test file attached.

    And the output:

    $ python testmem.py psycopg2 version 2.4.5 (dt dec pq3 ext)
    Record size: 1024 B; Memory used: 8.8 MB
    Record size: 16339 B; Memory used: 9.0 MB
    Record size: 16340 B; Memory used: 205.3 MB
    $

    August 10th, 2012 @ 11:24 PM

  2. flox
  3. flox
    flox
    • Assigned user cleared.

    August 10th, 2012 @ 11:26 PM

  4. flox
    flox

    I changed the test_fetch() function to release the memory.
    For some reason, "del big_var" is not enough. A "gc.collect()" call is required.

    def test_fetch(cursor):
        big_var = range(1 << 23)
        del big_var
        gc.collect()
        cursor.fetchone()
    

    $ python --version Python 2.6.6
    $ python testmem.py psycopg2 version 2.4.5 (dt dec pq3 ext)
    Record size: 1024 B; Memory used: 8.8 MB
    Record size: 16339 B; Memory used: 8.8 MB
    Record size: 16340 B; Memory used: 8.8 MB
    $

    August 10th, 2012 @ 11:57 PM

  5. Daniele Varrazzo
    Daniele Varrazzo

    So it's no link in psycopg, just the way the python gc works, right?

    August 14th, 2012 @ 09:18 PM

  6. flox
    flox

    I am running this on Debian Squeeze.
    It's possibly related to the way the memory is allocated then released.

    I don't have other test case than the "testmem.py" above.
    Could be closed, since I managed to deal with named cursors without any other memory issue.

    August 18th, 2012 @ 08:48 AM

  7. flox
    flox
    • Title changed from Memory leak with named cursors to Memory not released immediately with named cursors

    August 18th, 2012 @ 08:49 AM

  8. Daniele Varrazzo
    Daniele Varrazzo
    • State changed from new to invalid

    kthxbye!

    August 28th, 2012 @ 08:14 PM