This project is archived and is in readonly mode.
Using with gevent causes segfault
Reported by Vinay Sajip | October 2nd, 2013 @ 03:41 PM
I have an application using gevent 0.13.8 + psycopg2 2.5.1 which segfaults when an attempt is made to 'green' psycopg using a slight variant of Daniele Varrazzo's code provided by Denis Bilenko here.
A snippet from my code:
if __name__ == '__main__':
from gevent import monkey
from gevent.pywsgi import WSGIServer
from psycopg_green import make_psycopg_green
monkey.patch_all()
#make_psycopg_green() # uncommenting this line -> segfault
...
app = WebApp(get_session_factory())
WSGIServer(('', 8000), app, log=None).serve_forever()
The segfault occurs in pyscopg2 C code. When the
make_psycopg_green() call is commented out, no
segfault occurs and operation is as expected, though of course
psycopg2 calls are synchronous.
The failure only occurs when I am invoking multiple web requests from concurrent clients, which cause multiple greenlets to be spawned to run long-running jobs which invoke psycopg2 through the SQLAlchemy ORM.
Stack trace from core dump using gdb (in case it's
any use):
Core was generated by `python gwebapp.py'.
Program terminated with signal 11, Segmentation fault.
Core was generated by `python gwebapp.py'.
Program terminated with signal 11, Segmentation fault.
#0 0x00000000004606a9 in PyDict_SetItem ()
(gdb) where
#0 0x00000000004606a9 in PyDict_SetItem ()
#1 0x0000000000461a48 in PyDict_SetItemString ()
#2 0x00007fc172da98ba in psyco_error_reduce (self=0x25fb1b8)
at psycopg/error_type.c:166
#3 0x000000000041ad4a in PyObject_Call ()
#4 0x00000000004b5ed6 in PyEval_CallObjectWithKeywords ()
#5 0x000000000047c457 in ?? ()
#6 0x000000000041ad4a in PyObject_Call ()
#7 0x0000000000526f8e in ?? ()
#8 0x00000000005291d6 in ?? ()
#9 0x00000000005276c3 in ?? ()
#10 0x00000000005270f3 in ?? ()
#11 0x00000000005291d6 in ?? ()
#12 0x00000000005276c3 in ?? ()
#13 0x00000000005277c4 in ?? ()
#14 0x00000000005288d3 in ?? ()
#15 0x00000000005289a5 in ?? ()
#16 0x00000000004b66b9 in PyEval_EvalFrameEx ()
#17 0x00000000004b6f57 in PyEval_EvalFrameEx ()
#18 0x00000000004b6f57 in PyEval_EvalFrameEx ()
#19 0x00000000004bcdbd in PyEval_EvalCodeEx ()
#20 0x000000000044914f in ?? ()
#21 0x000000000041ad4a in PyObject_Call ()
#22 0x000000000043092e in ?? ()
#23 0x000000000041ad4a in PyObject_Call ()
#24 0x00000000004b6d7e in PyEval_EvalFrameEx ()
#25 0x00000000004b6f57 in PyEval_EvalFrameEx ()
#26 0x00000000004b6f57 in PyEval_EvalFrameEx ()
#27 0x00000000004b6f57 in PyEval_EvalFrameEx ()
#28 0x00000000004bcdbd in PyEval_EvalCodeEx ()
#29 0x0000000000449308 in ?? ()
#30 0x000000000041ad4a in PyObject_Call ()
#31 0x00000000004b8498 in PyEval_EvalFrameEx ()
#32 0x00000000004bcdbd in PyEval_EvalCodeEx ()
#33 0x0000000000449308 in ?? ()
#34 0x000000000041ad4a in PyObject_Call ()
#35 0x00000000004b8498 in PyEval_EvalFrameEx ()
#36 0x00000000004bcdbd in PyEval_EvalCodeEx ()
#37 0x00000000004b6c34 in PyEval_EvalFrameEx ()
#38 0x00000000004bd3ee in PyEval_EvalCodeEx ()
#39 0x0000000000449308 in ?? ()
#40 0x000000000041ad4a in PyObject_Call ()
#41 0x00000000004b8498 in PyEval_EvalFrameEx ()
#42 0x00000000004bd3ee in PyEval_EvalCodeEx ()
#43 0x0000000000449308 in ?? ()
#44 0x000000000041ad4a in PyObject_Call ()
#45 0x00000000004b8498 in PyEval_EvalFrameEx ()
#46 0x00000000004bcdbd in PyEval_EvalCodeEx ()
#47 0x00000000004b6c34 in PyEval_EvalFrameEx ()
#48 0x00000000004bcdbd in PyEval_EvalCodeEx ()
#49 0x00000000004b6c34 in PyEval_EvalFrameEx ()
#50 0x00000000004bcdbd in PyEval_EvalCodeEx ()
#51 0x000000000044914f in ?? ()
#52 0x000000000041ad4a in PyObject_Call ()
#53 0x000000000043092e in ?? ()
#54 0x000000000041ad4a in PyObject_Call ()
#55 0x0000000000480716 in ?? ()
#56 0x000000000041ad4a in PyObject_Call ()
#57 0x00000000004b6d7e in PyEval_EvalFrameEx ()
#58 0x00000000004b6f57 in PyEval_EvalFrameEx ()
#59 0x00000000004b6f57 in PyEval_EvalFrameEx ()
#60 0x00000000004b6f57 in PyEval_EvalFrameEx ()
#61 0x00000000004b6f57 in PyEval_EvalFrameEx ()
#62 0x00000000004bcdbd in PyEval_EvalCodeEx ()
#63 0x0000000000449308 in ?? ()
#64 0x000000000041ad4a in PyObject_Call ()
#65 0x00000000004b8498 in PyEval_EvalFrameEx ()
#66 0x00000000004bcdbd in PyEval_EvalCodeEx ()
#67 0x0000000000449308 in ?? ()
#68 0x000000000041ad4a in PyObject_Call ()
#69 0x000000000043092e in ?? ()
#70 0x000000000041ad4a in PyObject_Call ()
#71 0x00000000004b5ed6 in PyEval_CallObjectWithKeywords ()
#72 0x00007fc17329ea56 in g_initialstub (mark=<optimized out>)
at greenlet.c:804
#73 0x00007fc17329e398 in g_switch (target=0x1875f50, args=0x7fc175e66050,
kwargs=0x3741130) at greenlet.c:576
#74 0x00007fc17329ef26 in green_switch (self=<optimized out>,
args=<optimized out>, kwargs=<optimized out>) at greenlet.c:1092
#75 0x000000000041ad4a in PyObject_Call ()
#76 0x00007fc17390b3f3 in __pyx_f_6gevent_4core___simple_handler (
__pyx_v_fd=<optimized out>, __pyx_v_evtype=<optimized out>,
__pyx_v_arg=0x3eb1488) at gevent/core.c:5130
#77 0x00007fc1736b8d3c in event_base_loop () from /usr/lib/libevent-2.0.so.5
#78 0x00007fc1738fe105 in __pyx_pf_6gevent_4core_2dispatch (
__pyx_self=<optimized out>) at gevent/core.c:6083
#79 __pyx_pw_6gevent_4core_3dispatch (__pyx_self=<optimized out>,
unused=<optimized out>) at gevent/core.c:6036
#80 0x00000000004b7281 in PyEval_EvalFrameEx ()
#81 0x00000000004bcdbd in PyEval_EvalCodeEx ()
#82 0x000000000044914f in ?? ()
#83 0x000000000041ad4a in PyObject_Call ()
#84 0x000000000043092e in ?? ()
#85 0x000000000041ad4a in PyObject_Call ()
#86 0x00000000004b5ed6 in PyEval_CallObjectWithKeywords ()
#87 0x00007fc17329ea56 in g_initialstub (mark=<optimized out>)
at greenlet.c:804
#88 0x00007fc17329e398 in g_switch (target=0x25f7050, args=0x7fc175e66050,
kwargs=0x0) at greenlet.c:576
#89 0x00007fc17329ef26 in green_switch (self=<optimized out>,
args=<optimized out>, kwargs=<optimized out>) at greenlet.c:1092
#90 0x000000000041ad4a in PyObject_Call ()
#91 0x00000000004b5ed6 in PyEval_CallObjectWithKeywords ()
#92 0x00000000004399a7 in ?? ()
#93 0x000000000041ad4a in PyObject_Call ()
#94 0x00000000004b6d7e in PyEval_EvalFrameEx ()
#95 0x00000000004b6f57 in PyEval_EvalFrameEx ()
#96 0x00000000004bcdbd in PyEval_EvalCodeEx ()
#97 0x00000000004b6c34 in PyEval_EvalFrameEx ()
#98 0x00000000004bcdbd in PyEval_EvalCodeEx ()
#99 0x00000000004b6c34 in PyEval_EvalFrameEx ()
#100 0x00000000004bcdbd in PyEval_EvalCodeEx ()
#101 0x00000000004bd892 in PyEval_EvalCode ()
#102 0x00000000004dcd82 in ?? ()
#103 0x00000000004dd974 in PyRun_FileExFlags ()
tagged:rel-2.2.1
tagged:os-linux
tagged:gevent
Comments and changes to this ticket
-
Daniele Varrazzo October 2nd, 2013 @ 04:06 PM
- State changed from “new” to “resolved”
It's in the exception pickling: I think it's caused by bug #170
Core was generated by `python gwebapp.py'. Program terminated with signal 11, Segmentation fault. [#0](/projects/62710/tickets/0 "Ticket #0") 0x00000000004606a9 in PyDict_SetItem () (gdb) where [#0](/projects/62710/tickets/0 "Ticket #0") 0x00000000004606a9 in PyDict_SetItem () [#1](/projects/62710/tickets/1 "Ticket #1") 0x0000000000461a48 in PyDict_SetItemString () [#2](/projects/62710/tickets/2 "Ticket #2") 0x00007fc172da98ba in psyco_error_reduce (self=0x25fb1b8) at psycopg/error_type.c:166It was fixed but not released yet. Do you want to test with that patch on?
I think we'll need a release soonish.
-
Vinay Sajip October 2nd, 2013 @ 05:08 PM
I can confirm that with your code from the master branch of https://github.com/dvarrazzo/psycopg, the segfault goes away. Thanks!
Create your profile
Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป
<b>WARNING:</b> the informations in this tracker are archived. Please submit new tickets or comments to <a href="https://github.com/psycopg/psycopg2/issues">the new tracker</a>.
<br/>
Psycopg is the most used PostgreSQL adapter for the Python programming language. At the core it fully implements the Python DB API 2.0 specifications. Several extensions allow access to many of the features offered by PostgreSQL.
Daniele Varrazzo
Federico Di Gregorio
Vinay Sajip