Segmentation Fault on Centos 7 when Connection Pooling Enable

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Segmentation Fault on Centos 7 when Connection Pooling Enable

Davide Pagnin
Hi,

we are trying to port our application (php web application) from Windows world to linux.
Our database is on IBM i system machine, which is a DB2 flavor.

We realized that by using Connection Pooling (on Windows) we avoid connection setup,
in most of the cases, thus the application gain performances (on simple pages, we cut
connection time from 0,25-0,35 sec to 0,02-0,03 sec), this anyway is not the reason of
the mail.

As I said, when we enabled connection pooling on the linux centos 7.2 we are using
for development and testing, we faced core dumps of the httpd server.

At the moment we are using stock unixodbc rpm version 2.3.1 that came from centos (thus redhat)
but I've reproduced the core dumps also on 2.3.4 recompiled on our development system.

Using valgrind on the httpd process gives this traces of invalid writes:

==23906== Invalid write of size 8
==23906==    at 0x23AFD0ED: UnknownInlinedFun (stdio2.h:33)
==23906==    by 0x23AFD0ED: __SQLFreeHandle (SQLFreeHandle.c:329)
==23906==    by 0x267F457B: odbc_handle_closer (odbc_driver.c:134)
==23906==    by 0x23D53DD5: dbh_free (pdo_dbh.c:1523)
==23906==    by 0x1299308F: zend_objects_store_free_object_storage (zend_objects_API.c:99)
==23906==    by 0x129492B2: shutdown_executor (zend_execute_API.c:357)
==23906==    by 0x12958137: zend_deactivate (zend.c:977)
==23906==    by 0x128F74E0: php_request_shutdown (main.c:1833)
==23906==    by 0x129E316E: php_apache_request_dtor (sapi_apache2.c:518)
==23906==    by 0x129E316E: php_handler (sapi_apache2.c:690)
==23906==    by 0x14728F: ap_run_handler (config.c:169)
==23906==    by 0x1477D8: ap_invoke_handler (config.c:439)
==23906==    by 0x15BBC9: ap_process_async_request (http_request.c:317)
==23906==    by 0x15BEA3: ap_process_request (http_request.c:363)
==23906==  Address 0x28885060 is 16 bytes inside a block of size 1,512 free'd
==23906==    at 0x4C2ACDD: free (vg_replace_malloc.c:530)
==23906==    by 0x23B2001E: __release_env (__handles.c:567)
==23906==    by 0x23AFD2DC: __SQLFreeHandle (SQLFreeHandle.c:247)
==23906==    by 0x267F4590: odbc_handle_closer (odbc_driver.c:137)
==23906==    by 0x23D53DD5: dbh_free (pdo_dbh.c:1523)
==23906==    by 0x1299308F: zend_objects_store_free_object_storage (zend_objects_API.c:99)
==23906==    by 0x129492B2: shutdown_executor (zend_execute_API.c:357)
==23906==    by 0x12958137: zend_deactivate (zend.c:977)
==23906==    by 0x128F74E0: php_request_shutdown (main.c:1833)
==23906==    by 0x129E316E: php_apache_request_dtor (sapi_apache2.c:518)
==23906==    by 0x129E316E: php_handler (sapi_apache2.c:690)
==23906==    by 0x14728F: ap_run_handler (config.c:169)
==23906==    by 0x1477D8: ap_invoke_handler (config.c:439)
==23906==  Block was alloc'd at
==23906==    at 0x4C2B975: calloc (vg_replace_malloc.c:711)
==23906==    by 0x23B1FD38: __alloc_env (__handles.c:400)
==23906==    by 0x23AEC3AC: __SQLAllocHandle (SQLAllocHandle.c:310)
==23906==    by 0x267F48D6: pdo_odbc_handle_factory (odbc_driver.c:402)
==23906==    by 0x23D55EBB: zim_PDO_dbh_constructor (pdo_dbh.c:356)
==23906==    by 0x129D5B6A: ZEND_DO_FCALL_SPEC_HANDLER (zend_vm_execute.h:842)
==23906==    by 0x1299722A: execute_ex (zend_vm_execute.h:414)
==23906==    by 0x129E187E: zend_execute (zend_vm_execute.h:458)
==23906==    by 0x12958492: zend_execute_scripts (zend.c:1437)
==23906==    by 0x128F8957: php_execute_script (main.c:2494)
==23906==    by 0x129E324C: php_handler (sapi_apache2.c:678)
==23906==    by 0x14728F: ap_run_handler (config.c:169)

==23906== Invalid write of size 8
==23906==    at 0x23AFD103: UnknownInlinedFun (stdio2.h:33)
==23906==    by 0x23AFD103: __SQLFreeHandle (SQLFreeHandle.c:329)
==23906==    by 0x267F457B: odbc_handle_closer (odbc_driver.c:134)
==23906==    by 0x23D53DD5: dbh_free (pdo_dbh.c:1523)
==23906==    by 0x1299308F: zend_objects_store_free_object_storage (zend_objects_API.c:99)
==23906==    by 0x129492B2: shutdown_executor (zend_execute_API.c:357)
==23906==    by 0x12958137: zend_deactivate (zend.c:977)
==23906==    by 0x128F74E0: php_request_shutdown (main.c:1833)
==23906==    by 0x129E316E: php_apache_request_dtor (sapi_apache2.c:518)
==23906==    by 0x129E316E: php_handler (sapi_apache2.c:690)
==23906==    by 0x14728F: ap_run_handler (config.c:169)
==23906==    by 0x1477D8: ap_invoke_handler (config.c:439)
==23906==    by 0x15BBC9: ap_process_async_request (http_request.c:317)
==23906==    by 0x15BEA3: ap_process_request (http_request.c:363)
==23906==  Address 0x28885068 is 24 bytes inside a block of size 1,512 free'd
==23906==    at 0x4C2ACDD: free (vg_replace_malloc.c:530)
==23906==    by 0x23B2001E: __release_env (__handles.c:567)
==23906==    by 0x23AFD2DC: __SQLFreeHandle (SQLFreeHandle.c:247)
==23906==    by 0x267F4590: odbc_handle_closer (odbc_driver.c:137)
==23906==    by 0x23D53DD5: dbh_free (pdo_dbh.c:1523)
==23906==    by 0x1299308F: zend_objects_store_free_object_storage (zend_objects_API.c:99)
==23906==    by 0x129492B2: shutdown_executor (zend_execute_API.c:357)
==23906==    by 0x12958137: zend_deactivate (zend.c:977)
==23906==    by 0x128F74E0: php_request_shutdown (main.c:1833)
==23906==    by 0x129E316E: php_apache_request_dtor (sapi_apache2.c:518)
==23906==    by 0x129E316E: php_handler (sapi_apache2.c:690)
==23906==    by 0x14728F: ap_run_handler (config.c:169)
==23906==    by 0x1477D8: ap_invoke_handler (config.c:439)
==23906==  Block was alloc'd at
==23906==    at 0x4C2B975: calloc (vg_replace_malloc.c:711)
==23906==    by 0x23B1FD38: __alloc_env (__handles.c:400)
==23906==    by 0x23AEC3AC: __SQLAllocHandle (SQLAllocHandle.c:310)
==23906==    by 0x267F48D6: pdo_odbc_handle_factory (odbc_driver.c:402)
==23906==    by 0x23D55EBB: zim_PDO_dbh_constructor (pdo_dbh.c:356)
==23906==    by 0x129D5B6A: ZEND_DO_FCALL_SPEC_HANDLER (zend_vm_execute.h:842)
==23906==    by 0x1299722A: execute_ex (zend_vm_execute.h:414)
==23906==    by 0x129E187E: zend_execute (zend_vm_execute.h:458)
==23906==    by 0x12958492: zend_execute_scripts (zend.c:1437)
==23906==    by 0x128F8957: php_execute_script (main.c:2494)
==23906==    by 0x129E324C: php_handler (sapi_apache2.c:678)
==23906==    by 0x14728F: ap_run_handler (config.c:169)

and also some Invalid read

==23906== Invalid read of size 4
==23906==    at 0x23AFD07D: __SQLFreeHandle (SQLFreeHandle.c:312)
==23906==    by 0x267F457B: odbc_handle_closer (odbc_driver.c:134)
==23906==    by 0x23D53DD5: dbh_free (pdo_dbh.c:1523)
==23906==    by 0x1299308F: zend_objects_store_free_object_storage (zend_objects_API.c:99)
==23906==    by 0x129492B2: shutdown_executor (zend_execute_API.c:357)
==23906==    by 0x12958137: zend_deactivate (zend.c:977)
==23906==    by 0x128F74E0: php_request_shutdown (main.c:1833)
==23906==    by 0x129E316E: php_apache_request_dtor (sapi_apache2.c:518)
==23906==    by 0x129E316E: php_handler (sapi_apache2.c:690)
==23906==    by 0x14728F: ap_run_handler (config.c:169)
==23906==    by 0x1477D8: ap_invoke_handler (config.c:439)
==23906==    by 0x15BBC9: ap_process_async_request (http_request.c:317)
==23906==    by 0x15BEA3: ap_process_request (http_request.c:363)
==23906==  Address 0x28885468 is 1,048 bytes inside a block of size 1,512 free'd
==23906==    at 0x4C2ACDD: free (vg_replace_malloc.c:530)
==23906==    by 0x23B2001E: __release_env (__handles.c:567)
==23906==    by 0x23AFD2DC: __SQLFreeHandle (SQLFreeHandle.c:247)
==23906==    by 0x267F4590: odbc_handle_closer (odbc_driver.c:137)
==23906==    by 0x23D53DD5: dbh_free (pdo_dbh.c:1523)
==23906==    by 0x1299308F: zend_objects_store_free_object_storage (zend_objects_API.c:99)
==23906==    by 0x129492B2: shutdown_executor (zend_execute_API.c:357)
==23906==    by 0x12958137: zend_deactivate (zend.c:977)
==23906==    by 0x128F74E0: php_request_shutdown (main.c:1833)
==23906==    by 0x129E316E: php_apache_request_dtor (sapi_apache2.c:518)
==23906==    by 0x129E316E: php_handler (sapi_apache2.c:690)
==23906==    by 0x14728F: ap_run_handler (config.c:169)
==23906==    by 0x1477D8: ap_invoke_handler (config.c:439)
==23906==  Block was alloc'd at
==23906==    at 0x4C2B975: calloc (vg_replace_malloc.c:711)
==23906==    by 0x23B1FD38: __alloc_env (__handles.c:400)
==23906==    by 0x23AEC3AC: __SQLAllocHandle (SQLAllocHandle.c:310)
==23906==    by 0x267F48D6: pdo_odbc_handle_factory (odbc_driver.c:402)
==23906==    by 0x23D55EBB: zim_PDO_dbh_constructor (pdo_dbh.c:356)
==23906==    by 0x129D5B6A: ZEND_DO_FCALL_SPEC_HANDLER (zend_vm_execute.h:842)
==23906==    by 0x1299722A: execute_ex (zend_vm_execute.h:414)
==23906==    by 0x129E187E: zend_execute (zend_vm_execute.h:458)
==23906==    by 0x12958492: zend_execute_scripts (zend.c:1437)
==23906==    by 0x128F8957: php_execute_script (main.c:2494)
==23906==    by 0x129E324C: php_handler (sapi_apache2.c:678)
==23906==    by 0x14728F: ap_run_handler (config.c:169)

==23906== Invalid read of size 4
==23906==    at 0x23B1FB40: __map_type (__connection.c:258)
==23906==    by 0x23AFED25: SQLGetData (SQLGetData.c:443)
==23906==    by 0x267F5455: odbc_stmt_get_col (odbc_stmt.c:664)
==23906==    by 0x23D5B508: fetch_value (pdo_stmt.c:559)
==23906==    by 0x23D5B508: do_fetch.constprop.14 (pdo_stmt.c:1022)
==23906==    by 0x23D5CD22: zim_PDOStatement_fetchAll (pdo_stmt.c:1503)
==23906==    by 0x129D5B6A: ZEND_DO_FCALL_SPEC_HANDLER (zend_vm_execute.h:842)
==23906==    by 0x1299722A: execute_ex (zend_vm_execute.h:414)
==23906==    by 0x129E187E: zend_execute (zend_vm_execute.h:458)
==23906==    by 0x12958492: zend_execute_scripts (zend.c:1437)
==23906==    by 0x128F8957: php_execute_script (main.c:2494)
==23906==    by 0x129E324C: php_handler (sapi_apache2.c:678)
==23906==    by 0x14728F: ap_run_handler (config.c:169)
==23906==  Address 0x28885464 is 1,044 bytes inside a block of size 1,512 free'd
==23906==    at 0x4C2ACDD: free (vg_replace_malloc.c:530)
==23906==    by 0x23B2001E: __release_env (__handles.c:567)
==23906==    by 0x23AFD2DC: __SQLFreeHandle (SQLFreeHandle.c:247)
==23906==    by 0x267F4590: odbc_handle_closer (odbc_driver.c:137)
==23906==    by 0x23D53DD5: dbh_free (pdo_dbh.c:1523)
==23906==    by 0x1299308F: zend_objects_store_free_object_storage (zend_objects_API.c:99)
==23906==    by 0x129492B2: shutdown_executor (zend_execute_API.c:357)
==23906==    by 0x12958137: zend_deactivate (zend.c:977)
==23906==    by 0x128F74E0: php_request_shutdown (main.c:1833)
==23906==    by 0x129E316E: php_apache_request_dtor (sapi_apache2.c:518)
==23906==    by 0x129E316E: php_handler (sapi_apache2.c:690)
==23906==    by 0x14728F: ap_run_handler (config.c:169)
==23906==    by 0x1477D8: ap_invoke_handler (config.c:439)
==23906==  Block was alloc'd at
==23906==    at 0x4C2B975: calloc (vg_replace_malloc.c:711)
==23906==    by 0x23B1FD38: __alloc_env (__handles.c:400)
==23906==    by 0x23AEC3AC: __SQLAllocHandle (SQLAllocHandle.c:310)
==23906==    by 0x267F48D6: pdo_odbc_handle_factory (odbc_driver.c:402)
==23906==    by 0x23D55EBB: zim_PDO_dbh_constructor (pdo_dbh.c:356)
==23906==    by 0x129D5B6A: ZEND_DO_FCALL_SPEC_HANDLER (zend_vm_execute.h:842)
==23906==    by 0x1299722A: execute_ex (zend_vm_execute.h:414)
==23906==    by 0x129E187E: zend_execute (zend_vm_execute.h:458)
==23906==    by 0x12958492: zend_execute_scripts (zend.c:1437)
==23906==    by 0x128F8957: php_execute_script (main.c:2494)
==23906==    by 0x129E324C: php_handler (sapi_apache2.c:678)
==23906==    by 0x14728F: ap_run_handler (config.c:169)

I tried to analizes the code on SQLFreeHandle.c and as a quick and dirty fix
I've removed the call to __release_env:

diff -ur unixODBC-2.3.1/DriverManager/SQLFreeHandle.c unixODBC-2.3.1-new/DriverManager/SQLFreeHandle.c
--- unixODBC-2.3.1/DriverManager/SQLFreeHandle.c        2011-08-04 15:06:01.000000000 +0200
+++ unixODBC-2.3.1-new/DriverManager/SQLFreeHandle.c    2016-12-15 19:25:26.640469822 +0100
@@ -244,7 +244,6 @@

             thread_release( SQL_HANDLE_ENV, environment );

-            __release_env( environment );
             return SQL_SUCCESS;
         }
         break;

after this the core dumps are no more, but I suspect that now i've created a memory leak
May be we could find a better fix with your help?


Regards

Davide Pagnin
IT Architect
Delta System SPA


Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni contenute in questo messaggio sono riservate ed a uso esclusivo del destinatario. Qualora il messaggio in parola Le fosse pervenuto per errore, La invitiamo ad eliminarlo senza copiarlo e a non inoltrarlo a terzi, dandocene gentilmente comunicazione. Grazie.

Pursuant to Legislative Decree No. 196/2003, you are hereby informed that this message contains confidential information intended only for the use of the addressee. If you are not the addressee, and have received this message by mistake, please delete it and immediately notify us. You may not copy or disseminate this message to anyone. Thank you.


_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support
Reply | Threaded
Open this post in threaded view
|

Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

Nick Gorham-2
On 20/12/16 13:32, Davide Pagnin wrote:
Hi,

we are trying to port our application (php web application) from Windows world to linux.
Our database is on IBM i system machine, which is a DB2 flavor.

Ok, thanks for the report, I will take a look, May be in the new year as you have a workaround. Do you have

DontDLClose=1

Set against the driver in odbcinst,ini as I have seen many drivers get upset when the env is released.

--
Nick

_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support
Reply | Threaded
Open this post in threaded view
|

Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

Davide Pagnin
In reply to this post by Davide Pagnin
this is the relevant odbcinst.ini content:
[ODBC]
Pooling=1
Trace=0
TraceFile=/tmp/odbcinst.log

...

[IBM i Access ODBC Driver 64-bit]
Description=IBM i Access for Linux 64-bit ODBC Driver
Driver=/opt/ibm/iaccess/lib64/libcwbodbc.so
Setup=/opt/ibm/iaccess/lib64/libcwbodbcs.so
Threading=0
DontDLClose=1
UsageCount=1
CPTimeout=300

so, yes, we have DontDLClose set to 1

this is odbc.ini content:
[DB2HOST]
Description = Database ibm i
Driver = IBM i Access ODBC Driver 64-bit
System = 192.168.200.2
UserID =
Password =
Naming = 0
DefaultLibraries = QGPL
Database =
ConnectionType = 0
CommitMode = 0
ExtendedDynamic = 1
DefaultPkgLibrary = QGPL
DefaultPackage = A/DEFAULT(IBM),2,0,1,0,512
AllowDataCompression = 1
BlockSizeKB = 512
MaxFieldLength = 512
LibraryView = 0
AllowUnsupportedChar = 0
ForceTranslation = 0
CCSID = 1208


-----[hidden email] ha scritto: -----
A: [hidden email]
Da: [hidden email]
Inviato da: [hidden email]
Data: 21/12/2016 13:00
Oggetto: unixODBC-support Digest, Vol 141, Issue 3

------------------------------

Message: 2
Date: Tue, 20 Dec 2016 14:15:03 +0000
From: Nick Gorham <[hidden email]>
Subject: Re: [unixODBC-support] Segmentation Fault on Centos 7 when
Connection Pooling Enable
To: Support for the unixODBC project
<[hidden email]>
Message-ID: <[hidden email]>
Content-Type: text/plain; charset="windows-1252"; Format="flowed"

On 20/12/16 13:32, Davide Pagnin wrote:
> Hi,
>
> we are trying to port our application (php web application) from
> Windows world to linux.
> Our database is on IBM i system machine, which is a DB2 flavor.

Ok, thanks for the report, I will take a look, May be in the new year as
you have a workaround. Do you have

DontDLClose=1

Set against the driver in odbcinst,ini as I have seen many drivers get
upset when the env is released.

--
Nick


Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni contenute in questo messaggio sono riservate ed a uso esclusivo del destinatario. Qualora il messaggio in parola Le fosse pervenuto per errore, La invitiamo ad eliminarlo senza copiarlo e a non inoltrarlo a terzi, dandocene gentilmente comunicazione. Grazie.

Pursuant to Legislative Decree No. 196/2003, you are hereby informed that this message contains confidential information intended only for the use of the addressee. If you are not the addressee, and have received this message by mistake, please delete it and immediately notify us. You may not copy or disseminate this message to anyone. Thank you.


_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support
Reply | Threaded
Open this post in threaded view
|

Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

Kevin Adler
[hidden email] wrote on 12/22/2016 02:41:18 PM:

> From: "Davide Pagnin" <[hidden email]>

> To: [hidden email]
> Date: 12/22/2016 02:45 PM
> Subject: Re: [unixODBC-support] Segmentation Fault on Centos 7 when
> Connection Pooling Enable

> Sent by: [hidden email]
>

> this is the relevant odbcinst.ini content:
> [ODBC]
> Pooling=1
> Trace=0
> TraceFile=/tmp/odbcinst.log
>
> ...
>
> [IBM i Access ODBC Driver 64-bit]
> Description=IBM i Access for Linux 64-bit ODBC Driver
> Driver=/opt/ibm/iaccess/lib64/libcwbodbc.so
> Setup=/opt/ibm/iaccess/lib64/libcwbodbcs.so
> Threading=0
> DontDLClose=1
> UsageCount=1
> CPTimeout=300

The last time I tried connection pooling it seemed to work. I'll set up a CentOS 7 box next week and give it a shot. Can I ask what version of the RPM you have installed?

_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support
Reply | Threaded
Open this post in threaded view
|

Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

Nick Gorham-2
In reply to this post by Davide Pagnin
On 20/12/16 13:32, Davide Pagnin wrote:
Hi,

we are trying to port our application (php web application) from Windows world to linux.
Our database is on IBM i system machine, which is a DB2 flavor.

We realized that by using Connection Pooling (on Windows) we avoid connection setup,
in most of the cases, thus the application gain performances (on simple pages, we cut
connection time from 0,25-0,35 sec to 0,02-0,03 sec), this anyway is not the reason of
the mail.

As I said, when we enabled connection pooling on the linux centos 7.2 we are using
for development and testing, we faced core dumps of the httpd server.

At the moment we are using stock unixodbc rpm version 2.3.1 that came from centos (thus redhat)
but I've reproduced the core dumps also on 2.3.4 recompiled on our development system.

Ok, I think I have reproduced and fixed the problem. The issue was that the pooled connection had a reference back to the creating environment. But your app was releasing the environment and reconnecting. The DM should now release all pooled connections that were related to a environment when its released. Not back in svn yet, but the build of unixODBC-2.3.5-pre on the ftp site should have the fix. Give it a try and see if it helps.

--
Nick

_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support
Reply | Threaded
Open this post in threaded view
|

Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

Davide Pagnin
Hi,

I've recompiled the unixodbc from source taken from ftp as 2.3.5-pre

I doesn't dump anymore but it is not pooling the connections anymore.

I'm not a programmer, so I don't know if the fix you have introduced is right or not,
but if the fix you introduced is correct (stripping connections of released environment),
what programmers should do to enjoy connection pooling advantages?

They have not to release the environment?

What I'm sure, at the moment, is that on windows environment, our application
do benefit from connection pooling, on linux it used to crash the httpd daemon
and with your fix it does not benefit from pooling. I expected linux behavior to copy windows,
but may be there is a flaw also on windows side that is not visible or a memory leak
(which is what I probably introduced with my "quick" fix)

May be you could try to explain me, what you think is the "correct" behaviour?

Thanks in advance

Regards
Davide


-----[hidden email] ha scritto: -----
A: [hidden email]
Da: Nick Gorham
Inviato da: [hidden email]
Data: 06/01/2017 10:50
Oggetto: Re: [unixODBC-support] Segmentation Fault on Centos 7 when Connection Pooling Enable

On 20/12/16 13:32, Davide Pagnin wrote:
Hi,

we are trying to port our application (php web application) from Windows world to linux.
Our database is on IBM i system machine, which is a DB2 flavor.

We realized that by using Connection Pooling (on Windows) we avoid connection setup,
in most of the cases, thus the application gain performances (on simple pages, we cut
connection time from 0,25-0,35 sec to 0,02-0,03 sec), this anyway is not the reason of
the mail.

As I said, when we enabled connection pooling on the linux centos 7.2 we are using
for development and testing, we faced core dumps of the httpd server.

At the moment we are using stock unixodbc rpm version 2.3.1 that came from centos (thus redhat)
but I've reproduced the core dumps also on 2.3.4 recompiled on our development system.

Ok, I think I have reproduced and fixed the problem. The issue was that the pooled connection had a reference back to the creating environment. But your app was releasing the environment and reconnecting. The DM should now release all pooled connections that were related to a environment when its released. Not back in svn yet, but the build of unixODBC-2.3.5-pre on the ftp site should have the fix. Give it a try and see if it helps.

--
Nick
_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support


Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni contenute in questo messaggio sono riservate ed a uso esclusivo del destinatario. Qualora il messaggio in parola Le fosse pervenuto per errore, La invitiamo ad eliminarlo senza copiarlo e a non inoltrarlo a terzi, dandocene gentilmente comunicazione. Grazie.

Pursuant to Legislative Decree No. 196/2003, you are hereby informed that this message contains confidential information intended only for the use of the addressee. If you are not the addressee, and have received this message by mistake, please delete it and immediately notify us. You may not copy or disseminate this message to anyone. Thank you.


_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support
Reply | Threaded
Open this post in threaded view
|

Re: Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

Nick Gorham-2
On 09/01/17 14:25, Davide Pagnin wrote:
Hi,

I've recompiled the unixodbc from source taken from ftp as 2.3.5-pre

I doesn't dump anymore but it is not pooling the connections anymore.

I'm not a programmer, so I don't know if the fix you have introduced is right or not,
but if the fix you introduced is correct (stripping connections of released environment),
what programmers should do to enjoy connection pooling advantages?

They have not to release the environment?

What I'm sure, at the moment, is that on windows environment, our application
do benefit from connection pooling, on linux it used to crash the httpd daemon
and with your fix it does not benefit from pooling. I expected linux behavior to copy windows,
but may be there is a flaw also on windows side that is not visible or a memory leak
(which is what I probably introduced with my "quick" fix)

May be you could try to explain me, what you think is the "correct" behaviour?

Thanks in advance

Regards
Davide

Hi,

I expected this. My first aim was to remove the seg fault, at least by doing this we have identified that the problem I found is the same as the one that was causing you a problem.

Not sure what the "correct" behaviour is (other than a seg fault is rarely correct :-)). It is possible that the orphaned pooled connections (those left when the application releases the environment) could be allocated to the new environment when its allocated and used, but this worries me a little as it allows a leakage of information (or at least the connection state) between what the application would reasonably expect to be isolated connections. The simple solution would be to not release the environment.

Attempting to copy the windows behaviour exactly is problematic because of the different way DLL's on windows and shared objects on *nix operate.

It would be a simple task to null the pooled connection environment on release of the environment instead of completely removing them, and on reuse of the pooled connection reconnecting to the new environment, but for the above reasons I feel it would be unwise. I will leave the question open to see if any others have views on the preferred solution.

Is it not possible to alter the calling application to maintain environments?

--
Nick

_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support
Reply | Threaded
Open this post in threaded view
|

Re: Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

Kevin Adler
----- Original message -----
From: Nick Gorham <[hidden email]>
Sent by: [hidden email]
To: [hidden email]
Cc:
Subject: Re: [unixODBC-support] Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable
Date: Mon, Jan 9, 2017 8:49 AM
 
On 09/01/17 14:25, Davide Pagnin wrote:
Hi,

I've recompiled the unixodbc from source taken from ftp as 2.3.5-pre

I doesn't dump anymore but it is not pooling the connections anymore.

I'm not a programmer, so I don't know if the fix you have introduced is right or not,
but if the fix you introduced is correct (stripping connections of released environment),
what programmers should do to enjoy connection pooling advantages?

They have not to release the environment?

What I'm sure, at the moment, is that on windows environment, our application
do benefit from connection pooling, on linux it used to crash the httpd daemon
and with your fix it does not benefit from pooling. I expected linux behavior to copy windows,
but may be there is a flaw also on windows side that is not visible or a memory leak
(which is what I probably introduced with my "quick" fix)

May be you could try to explain me, what you think is the "correct" behaviour?

Thanks in advance

Regards
Davide

Hi,

I expected this. My first aim was to remove the seg fault, at least by doing this we have identified that the problem I found is the same as the one that was causing you a problem.

Not sure what the "correct" behaviour is (other than a seg fault is rarely correct :-)). It is possible that the orphaned pooled connections (those left when the application releases the environment) could be allocated to the new environment when its allocated and used, but this worries me a little as it allows a leakage of information (or at least the connection state) between what the application would reasonably expect to be isolated connections. The simple solution would be to not release the environment.

Attempting to copy the windows behaviour exactly is problematic because of the different way DLL's on windows and shared objects on *nix operate.

It would be a simple task to null the pooled connection environment on release of the environment instead of completely removing them, and on reuse of the pooled connection reconnecting to the new environment, but for the above reasons I feel it would be unwise. I will leave the question open to see if any others have views on the preferred solution.

Is it not possible to alter the calling application to maintain environments?
 
Here's how Windows Driver Manager does it: https://msdn.microsoft.com/en-us/library/ms716319(v=vs.85).aspx

Key points are 2 and 3:
 

2) Allocates an environment by calling SQLAllocHandle with the HandleType argument set to SQL_HANDLE_ENV. The environment allocated by this call will be an implicit shared environment because connection pooling has been enabled. The environment to be used is not determined, however, until SQLAllocHandle with a HandleType of SQL_HANDLE_DBC is called on this environment.

3) Allocates a connection by calling SQLAllocHandle with InputHandle set to SQL_HANDLE_DBC, and the InputHandle set to the environment handle allocated for connection pooling. The Driver Manager attempts to find an existing environment that matches the environment attributes set by the application. If no such environment exists, one is created, with a reference count (maintained by the Driver Manager) of 1. If a matching shared environment is found, the environment is returned to the application and its reference count is incremented. (The actual connection to be used is not determined by the Driver Manager until SQLConnect or SQLDriverConnect is called.)

 
I don't think would be impossible to do this within unixODBC, but it would be quite a bit of work to make happen. You'd need essentially two layers of environments: one temporary one to hold the environment attributes that are set by the application, then once they try to allocate a connection handle, compare those to the environment handles that are stored already. If you find one that matches, increase its refcount, otherwise allocate a new environment with those attributes and store it with a refcount of 1. Then when the application tries to use their environment handle, switch it out with the shared environment handle and continue the request. When they attempt to free the environment handle, decrement the refcount for the shared handle and free any resources for the outer handle if necessary (you could do this at dbc handle allocation time as well).
 
 
Then the only missing pieces are supporting SQL_ATTR_CONNECTION_POOLING/SQL_CP_ONE_PER_DRIVER and SQL_ATTR_CP_MATCH/SQL_CP_RELAXED_MATCH.
 
For ODBC 3.8 support, there's also SQL_ATTR_CONNECTION_POOLING/SQL_CP_DRIVER_AWARE and more as well.


_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support
Reply | Threaded
Open this post in threaded view
|

Rif: Re: Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

Davide Pagnin
In reply to this post by Nick Gorham-2
Hi,

I'd rather be able to contribute more constructively to
the discussion but as I said my skills as a programmer are limited.

With regard to the doubts about the security risks of the ability to reuse
the same connection after the environment has been released, I would like to
understand what further security risks are posed than what is written
in this presentation of the connection pooling for unixODBC:
http://www.unixodbc.org/doc/conn_pool.html

In my opinion, if the security risks/concerns are prevalent, the simple answer
is to not use connection pooling at all.

So, may be keeping the connection open and put the environment to NULL could
be a acceptable compromise, but perhaps even this choice will not make pooling "work",
as we saw it working on windows.

Anyway, our application is written in php, we are using pdo_odbc, I asked my collegues
if they can "mantain" the environment, but they said that they are not aware of a way
of keeping the environment outside of the same php script, which they already do.

May be you know how to do?

Regards

Davide Pagnin


-----[hidden email] ha scritto: -----
A: [hidden email]
Da: Nick Gorham
Inviato da: [hidden email]
Data: 09/01/2017 15:45
Oggetto: Re: [unixODBC-support] Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

On 09/01/17 14:25, Davide Pagnin wrote:
Hi,

I've recompiled the unixodbc from source taken from ftp as 2.3.5-pre

I doesn't dump anymore but it is not pooling the connections anymore.

I'm not a programmer, so I don't know if the fix you have introduced is right or not,
but if the fix you introduced is correct (stripping connections of released environment),
what programmers should do to enjoy connection pooling advantages?

They have not to release the environment?

What I'm sure, at the moment, is that on windows environment, our application
do benefit from connection pooling, on linux it used to crash the httpd daemon
and with your fix it does not benefit from pooling. I expected linux behavior to copy windows,
but may be there is a flaw also on windows side that is not visible or a memory leak
(which is what I probably introduced with my "quick" fix)

May be you could try to explain me, what you think is the "correct" behaviour?

Thanks in advance

Regards
Davide

Hi,

I expected this. My first aim was to remove the seg fault, at least by doing this we have identified that the problem I found is the same as the one that was causing you a problem.

Not sure what the "correct" behaviour is (other than a seg fault is rarely correct :-)). It is possible that the orphaned pooled connections (those left when the application releases the environment) could be allocated to the new environment when its allocated and used, but this worries me a little as it allows a leakage of information (or at least the connection state) between what the application would reasonably expect to be isolated connections. The simple solution would be to not release the environment.

Attempting to copy the windows behaviour exactly is problematic because of the different way DLL's on windows and shared objects on *nix operate.

It would be a simple task to null the pooled connection environment on release of the environment instead of completely removing them, and on reuse of the pooled connection reconnecting to the new environment, but for the above reasons I feel it would be unwise. I will leave the question open to see if any others have views on the preferred solution.

Is it not possible to alter the calling application to maintain environments?

--
Nick
_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support


Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni contenute in questo messaggio sono riservate ed a uso esclusivo del destinatario. Qualora il messaggio in parola Le fosse pervenuto per errore, La invitiamo ad eliminarlo senza copiarlo e a non inoltrarlo a terzi, dandocene gentilmente comunicazione. Grazie.

Pursuant to Legislative Decree No. 196/2003, you are hereby informed that this message contains confidential information intended only for the use of the addressee. If you are not the addressee, and have received this message by mistake, please delete it and immediately notify us. You may not copy or disseminate this message to anyone. Thank you.


_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support
Reply | Threaded
Open this post in threaded view
|

Re: Rif: Re: Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

Nick Gorham-2
On 11/01/17 17:24, Davide Pagnin wrote:
Hi,

I'd rather be able to contribute more constructively to
the discussion but as I said my skills as a programmer are limited.

With regard to the doubts about the security risks of the ability to reuse
the same connection after the environment has been released, I would like to
understand what further security risks are posed than what is written
in this presentation of the connection pooling for unixODBC:
http://www.unixodbc.org/doc/conn_pool.html

In my opinion, if the security risks/concerns are prevalent, the simple answer
is to not use connection pooling at all.

Yes, I tend to agree with that.

So, may be keeping the connection open and put the environment to NULL could
be a acceptable compromise, but perhaps even this choice will not make pooling "work",
as we saw it working on windows.

Anyway, our application is written in php, we are using pdo_odbc, I asked my collegues
if they can "mantain" the environment, but they said that they are not aware of a way
of keeping the environment outside of the same php script, which they already do.

May be you know how to do?

Regards

Ok, as always, I prefer to solve the current problem rather than make a lot of work that solves a problem that no one actually has. So I have made a simple change to the the code on the ftp site unixODBC-2.3.5-pre that should avoid the problem of the old env handle causing a seg fault, but allow the orphaned pooled connection to be reused.

Can you give that code a try and see how it plays.

--
Nick

_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support
Reply | Threaded
Open this post in threaded view
|

Rif: Re: Rif: Re: Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

Davide Pagnin
Hi,

I've played a bit with the new code in the afterhours yesterday night.

Pooling does work (I mean, we enjoy no connection setup delay after the initial one),
but then, when we pause and wait some time (I suppose when CPTimeout kick in)
we get segfault on httpd.

This is the backtrace of gdb:

Program terminated with signal 11, Segmentation fault.
#0  0x00007f8dadde15d3 in release_env (connection=connection@entry=0x7f8dc7a1c4f0) at SQLConnect.c:2440
2440                env_lib_list = connection -> environment -> env_lib_list;
#1  0x00007f8dadde1a19 in close_pooled_connection (ptr=ptr@entry=0x7f8dc7a1bf40) at SQLConnect.c:2874

I will investigate more, but In the meantime I wanted to give you some feedback.

Regards

Davide


-----[hidden email] ha scritto: -----
A: [hidden email]
Da: Nick Gorham
Inviato da: [hidden email]
Data: 11/01/2017 18:46
Oggetto: Re: [unixODBC-support] Rif: Re: Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

On 11/01/17 17:24, Davide Pagnin wrote:
Hi,

I'd rather be able to contribute more constructively to
the discussion but as I said my skills as a programmer are limited.

With regard to the doubts about the security risks of the ability to reuse
the same connection after the environment has been released, I would like to
understand what further security risks are posed than what is written
in this presentation of the connection pooling for unixODBC:
http://www.unixodbc.org/doc/conn_pool.html

In my opinion, if the security risks/concerns are prevalent, the simple answer
is to not use connection pooling at all.

Yes, I tend to agree with that.

So, may be keeping the connection open and put the environment to NULL could
be a acceptable compromise, but perhaps even this choice will not make pooling "work",
as we saw it working on windows.

Anyway, our application is written in php, we are using pdo_odbc, I asked my collegues
if they can "mantain" the environment, but they said that they are not aware of a way
of keeping the environment outside of the same php script, which they already do.

May be you know how to do?

Regards

Ok, as always, I prefer to solve the current problem rather than make a lot of work that solves a problem that no one actually has. So I have made a simple change to the the code on the ftp site unixODBC-2.3.5-pre that should avoid the problem of the old env handle causing a seg fault, but allow the orphaned pooled connection to be reused.

Can you give that code a try and see how it plays.

--
Nick
_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support


Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni contenute in questo messaggio sono riservate ed a uso esclusivo del destinatario. Qualora il messaggio in parola Le fosse pervenuto per errore, La invitiamo ad eliminarlo senza copiarlo e a non inoltrarlo a terzi, dandocene gentilmente comunicazione. Grazie.

Pursuant to Legislative Decree No. 196/2003, you are hereby informed that this message contains confidential information intended only for the use of the addressee. If you are not the addressee, and have received this message by mistake, please delete it and immediately notify us. You may not copy or disseminate this message to anyone. Thank you.


_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support
Reply | Threaded
Open this post in threaded view
|

Re: Rif: Re: Rif: Re: Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

Nick Gorham-2
On 12/01/17 08:01, Davide Pagnin wrote:
Hi,

I've played a bit with the new code in the afterhours yesterday night.

Pooling does work (I mean, we enjoy no connection setup delay after the initial one),
but then, when we pause and wait some time (I suppose when CPTimeout kick in)
we get segfault on httpd.

This is the backtrace of gdb:

Program terminated with signal 11, Segmentation fault.
#0  0x00007f8dadde15d3 in release_env (connection=connection@entry=0x7f8dc7a1c4f0) at SQLConnect.c:2440
2440                env_lib_list = connection -> environment -> env_lib_list;
#1  0x00007f8dadde1a19 in close_pooled_connection (ptr=ptr@entry=0x7f8dc7a1bf40) at SQLConnect.c:2874

I will investigate more, but In the meantime I wanted to give you some feedback.

My fault, I missed a null dereference in the case of a timeout. Try the zip thats there now.

--
Nick

_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support
Reply | Threaded
Open this post in threaded view
|

Rif: Re: Rif: Re: Rif: Re: Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

Davide Pagnin
Hi,

I've recompiled the last tar.gz and after half a day of use there is no segfault
and the pooling does work (no connection setup delay after first time).

Unfortunately there is perhaps one last issue, even if we have a CPTimeout of 60 seconds,
the connections are not closed until the httpd child daemon is closed.

Without pooling connections are closed immediately, as expect.

Can this be a side effect of the fix of pooling we are testing?

Regards

Davide


-----[hidden email] ha scritto: -----
A: [hidden email]
Da: Nick Gorham
Inviato da: [hidden email]
Data: 12/01/2017 11:19
Oggetto: Re: [unixODBC-support] Rif: Re: Rif: Re: Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

On 12/01/17 08:01, Davide Pagnin wrote:
Hi,

I've played a bit with the new code in the afterhours yesterday night.

Pooling does work (I mean, we enjoy no connection setup delay after the initial one),
but then, when we pause and wait some time (I suppose when CPTimeout kick in)
we get segfault on httpd.

This is the backtrace of gdb:

Program terminated with signal 11, Segmentation fault.
#0  0x00007f8dadde15d3 in release_env (connection=connection@entry=0x7f8dc7a1c4f0) at SQLConnect.c:2440
2440                env_lib_list = connection -> environment -> env_lib_list;
#1  0x00007f8dadde1a19 in close_pooled_connection (ptr=ptr@entry=0x7f8dc7a1bf40) at SQLConnect.c:2874

I will investigate more, but In the meantime I wanted to give you some feedback.

My fault, I missed a null dereference in the case of a timeout. Try the zip thats there now.

--
Nick
_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support


Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni contenute in questo messaggio sono riservate ed a uso esclusivo del destinatario. Qualora il messaggio in parola Le fosse pervenuto per errore, La invitiamo ad eliminarlo senza copiarlo e a non inoltrarlo a terzi, dandocene gentilmente comunicazione. Grazie.

Pursuant to Legislative Decree No. 196/2003, you are hereby informed that this message contains confidential information intended only for the use of the addressee. If you are not the addressee, and have received this message by mistake, please delete it and immediately notify us. You may not copy or disseminate this message to anyone. Thank you.


_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support
Reply | Threaded
Open this post in threaded view
|

Re: Rif: Re: Rif: Re: Rif: Re: Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

Nick Gorham-2
On 12/01/17 16:08, Davide Pagnin wrote:
Hi,

I've recompiled the last tar.gz and after half a day of use there is no segfault
and the pooling does work (no connection setup delay after first time).

Unfortunately there is perhaps one last issue, even if we have a CPTimeout of 60 seconds,
the connections are not closed until the httpd child daemon is closed.

Without pooling connections are closed immediately, as expect.

Can this be a side effect of the fix of pooling we are testing?

Regards

Davide

Probably. I will take a look.

--
Nick

_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support
Reply | Threaded
Open this post in threaded view
|

Re: Rif: Re: Rif: Re: Rif: Re: Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

Nick Gorham-2
On 12/01/17 16:30, Nick Gorham wrote:
On 12/01/17 16:08, Davide Pagnin wrote:
Hi,

I've recompiled the last tar.gz and after half a day of use there is no segfault
and the pooling does work (no connection setup delay after first time).

Unfortunately there is perhaps one last issue, even if we have a CPTimeout of 60 seconds,
the connections are not closed until the httpd child daemon is closed.

Without pooling connections are closed immediately, as expect.

Can this be a side effect of the fix of pooling we are testing?

Regards

Davide

Probably. I will take a look.

Do you mean that more and more connections build up, or that the number of connections never reduces even if its idle.

It only checks for expired connections when a new connection comes in, so it won't asynchronously release connections.

--
Nick

_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support
Reply | Threaded
Open this post in threaded view
|

Rif: Re: Rif: Re: Rif: Re: Rif: Re: Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

Davide Pagnin
Hi,

you're right, the number of connection never reduces even if is idle,
but we don't see a continuos build up.

We are working in a testing environment and no more than 2 users work on that,
but we have 8 httpd processes that can keep open 1 connection each.

What has made me think wrong was that with the latest - 1 iteration of the code,
we connected, worked, than logged out and closed the browsed and after some time
the connections were closed by a segfault, so I was sure that something was
running asynchronously.

Regards
Davide

-----[hidden email] ha scritto: -----
A: [hidden email]
Da: Nick Gorham
Inviato da: [hidden email]
Data: 12/01/2017 17:51
Oggetto: Re: [unixODBC-support] Rif: Re: Rif: Re: Rif: Re: Rif: Re: Segmentation Fault on Centos 7 when Connection Pooling Enable

On 12/01/17 16:30, Nick Gorham wrote:
On 12/01/17 16:08, Davide Pagnin wrote:
Hi,

I've recompiled the last tar.gz and after half a day of use there is no segfault
and the pooling does work (no connection setup delay after first time).

Unfortunately there is perhaps one last issue, even if we have a CPTimeout of 60 seconds,
the connections are not closed until the httpd child daemon is closed.

Without pooling connections are closed immediately, as expect.

Can this be a side effect of the fix of pooling we are testing?

Regards

Davide

Probably. I will take a look.

Do you mean that more and more connections build up, or that the number of connections never reduces even if its idle.

It only checks for expired connections when a new connection comes in, so it won't asynchronously release connections.

--
Nick
_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support


Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni contenute in questo messaggio sono riservate ed a uso esclusivo del destinatario. Qualora il messaggio in parola Le fosse pervenuto per errore, La invitiamo ad eliminarlo senza copiarlo e a non inoltrarlo a terzi, dandocene gentilmente comunicazione. Grazie.

Pursuant to Legislative Decree No. 196/2003, you are hereby informed that this message contains confidential information intended only for the use of the addressee. If you are not the addressee, and have received this message by mistake, please delete it and immediately notify us. You may not copy or disseminate this message to anyone. Thank you.


_______________________________________________
unixODBC-support mailing list
[hidden email]
http://mailman.unixodbc.org/mailman/listinfo/unixodbc-support