An unhandled security exception has occurred - Boncode Connector Issue?

idtDev01 · November 21, 2016, 9:43pm

Hi Bilal,
We had an issue today that brought the production server down. This is
the first error that we’ve seen since updating the connector over a month
ago. The system event log is shown below. Any suggestions on how to
correct this?

Thanks!

Event code: 4010

Event message: An unhandled security exception has occurred.

Event time: 11/21/2016 2:15:45 PM

Event time (UTC): 11/21/2016 9:15:45 PM

Event ID: 6025407eb4c74ef2a683384047ae4f22

Event sequence: 34

Event occurrence: 1

Event detail code: 0

Application information:

Application domain: redacted

Trust level: Full 

Application Virtual Path: / 

Application Path: C:\inetpub\wwwroot\application\ 

Machine name: redacted

Process information:

Process ID: 8608 

Process name: w3wp.exe 

Account name: IIS APPPOOL\DefaultAppPool

Request information:

Request URL: redacted

Request path: /secure/index.cfm 

User host address: redacted

User:  

Is authenticated: False 

Authentication Type:  

Thread account name: IIS APPPOOL\DefaultAppPool

Bilal · November 22, 2016, 4:57pm

Try creating an event source for the connector so it can record the issue:

http://www.boncode.net/connector/webdocs/Tomcat_Connector.htm#_Toc459890745

Best,
BilalOn Monday, November 21, 2016 at 4:43:54 PM UTC-5, Jace Coleman wrote:

Hi Bilal,
We had an issue today that brought the production server down. This is
the first error that we’ve seen since updating the connector over a month
ago. The system event log is shown below. Any suggestions on how to
correct this?

Thanks!

Event code: 4010

Event message: An unhandled security exception has occurred.

Event time: 11/21/2016 2:15:45 PM

Event time (UTC): 11/21/2016 9:15:45 PM

Event ID: 6025407eb4c74ef2a683384047ae4f22

Event sequence: 34

Event occurrence: 1

Event detail code: 0

Application information:
Application domain: redacted

Trust level: Full 

Application Virtual Path: / 

Application Path: C:\inetpub\wwwroot\application\ 

Machine name: redacted
Process information:
Process ID: 8608 

Process name: w3wp.exe 

Account name: IIS APPPOOL\DefaultAppPool 
Request information:
Request URL: redacted

Request path: /secure/index.cfm 

User host address: redacted

User:  

Is authenticated: False 

Authentication Type:  

Thread account name: IIS APPPOOL\DefaultAppPool 

idtDev01 · November 22, 2016, 9:42pm

Thanks Bilal, we will give that a try and report back if anything useful
pops up. Your assistance is always appreciated!

idtDev01 · December 12, 2016, 3:09pm

We created an event source for the Boncode connector a few weeks back
thanks to Bilal’s recommendation and just had another instance where IIS
worker process locked up at 100% cpu. The log indicates that there was an
error during the spool to the client, but as far as I know this shouldn’t
cause IIS to lock up. Any suggestions on where to look next?

Error during spool to client (browser may have navigated away): The remote
host closed the connection. The error code is 0x800704CD. at
System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result,
Boolean throwOnDisconnect)

at System.Web.Hosting.IIS7WorkerRequest.ExplicitFlush()

at System.Web.HttpResponse.Flush(Boolean finalFlush, Boolean async)

at BonCodeIIS.BonCodeCallHandler.PrintFlush(BonCodeAJP13PacketCollection
flushCollection)

Thanks!

idtDev01 · December 13, 2016, 5:57pm

You’re awesome Bilal! I’ll give it a try and post the results.

Thanks!!
Jace

Bilal · December 13, 2016, 5:31pm

Jace,
this error most likely occurs because users have navigated away or closed
their browsers. IIS is still trying to push data to these users. The end
result is this error. In general, disconnects are unavoidable in the
web-communication cycle. This just records the occurrence.
However, if you believe this to be the issue for increased CPU you can
upgrade to version 1.0.32+ which reduces the occurrence of communication
with already disconnected clients.

Also to look at in that case:

using a connection pool: increase maxconnections to >10, e.g.:
500
increase packet size on both tomcat and boncode side
reduce idle timeout on IIS process (Application Pool Advanced
Settings:Process Model), >2 and <5 mins
have a connectionTimeout on Tomcat server.xml AJP protocol definition

Most of this has a mention in the docs if you are not certain how to do it.
I would start with the upgrade though.

Best,
Bilal

idtDev01 · December 15, 2016, 5:47pm

Hi Bilal, we updated to the newest connector release last night (AJP13_v1033)
and found that the site was having intermittent connection errors this
morning which was producing the output below:

Generic Connector Communication Error:------------------------------

Please check and adjust your setup:
Ensure that Tomcat is running on given host and port.
If this is a timeout error consider adjusting IIS timeout by changing
executionTimeout attribute in web.config (see manual).

We have since reverted back to AJP13_v1029B which appears to have resolved
the connection error above but leaves us vulnerable to IIS worker processes
locking up again.

On Tuesday, December 13, 2016 at 10:57:43 AM UTC-7, Jace Coleman wrote:

You’re awesome Bilal! I’ll give it a try and post the results.

Thanks!!
Jace

Bilal · December 16, 2016, 8:59pm

Jace,
you will need to post your server.xml line that has the AJP protocol
definition and the boncode setting file.
Please remove any sensitive information before posting.
Also post any error messages from Windows log.

Best,
Bilal

Bilal · December 19, 2016, 11:35pm

Jace,
I am seeing a potential condition where tomcat is closing the connection
after 60 seconds while IIS will maintain them for 20 mins (the default).
This can lead to errors you are seeing when IIS is trying to reuse a
channel that was closed. After displaying the error, upon refresh, this
should go away since the system would refresh connection. Is that what you
see?

Can we try this.
Increase the timeout on tomcat to: connectionTimeout=“1210000”

Then change IIS to a corresponding timeout but a little shorter (see image).

https://lh3.googleusercontent.com/-sujsoLR4ETk/WFhupEhoSQI/AAAAAAAABGM/eEpBRvwVjzcg4PSeBatEtaZj5kt5PhDuwCLcB/s1600/iisIdleTimeout.PNG

Let me know how this goes,
Best,
BilalOn Monday, December 19, 2016 at 11:01:04 AM UTC-5, Jace Coleman wrote:

Hi Bilal,
Here’s our server settings and log information. Thanks for taking a look.

<Connector port=“8009”
       protocol="AJP/1.3"

       redirectPort="8443"

       tomcatAuthentication="false"

       maxThreads="500"

       connectionTimeout="60000"

       packetSize="65536"/>
8009

localhost

500

0

True

True

False

False

65536

Log Name: Application

Source: BonCodeConnector

Date: 12/16/2016 11:58:49 AM

Event ID: 417

Task Category: None

Level: Warning

Keywords: Classic

User: N/A

Computer:

Description:

Error during spool to client (browser may have navigated away): The remote
host closed the connection. The error code is 0x80070057. at
System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result,
Boolean throwOnDisconnect)

at System.Web.Hosting.IIS7WorkerRequest.ExplicitFlush()

at System.Web.HttpResponse.Flush(Boolean finalFlush, Boolean async)

at
BonCodeIIS.BonCodeCallHandler.PrintFlush(BonCodeAJP13PacketCollection
flushCollection)

Event Xml:
<Provider Name="BonCodeConnector" />

<EventID Qualifiers="0">417</EventID>

<Level>3</Level>

<Task>0</Task>

<Keywords>0x80000000000000</Keywords>

<TimeCreated SystemTime="2016-12-16T18:58:49.000000000Z" />

<EventRecordID>40052</EventRecordID>

<Channel>Application</Channel>

<Computer>WIN-0000000000</Computer>

<Security />
<Data>Error during spool to client (browser may have navigated away): 
The remote host closed the connection. The error code is 0x80070057. at
System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result,
Boolean throwOnDisconnect)

at System.Web.Hosting.IIS7WorkerRequest.ExplicitFlush()

at System.Web.HttpResponse.Flush(Boolean finalFlush, Boolean async)

at
BonCodeIIS.BonCodeCallHandler.PrintFlush(BonCodeAJP13PacketCollection
flushCollection)

idtDev01 · December 19, 2016, 4:01pm

Hi Bilal,
Here’s our server settings and log information. Thanks for taking a look.

<Connector port=“8009”

       protocol="AJP/1.3"

       redirectPort="8443"

       tomcatAuthentication="false"

       maxThreads="500"

       connectionTimeout="60000"

       packetSize="65536"/>

8009

localhost

500

0

True

False

65536

Log Name: Application

Source: BonCodeConnector

Event ID: 417

Task Category: None

Level: Warning

Keywords: Classic

User: N/A

Computer:

Description:

Error during spool to client (browser may have navigated away): The remote
host closed the connection. The error code is 0x80070057. at
System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result,
Boolean throwOnDisconnect)

at System.Web.Hosting.IIS7WorkerRequest.ExplicitFlush()

at System.Web.HttpResponse.Flush(Boolean finalFlush, Boolean async)

at BonCodeIIS.BonCodeCallHandler.PrintFlush(BonCodeAJP13PacketCollection
flushCollection)

Event Xml:

<Provider Name="BonCodeConnector" />

<EventID Qualifiers="0">417</EventID>

<Level>3</Level>

<Task>0</Task>

<Keywords>0x80000000000000</Keywords>

<TimeCreated SystemTime="2016-12-16T18:58:49.000000000Z" />

<EventRecordID>40052</EventRecordID>

<Channel>Application</Channel>

<Computer>WIN-0000000000</Computer>

<Security />

<Data>Error during spool to client (browser may have navigated away):

The remote host closed the connection. The error code is 0x80070057. at
System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result,
Boolean throwOnDisconnect)

at System.Web.Hosting.IIS7WorkerRequest.ExplicitFlush()

at System.Web.HttpResponse.Flush(Boolean finalFlush, Boolean async)

at BonCodeIIS.BonCodeCallHandler.PrintFlush(BonCodeAJP13PacketCollection
flushCollection)

Date: 12/16/2016 11:58:49 AM