Best Practices: The Blue Coat Proxy AV and ProxySG Appliances

Technical Brief

420 North Mary Avenue Sunnyvale, CA 94085 www.bluecoat.com Send comments about this technical brief to
[email protected]

1.866.30.BCOAT 408.220.2200 Direct 408.220.2250 Fax

Copyright 1999-2008 Blue Coat Systems, Inc. All rights reserved worldwide. No part of this document may be reproduced by any means nor modified, decompiled, disassembled, published or distributed, in whole or in part, or translated to any electronic medium or other means without the written consent of Blue Coat Systems, Inc. All right, title and interest in and to the Software and documentation are and shall remain the exclusive property of Blue Coat Systems, Inc. and its licensors. ProxyAV, CacheOS, SGOS, SG, Spyware Interceptor, Scope, ProxyRA Connector, ProxyRA Manager, Remote Access and MACH5 are trademarks of Blue Coat Systems, Inc. and CacheFlow, Blue Coat, Accelerating The Internet, ProxySG, WinProxy, AccessNow, Ositis, Powering Internet Management, The Ultimate Internet Sharing Solution, Cerberian, Permeo, Permeo Technologies, Inc., and the Cerberian and Permeo logos are registered trademarks of Blue Coat Systems, Inc. All other trademarks contained in this document and in the Software are the property of their respective owners. BLUE COAT SYSTEMS, INC. DISCLAIMS ALL WARRANTIES, CONDITIONS OR OTHER TERMS, EXPRESS OR IMPLIED, STATUTORY OR OTHERWISE, ON SOFTWARE AND DOCUMENTATION FURNISHED HEREUNDER INCLUDING WITHOUT LIMITATION THE WARRANTIES OF DESIGN, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL BLUE COAT SYSTEMS, INC., ITS SUPPLIERS OR ITS LICENSORS BE LIABLE FOR ANY DAMAGES, WHETHER ARISING IN TORT, CONTRACT OR ANY OTHER LEGAL THEORY EVEN IF BLUE COAT SYSTEMS, INC. HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Blue Coat Systems, Inc. Documentation and Education Services

ii

Contents
Chapter 1: About Technical Briefs

Overview...................................................................................................................................................... 5 Issue #1Very Large Data Streams or Slow Downloads ..................................................................... 5 About Intelligent Connection Traffic Monitoring (ICTM) .............................................................. 5 Solution A ............................................................................................................................................... 7 Solution B................................................................................................................................................ 9 Issue #2 - More Than Seventy Byte Ranges Within One Request........................................................ 9 Issue ......................................................................................................................................................... 9 Resolution ............................................................................................................................................... 9 Additional ICAP Policy Optimization Suggestions ............................................................................ 10 Aggressive Client Retries ................................................................................................................... 10 Avoid scanning virus pattern updates ............................................................................................. 10 Installing the CPL Described in This Document.................................................................................. 11
Appendix A: ProxySG and ProxyAV Configuration Interaction

Configuration Interaction Behavior ....................................................................................................... 13

Appendix B: ProxySG and ProxyAV Appliance Possible Errors

Proceedable Errors .............................................................................................................................. 15 Unproceedable Errors ......................................................................................................................... 15 Errors Produced by the ProxySG ...................................................................................................... 15

Blue Coat Systems, Inc. Documentation and Education Services

iii

Best Practices: ProxyAV and ProxySG

Technical Brief

iv

Blue Coat Systems, Inc. Documentation and Education Services

About Technical Briefs

Technical briefs illustrate the features and capabilities of Blue Coat products. By describing generic solutions, technical briefs provide a foundation that Blue Coat customers use to understand how Blue Coat products are used to solve specific problems. Technical briefs are not intended to solve customer-specific requests; if you require a customized solution to address a specific concern, contact Blue Coat Professional Services at [email protected].

Overview
The integration of the ProxySG appliance with the ProxyAV appliance (or third-party ICAP scanner) enables network administrators to scan all Web traffic for viruses and malware. Anti-virus (AV) scanning can be achieved with minimal configuration; however, some client/server network behavior can impact the performance of the ProxySG/ProxyAV appliance solution, and can thus impact user satisfaction. These issues usually result in slowness, broken applications, and broken pages. If you encounter these issues, apply the policies provided below.
Note: For optimum performance and to minimize issues, the ProxySG and ProxyAV appliances must be on the same subnet and not separated by a router. Verify that the speed and duplex of the ProxyAV network interface and the connected switch port are configured the same.

Issue #1Very Large Data Streams or Slow Downloads

Certain client requests delivered through HTTP are not finite, but are instead streams of raw data. For example, audio streams, certain webcam images, and stock tickers are delivered as raw data. Typically, end users experience a general slowing in the retrieval of dataWeb browsers (or other applications) might even freeze. If the application or Web browser does begin working, the expected data might stream by very quickly. In extreme situations, a large number of large data-stream requests can cause delays to other requests, potentially affecting all HTTP traffic.

About Intelligent Connection Traffic Monitoring (ICTM)

The Intelligent Connection Traffic Monitoring (ICTM) feature allows the ProxyAV to drop download connections that are taking longer than a normal time frame to complete. This keeps resources available to download other objects. These slow downloads might be infinite data streams, such as a stock ticker. As this type of download never ends, excessive ProxySG and ProxyAV appliance resources are consumed.

Blue Coat Systems, Inc. Documentation and Education Services

Best Practices: ProxyAV and ProxySG

Technical Brief

When ICTM is enabled, the ProxyAV checks for slow downloads. If the specified warning threshold is reached, the ProxyAV notifies the administrator of the dropped URLs (through an e-mail or SNMP trap, if the option is selected), which allows for the proxy administrator to create policy to ignore these URLs. If the critical threshold is reached, the ProxyAV terminates the oldest, slowest connections so that the level below the threshold is maintained. For information about configuring this feature, see Chapter 3 in the Blue Coat ProxyAV Configuration and Management Guide.
Note: If you are using a third-party anti-virus solution with the ProxySG or do not have the ICTM feature enabled, use one of the solutions in this section.

Issue
Attempting to virus scan this type of data can potentially consume significant time and AV appliance resources (potentially slowing other scans)until an error is returned. If allowed to continue, these transfers fail with one of the following ICAP X-Error-Codes:
Maximum file size exceeded Scan timeout

The default configuration of the ProxyAV triggers such errors only after the file size exceeds 100MB or after 800 seconds of delay. Such a delay is unacceptable for the time-sensitive data required by webcams and stock tickers. Some client applications automatically retry a request if no response is received in a certain amount of time. Also, users might attempt to refresh the request when a response is delayed. Refreshing the request can lead to a high number of queued requests for the same object, which increases the competition for ProxyAV scanning resources. When a client application is especially aggressive, it impacts all network traffic as the ProxySG waits for ProxyAV responses.

Resolution
To avoid these issues, implement one of the following policies. These example policies use different approaches and are not intended to co-exist. Select only one.
Note: If you are using a third-party antivirus software, Blue Coat recommends that you implement the CPL in Solution A.

Blue Coat Systems, Inc. Documentation and Education Services

Technical Brief

Best Practices: ProxyAV and ProxySG

Solution A
To enhance user satisfaction and achieve maximum performance from the ProxyAV, some customers choose not to scan the data streams that are known to cause issues. One benefit of this policy is reduced load on the ProxyAV. The risk is that the exemption could potentially allow malicious content to slip viruses through unscanned. The following example policy is based on request/response patterns that indicate an overly large or slow download.
; -------------ICAP Best Practices---------------------------------------------;;; The actual ICAP respmod rule should already be defined, these actions will ;;; reset it back to (no) upon an attempt to scan a streaming object or an object ;;; that shouldn't be scanned <cache> delete_on_abandonment(yes) <cache> url.scheme=http condition=NOICAP response.icap_service(no) <Proxy> request.header.User-Agent="ProxyAV" patience_page(no) ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;; This condition will match if the content length is greater than ;; 99,999,999 bytes, or no content length is provided. Both of ;; these are signs that this may tie up a thread on the AV for too long. define condition NO_or_LARGE_CONTENT_LENGTH response.header.Content-Length=!"" response.header.Content-Length=!"^[0-9]{1,8}$" end condition NO_or_LARGE_CONTENT_LENGTH ;; Here are some common infinite stream media types, these will ;; also block some threads on the AV. define condition MEDIA_MIME_TYPES response.header.Content-Type="video/" response.header.Content-Type="application/streamingmedia" response.header.Content-Type="application/x-streamingmedia" response.header.Content-Type="application/vnd.rn" response.header.Content-Type="application/ogg" response.header.Content-Type="application/x-ogg" response.header.Content-Type="audio/" response.header.Content-Type="multipart/x-mixed-replace" end condition MEDIA_MIME_TYPES ;; None of these exist right now define condition Missbehaving_Modern_UserAgents ; Add modern user-agents known to missbehave to this condition ; and remove the comment character (semicolon) before Rule 3 above. ;request.header.User-Agent="" end condition Missbehaving_Modern_UserAgents define condition VIDEO_AUDIO_with_NO_or_LARGE_CONTENT_LENGTH condition=NO_or_LARGE_CONTENT_LENGTH condition=MEDIA_MIME_TYPES end condition VIDEO_AUDIO_with_NO_or_LARGE_CONTENT_LENGTH

Instructor Edition Blue Coat Systems, Inc. Documentation and Education Services

Best Practices: ProxyAV and ProxySG

Technical Brief

define condition UserAgents_with_NO_or_LARGE_CONTENT_LENGTH condition=NO_or_LARGE_CONTENT_LENGTH condition=Missbehaving_Modern_UserAgents end condition UserAgents_with_NO_or_LARGE_CONTENT_LENGTH define condition MissBehaving_Old_UserAgents request.header.User-Agent="Winamp" request.header.User-Agent="NSPlayer" request.header.User-Agent="RMA" request.header.User-Agent="ultravox" request.header.User-Agent="itunes" request.header.User-Agent="forest" request.header.User-Agent="Scottrader" request.header.User-Agent="SVN" end condition MissBehaving_Old_UserAgents define condition HTTPv0.9_UserAgents http.response.version=0.9 condition=MissBehaving_Old_UserAgents end condition HTTPv0.9_UserAgents define condition NOICAP condition=VIDEO_AUDIO_with_NO_or_LARGE_CONTENT_LENGTH condition=HTTPv0.9_UserAgents condition=UserAgents_with_NO_or_LARGE_CONTENT_LENGTH ; Yahoos stock ticker problem -15sep06 url.domain=//streamerapi.finance.yahoo.com url.domain=//stream.aol.com url.domain=//finance.google.com ; Other streaming media exceptions url.domain=//youtube.com url.domain=//pandora.com end condition NOICAP ; -------------End ICAP Best Practices-------------------------

Note:

Configure this policy so that it is evaluated last.

When deciding the scan/no-scan/fail_open/fail_closed options, carefully consider the inherent security issues. See Appendix A: "ProxySG and ProxyAV Configuration Interaction" on page 13 and Appendix B: "ProxySG and ProxyAV Appliance Possible Errors" on page 15 for more information. Refer to "Installing the CPL Described in This Document" on page 11 for instructions on installing this CPL.

Blue Coat Systems, Inc. Documentation and Education Services

Technical Brief

Best Practices: ProxyAV and ProxySG

Solution B
Some administrators choose to wait for one of the symptomatic errors (Maximum file size exceeded or Scan timeout) to occur and then serve the data stream unscanned. This approach ensures that all data is still sent to the ProxyAVthus, the maximum amount of scanning can occur. The downside to this approach is that all requests for infinite data-streams must reach the maximum file size or scan timeout configured on the ProxyAV. If a sufficient number of concurrent requests for such data streams occur, the request queue will slow or delay other traffic. The following policy example serves the data stream if the error is Maximum file size exceeded or Scan timeout. Other errors are denied.
<cache> response.icap_service(<resp_service>, fail_open) <proxy> condition=!maxfilesizeexceeded_or_scantimeout_errors exception(icap_error) define condition maxfilesizeexceeded_or_scantimeout_errors icap_error_code=max_file_size_exceeded icap_error_code=scan_timeout end condition maxfilesizeexceeded_or_scantimeout_errors

Refer to "Installing the CPL Described in This Document" on page 11 for instructions on installing this CPL.

Issue #2 - More Than Seventy Byte Ranges Within One Request

Some versions of the Adobe Acrobat browser plug-ins, when interacting with certain PDF documents, make requests with very large numbers of byte-range groupings. The HTTP byte-range request is a method of requesting only a portion of the data within an object. A single HTTP request can specify multiple byte ranges in a list using start and stop byte offsets.

Issue
The ProxyAV supports up to seventy byte ranges per request. For requests with fewer than seventy byte ranges, the object data is retrieved from the origin server and scanned normally. If the entire object is already in the cache, each byte range is extracted and served from the cached data. However, if a request has more than seventy byte ranges, the ProxySG is unable to serve the data from the cache and instead must retrieve the data from the origin server and rescan it. Some Acrobat plug-ins fail to handle the patience-page behavior of the proxy during these 70+ byte-range retrievals and, instead, display a blank screen. Such Acrobat plug-ins operate correctly for all other requests, even with regard to patience-page operation.

Instructor Edition Blue Coat Systems, Inc. Documentation and Education Services

Best Practices: ProxyAV and ProxySG

Technical Brief

Resolution
Normally, this issue can be resolved by upgrading the Acrobat plug-in. However, if an upgrade is not possible, or the particular PDF files continue to trigger this behavior, you can selectively disable the patience-page behavior to mitigate the issue. The following example policy disables the patience-page behavior for PDF objects from Blue Coat sites:
<proxy> url.domain=bluecoat.com url.extension=(pdf) patience_page(no)

Although all data is still scanned, use this type of policy only as needed. The lack of patience-page prompts often causes users to reload a page prematurely, which can result in user dissatisfaction and redundant scanning of the same data.

Additional ICAP Policy Optimization Suggestions

This section provides additional tips for optimizing your ICAP policy.

Aggressive Client Retries

When an HTTP request appears cacheable, the ProxySG completes the download, even if the requesting client has abandoned the connection. This allows the proxy to store a cached version of the object for future requests. However, for slow downloads, this behavior can result in each client request queuing a separate instance for scanning. To avoid the continued processing of a request after the client application has disconnected, you can enable the CPL property delete_on_abandonment for certain client applications. The following example policy prevents queuing of duplicate requests for a known aggressive client:
<cache> request.header.User-Agent="Winamp" delete_on_abandonment(yes)

Avoid scanning virus pattern updates

Each anti-virus vendor provides pattern file updates that necessarily contain portions (or descriptions) of viruses. Generally, these virus segments are encoded and are too small to be mistaken as a true virus by other AV vendors. But occasional false positives occur. These can be prevented by exempting virus pattern update locations from scanning, as the following example policy illustrates (place this policy after all other ICAP policies):
<cache> url.host=download.bluecoat.com response.icap_service(no) url.host=download.ositis.com response.icap_service(no)

10

Blue Coat Systems, Inc. Documentation and Education Services

Technical Brief

Best Practices: ProxyAV and ProxySG

Installing the CPL Described in This Document

The following procedure describes how to download and install the CPL described in this document.
Note: 1. Always save a copy of your existing policy file before making any changes.

Download the text file for the desired CPL. Download the "Solution A" CPL at:

http://techlabs.bluecoat.com/policy/icap_noscan.txt 2. 3. 4. 5. 6. 7. 8. Download the "Solution B" CPL at: http://techlabs.bluecoat.com/policy/icap_scan.txt Save the file to your desktop or other convenient location. Modify the policy to meet your requirements. Using the ProxySG Management Console, select Configuration > Policy > Policy Files. From the Install Local File from drop-down menu, select Text Editor, and then click Install. The Edit and Install the Local Policy File page displays. Open the CPL file you downloaded and copy the text. Return to the Edit and Install the Local Policy File page and paste the contents of the file at the end of the local policy file on your ProxySG. Click Install. A dialog displays, informing you whether the installation was successful. If necessary, correct any errors in the file and re-install it.

Instructor Edition Blue Coat Systems, Inc. Documentation and Education Services

11

Best Practices: ProxyAV and ProxySG

Technical Brief

12

Blue Coat Systems, Inc. Documentation and Education Services

Appendix A: ProxySG and ProxyAV Configuration Interaction

Configuration Interaction Behavior

The ProxySG can act on ProxyAV errors, depending on its fail_open setting and any policies that test the X-Error-Code ICAP header. Additionally, the ProxyAV can be configured to block or serve data when an error occurs. The interaction of these two settings is detailed below. If the ProxyAV is configured to block on an error, the ProxySG can fail-open and return the object data or it can fail-close and return an exception. If an exception is returned due to a fail-close configuration, the ProxySG includes the contents of the X-Error-Details ICAP header in the message. If the ProxyAV is configured to serve content on an error, the ProxySG fail_open and fail_closed settings are unimportant. The object data is returned to the client unless a policy test of the X-Error-Code ICAP header changes the ProxySG behavior, as shown in the following table.
AV Appliance Block ProxySG fail_open ProxySG fail_close ProxySG policy tests X-Error-Code ICAP header Data served (any cached object marked as requiring rescan) Exception served (with X-Error-Details included) Policy determined unless ProxySG is Fail-closed AV Appliance Serve Data served Data served Policy determined

Blue Coat Systems, Inc. Documentation and Education Services

13

Best Practices: ProxyAV and ProxySG

Technical Brief

14

Blue Coat Systems, Inc. Documentation and Education Services

Appendix B: ProxySG and ProxyAV Appliance Possible Errors

During an ICAP transaction, three types of errors can occur: Proceedable errorsThese are ProxyAV-detected errors that can block or serve depending on the AV appliance configuration. UnProceedable errorsThese errors always result in a ProxyAV-generated block. Errors generated by the ProxySGThese errors are related to a failure in the communication to the AV appliance and obey the fail_open and fail_closed configuration only.

Proceedable Errors
The ProxyAV proceedable errors are:
Scan timeout Decode error Password protected Insufficient space Maximum file size exceeded Maximum total size exceeded Maximum total files exceeded Internal error

When a proceedable error occurs, the ProxyAV can either block or serve. A ProxyAV block means that an ICAP 500 error is returned to the ProxySG appliance. A ProxyAV serve means that an ICAP 200 or 204 is returned to the ProxySG, along with an X-Error-Code header.

Unproceedable Errors
The ProxyAV unproceedable errors always return an ICAP 500 error, making them equivalent to the block behavior. These errors are:
File extension blocked Antivirus load failure Antivirus license expired Antivirus engine error

Errors Produced by the ProxySG

Errors produced by the ProxySG are usually because of communication failures with the ProxyAV. These errors are:
Connection failure

Blue Coat Systems, Inc. Documentation and Education Services

15

Best Practices: ProxyAV and ProxySG

Technical Brief

Request timeout Server unavailable Server error

Following these errors, the ProxySG fail_open and fail_closed configuration fully controls the behavior with regard to data or error served.

16

Blue Coat Systems, Inc. Documentation and Education Services