W3C

Common User Agent Problems

Once Upon A Time, A User Agent...

W3C Note 28 January 2003

This version:
http://www.w3.org/TR/2003/NOTE-cuap-20030128
Latest version:
http://www.w3.org/TR/cuap
Previous version:
http://www.w3.org/TR/2001/NOTE-cuap-20010206
Translations of this document:
http://www.w3.org/QA/translations#cuap
Editor:
Karl Dubost, W3C
Authors and contributors:
See Acknowledgments.

Copyright © 2003 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark, document use and software licensing rules apply.

Abstract

This document explains some common mistakes in user agents due to incorrect or incomplete implementation of specifications, and suggests remedies. It also suggests some "good behavior" where specifications themselves do not specify any particular behavior (e.g., in the face of error conditions). This document is not a complete set of guidelines for good user agent behavior.

This document does not incriminate specific user agents. W3C does not generally track bugs or errors in implementations. That information is generally tracked by the vendors themselves or third parties.

Status of this document

Publication status

This document is an update of an already published Note, published on January 28th, 2003, and made available for discussion only by the editor and authors as part of their work as W3C Team participants in the Quality Assurance Activity. Publication of this Note by W3C does not imply endorsement by W3C, including the W3C Team and Membership.

This document may be updated, replaced, or obsoleted by other documents at any time.

Comments

No formal commitment is made by W3C to invest additional resources in topics addressed by this Note. However, comments are welcome and the W3C Quality Assurance Team may publish an amended version should the amount and quality of the received comments prove it worthwhile or necessary. Though some of the previous comments has been added to this version. Some of them are still in discussion and might be added in a future version. We plan to publish in the next few months a new and improved version of this document to have the same organization than the CHIPS note. The problems related to XHTML, DOCTYPE and namespaces will be addressed in this future version.

Please send comments to the publicly archived mailing-list of the Quality Assurance Interest Group: www-qa@w3.org.

A list of acknowledged errors and proposed corrections can be found at http://www.w3.org/QA/2002/12/cuap-errata.

Translation

Translation of this document is welcome. However, before starting a translation of this document, please be sure to read the information on translations, in our Copyright FAQ, and check the list of existing translations of this document (available at http://www.w3.org/QA/translations#cuap). .

Other W3C Technical Reports and publications

A list of current W3C technical reports and publications, including Working Drafts and Notes, can be found at http://www.w3.org/TR/.

Table of Contents

Introduction

This document explains some common mistakes in user agents (browsers, spiders, etc.) due to incorrect or incomplete implementation of specifications, and suggests remedies. It also suggests "good behavior" where specifications themselves do not specify any particular behavior (e.g., in the face of error conditions).

This document only deals with the client-side aspect of HTTP, people looking for HTTP implementation problems in Web servers should have a look at the Web server counterpart of this document: Common HTTP Implementation Problems [CHIPS].

This document does not address accessibility issues for user agents. Please refer to W3C's User Agent Accessibility Guidelines 1.0 [UAAG10] for information on how to design user agents that are accessible to people with disabilities.

Scope of this document

This document is a set of known problems and/or good practices for user agents implementations and their use, aimed at:

Unless specifically mentioned, what is referred throughout this document as HTTP is RFC2616, a.k.a. HTTP/1.1 [RFC2616].

Conformance to this document

This document is informative.

This document has no conformance per se, but since it is about implementation of normative specifications (such as the HTTP/1.1 specification), or their use, one should consider following the guidelines and checkpoints described here as a good step toward conformance to these normative specifications.

As often as possible, references will be mentioned for each checkpoint.

This document uses RFC 2119 [RFC2119] keywords (capitalized MUST, MAY, SHOULD etc.) when referring to behaviors clearly defined by a normative specification. When not capitalized, these words should be interpreted as regular language and not as RFC2119 keywords.

1. Usability

This section focuses on the user's experience, including customization, user interface, and other usability issues. Some of the checkpoints suggested here depends on the user agents used and can be sometimes not applicable in terms of implementations.

1.1 When the user follows a link to a target anchor, highlight the target location.

Techniques:

References:

1.2 If the user attempts to follow a link that is broken because it designates a missing anchor, let the user know it is broken.

There are many ways to indicate to the user that a link is broken. The recommended behavior is as follows:

Wrong: Some user agents scroll to the top or bottom of the document when the user attempts to follow a broken link. This behavior is discouraged since it is indistinguishable from the correct behavior when a target is at the beginning or end of a document.

References:

1.3 Allow the user to retrieve Web resources even if the browser cannot render them.

User agents may not be able to render certain types of content on the Web either natively or through a plug-in (e.g., XML content, XSLT style sheets, RDF documents, DTDs, XML schemas, etc). User agents should allow users to retrieve and save these resources, otherwise users may not be able to access this Web content at all.

1.4 When the user requests to print a frameset, allow the user to select to print an individual frame or the frameset.

The presentation of the frameset could be achieved, for example, by:

Note: The authors do not encourage Web content developers to use frames as they can cause many usability and accessibility problems.

References:

1.5 Add support for new URI schemes in a straightforward way.

For instance, allow users to associate external programs with URI schemes. The user agent should inform the user when it does not recognize a URI scheme in content.

Example:

A user may want the "tel" scheme (e.g., tel:+33-4-12-34) to interact with their telephone. Or they may want the "irc" scheme (e.g., irc://irc.example.org/) to activate an IRC client on their desktop with a connection to the specified server.

Wrong: Some user agents ignore the scheme part (before the ":") when the scheme is unknown to them, interpret the colon character as though it were encoded as '%3A' and then treat the URI as though it were a relative URI, usually producing a broken link (and confusing users).

References:

1.6 Allow the user to override any mechanism for guessing URIs or keywords.

Many user agents compensate for incomplete URIs by applying a series of transformations with the hope of creating a URI that works. For example, many user agents transform the string www.w3.org into the URI http://www.w3.org/. The user should be able to control whether, for example, typing a keyword should invoke a Web search or whether the user agent should prepend http://www. and append .org/.

1.7 Warn users about incomplete documents and transfers.

Rendering an incomplete document as though it were complete is very likely to confuse users. Part of the document is missing, hence some anchors might not be present, possibly breaking some links. The user agent should notify the user that the document is incomplete.

The HTTP/1.1 specification describes this behavior for caches at the protocol level. Partial responses should also be made obvious to the user with a warning.

References:

1.8 Provide a mechanism to allow authentication information to expire.

Many browsers allow configuration to save HTTP authentication [RFC2616, RFC2617] information ("remember my password"). They should also allow users to "flush" that authentication information on request. For instance, the user may wish to leave the user agent running but tell it to forget the password to access the user's bank account.

Wrong: Most user agents consider that authentication information (e.g., password) provided by a user for a server/realm pair during a session is immutable for the duration of the session.

1.9 When a Web resource includes metadata that may be recognized by the user agent, allow the user to view that metadata.

Metadata – data about data – can provide very useful context to users about information on the Web. For instance, metadata about a book might include the book's author, title, publication date, publisher, etc. (refer to the Dublin Core [DC] for information about library-type metadata). Authors include metadata in HTML documents through a variety of elements and attributes (e.g., the TITLE and ADDRESS elements, the "alt", "title", and "summary" attributes, etc. Languages such as the Resource Description Framework [RDF] allow users to populate the Web with rich metadata. User agents should provide a user interface to allow users to view metadata. The user interface may vary according to the underlying markup language. For instance, many graphical browsers render the HTML "title" attribute (e.g., as a tool-tip) when the user selects or hovers over an element with that attribute specified.

References:

1.10 Allow the user to keep track of completed HTTP POST requests.

Users may wish to track and archive HTTP POST requests for the same reasons they wish to track and archive email. For instance, if the user places a book order through a form, and that form uses a POST request, the user should be able to store information about that transaction.

References:

1.11 Allow the user to bookmark negotiated resources.

The HTTP/1.1 protocol [RFC2616] allows the client to request a representation of a resource which is best suited to its needs (language, media type, etc); this mechanism is called "content negotiation".

When a resource is negotiated, the user might want to bookmark a particular version. For example, a document might be available in several languages under the same URI, and the user might want to point somebody to the Canadian version of this document, which has a different URI.

In such a case, it should be possible to bookmark either the original URI or the URI of the view that the user got. The original URI can be interpreted as being the generic object and the retrieved document as one view of this object.

Bookmarking a particular version of a negotiated resource is not always possible under HTTP semantics, because a) the particular version may not have its own URI and b) even if it does, HTTP does not guarantee that the user agent will be informed of this.

HTTP/1.1 defines the Content-Location header field as the way for the server to indicate the URI of the variant, and some servers do supply this Content-Location when negotiation took place most of the time. However, Content-Location is also used for some other things, and its inclusion in a response does not necessarily mean that content negotiation took place.

References:

1.12 Support time-saving transfer encoding mechanisms and send out TE headers announcing their support.

HTTP/1.1 [RFC2616] allows transfer encoding. An example of encoding is data compression, which speeds up Web browsing over a slow connection.

The HTTP/1.1 transfer encoding negotiation mechanism has been designed to avoid the need for the end user to get involved. Using the HTTP protocol, the server, proxy, and client implementations among themselves will be able to choose and use the most efficient transfer encoding. The more you support such mechanisms, the better it is.

Users might have enough knowledge or have help of user interfaces to fine-tune this process beyond what can be done automatically. The user agent should allow the user to set the transfer encoding in the HTTP requests sent out.

References:

1.13 Use the user interface language as the default value for language negotiation.

The user should be allowed to specify the set of languages that the user agent may use for language negotiation.

In case the user does not specify any language, the user agent may specify the language of its user interface as the preferred language, while allowing other languages with a lower preference, for example by sending

Accept-Language: dk, *;q=0.5

References:

1.14 Only advertise an encoding in Accept-encoding that you really accept.

A number of web sites suffer from bandwidth overload. By altering the server side scripting engine to support encoding compression or by inserting a compressing proxy, it is possible to dramatically reduce the operating costs. The down side is that a number of user agents advertise that they can handle gzip or deflate when they really are unable to do it.

References:

1.15 Remember traverse redirects

When the browser traverses a redirect, it should remember both the original URI and the target URI for marking links as visited.

2. Rendering

This section focuses on issues related to style sheets and link types.

2.1 Implement user style sheets. Allow the user to select from author and user style sheets or to ignore them.

A style sheet is a set of rules that specifies how to render a document on a graphical desktop computer monitor, on paper, as synthesized speech, etc. A document may have more than one style sheet associated with it, and users should be able to select from alternative style sheets.

References:

2.2 Respect media descriptors when applying style sheets.

Some markup and style sheet languages allow authors (e.g., @media construct in [CSS2], media attribute in [HTML 4.01]) to design documents that are rendered differently according to the characteristics of the output device: whether graphical display, television screen, handheld device, speech synthesizer, braille display, etc.

References:

2.3 If a CSS style sheet is missing, ignore it and continue processing.

Users must be able to view content even without CSS style sheets.

Wrong: In some user agents, missing style sheets result in a fatal error or result in the user agent not rendering content.

References:

2.4 Implement the HTML 4 recognized link types.

Section 6.12 of the HTML 4.01 Recommendation [HTML 4.01] lists some link types that may be used by authors to make assertions about linked Web resources. These include alternate, stylesheet, start, next, prev, contents, glossary, and others. Although the HTML 4.01 specification does not specify definitive rendering or behavior for these link types, user agents should interpret them in useful ways. For instance, the start, next, prev, and contents link types may be used to build a table of contents, or may be used to identify the print order of documents, etc.

3. Protocols implementation

This section focuses on the implementation of network protocols used to download resources from the Web.

3.1 Save resources retrieved from the Web on the local system using the appropriate system naming conventions.

The media type of a resource retrieved by HTTP [RFC2616] is determined by the content type and encoding returned by the server in the response headers.

If the user wants to save a resource locally, the user agent should respect the system naming conventions for files (e.g. PNG images usually have a .png extension).

Example:

http://www.w3.org/TR/1999/REC-html401-19991224/html40.ps is a view of the gzip'ed PostScript version of the HTML 4.01 specification. The HTTP headers sent by the server include:

Content-Type: application/postscript; qs=0.001
Content-Encoding: gzip

If saved locally, the filename on most computers should be html40.ps.gz for the applications to recognize the file type.

Wrong: Saving this compressed PostScript document as html40.ps is likely to confuse other applications.

References:

3.2 Respect the media type of a resource if one is explicitly given using the Content-Type HTTP header.

Example:

If an HTML document is returned with a Content-Type value of text/plain, the user agent must NOT render the document with another guessed Content-Type (like, for example, text/html).

Reference:

3.3 Respect the character set of a resource when one is explicitly given.

User agents must respect the character set when it is explicitly specified in the response. The character set can be given by the HTTP Content-Type headers and/or by the document-internal fallback (HTML meta element, etc).

References:

3.4 Do not treat HTTP temporary redirects as permanent redirects.

The HTTP/1.1 specification [RFC2616] specifies several types of redirects. The two most common are designated by the codes 301 (permanent) and 302 or 307 (temporary):

Wrong: User agents usually show the user (in the user interface) the URI that is the result of a temporary (302 or 307) redirect, as they would do for a permanent (301) redirect.

References:

3.5 If a host name has multiple DNS entries, try them all before concluding that the Web site is down.

Many Web sites have a single hostname like www.example.org resolve to multiple servers for the purpose of load balancing or mirroring. If one server is unreachable, others may still be up, so browsers should try to contact all the servers of a Web site before concluding that the Web site is down.

For example, the libwww implementation does it the right way and check the response time of all ip address, once done it sorts all the address to get the best one. An example is available in the open source implementation of the Domain Name Service Class.

3.6 List only supported media types in an HTTP Accept header.

HTTP/1.1 [RFC2616] defines content negotiation. The client sending out a request gives a list of media types that it is willing to accept; the server then returns a representation of the object requested in one of the specified formats if it is available.

When entities are embedded in a document (such as images in HTML documents), user agents should only send Accept headers for the formats they support.

Example:

If a user agent can render JPEG, PNG and GIF images, the list of media types accepted should be image/jpeg, image/png, image/gif.

Wrong: User agent agents should not send an HTTP header of Accept: */* since the server may support content types that the user agent does not. For instance, if a server is configured so that SVG images are preferred to PNG images, a user agent that only supports PNG, GIF, and JPEG will receive (unsupported) SVG rather than (supported) PNG.

Note: Some user agents send a Accept header that has '*/*' at the end, after all of the supported content types. This way, the server is free to send the resource in any format, which can then be processed by the user with another tool.

References:

4. URI handling

Resources are located on the Web using Uniform Resources Identifiers [RFC2396]. This section discusses how user agents should handle URIs.

4.1 Handle the fragment identifier of a URI when the HTTP request is redirected.

When a resource (URI1) has moved, an HTTP redirect can indicate its new location (URI2).

If URI1 has a fragment identifier #frag, then the new target that the user agent should be trying to reach would be URI2#frag. If URI2 already has a fragment identifier, then #frag must not be appended and the new target is URI2.

Wrong: Most current user agents do implement HTTP redirects but do not append the fragment identifier to the new URI, which generally confuses the user because they end up with the wrong resource.

References:

Example:

Suppose that a user requests the resource at http://www.w3.org/TR/WD-ruby/#changes and the server redirects the user agent to http://www.w3.org/TR/ruby/. Before fetching that latter URI, the browser should append the fragment identifier #changes to it: http://www.w3.org/TR/ruby/#changes.

Acknowledgments

The editor would like to thank the following W3C Team members for the initial input that led to the creation of this document. Hugo Haas has been the main author of the first version of this document:

The editor would also like to thank the following people for their review of the document:

References

CC/PP
"Composite Capability/Preference Profiles (CC/PP): A user side framework for content negotiation", Franklin Reynolds, Johan Hjelm, Spencer Dawkins, Sandeep Singhal, 27 July 1999. Available at http://www.w3.org/1999/07/NOTE-CCPP-19990727/.
CHIPS
Common HTTP Implementation Problems, Olivier Thereaux, 28 January 2003. Available at http://www.w3.org/TR/2003/NOTE-chips-20030128/. Latest version available at http://www.w3.org/TR/chips.
CSS2
"Cascading Style Sheets, Level 2", Bert Bos, Håkon Wium Lie, Chris Lilley, Ian Jacobs, 12 May 1998. Available at http://www.w3.org/TR/1998/REC-CSS2-19980512/.
DC
Dublin Core. Available at http://www.dublincore.org/.
HTML 4.01
"HTML 4.01 Specification", Dave Raggett, Arnaud Le Hors, Ian Jacobs, 24 December 1999. Available at http://www.w3.org/TR/1999/REC-html401-19991224/.
PURL
"Introduction to Persistent Uniform Resource Locators", Keith Shafer, Stuart Weibel, Erik Jul, Jon Fausey. Available at http://purl.oclc.org/OCLC/PURL/INET96.
RDF
"Resource Description Framework (RDF) Model and Syntax Specification", Ora Lassila, Ralph R. Swick, 22 February 1999. Available at http://www.w3.org/TR/1999/REC-rdf-syntax-19990222/.
RFC1630
"Universal Resource Identifiers in WWW", T. Berners-Lee, June 1994. Available at http://www.ietf.org/rfc/rfc1630.txt.
RFC2119
"Key words for use in RFCs to Indicate Requirement Levels", S. Bradner, March 1997. Available at http://www.ietf.org/rfc/rfc2119.txt
RFC2396
"Uniform Resource Identifiers (URI): Generic Syntax", T. Berners-Lee et al., August 1998. Available at http://www.ietf.org/rfc/rfc2396.txt.
RFC2616
"Hypertext Transfer Protocol -- HTTP/1.1", R. Fielding et al., June 1999. Available at http://www.ietf.org/rfc/rfc2616.txt.
RFC2617
"HTTP Authentication: Basic and Digest Access Authentication", J. Franks et al., June 1999. Available at http://www.ietf.org/rfc/rfc2617.txt.
RFC2717
"Registration Procedures for URL Scheme Names", R. Petke et al., November 1999. Available at http://www.ietf.org/rfc/rfc2717.txt.
RURL
"Handling of fragment identifiers in redirected URLs", B. Bos, 30 June 1999. Available at http://www.ics.uci.edu/pub/ietf/http/draft-bos-http-redirect-00.txt.
SCHEMES
"An Index of WWW Addressing Schemes", Dan Connolly, 2000. Available at http://www.w3.org/Addressing/schemes.
SCHEMES-IANA
"Uniform Resource Identifier (URI) SCHEMES", IANA. Available at http://www.iana.org/assignments/uri-schemes.
SELECTORS
"Selectors", Daniel Glazman et al., 13 November 2001. Available at http://www.w3.org/TR/2001/CR-css3-selectors-20011113/.
UAAG10
"User Agent Accessibility Guidelines 1.0", Jon Gunderson, Ian Jacobs, 17 December 2002. Available at http://www.w3.org/TR/2002/REC-UAAG10-20021217/.
UAWP
"Axioms of Web architecture: User Agent watch points", Tim Berners-Lee, 1998. Available at http://www.w3.org/DesignIssues/UserAgent.
XML-STYLE
"Associating Style Sheets with XML documents Version 1.0", James Clark, 29 June 1999. Available at http://www.w3.org/1999/06/REC-xml-stylesheet-19990629/.