Open Catalog Interface (OCI) - Manual For Open Icecat XML and Full Icecat XML
Open Catalog Interface (OCI) - Manual For Open Icecat XML and Full Icecat XML
Open Catalog Interface (OCI) - Manual For Open Icecat XML and Full Icecat XML
Icecat XML
This document describes the XML method of Icecat’s Open Catalog Interface (OCI): a set of
standards and file formats used in the Icecat XML repository. In summary, it gives the
basics of how to use the Icecat repository or how to publish data to Icecat conforming the
OCI.
Further, check out the Add-On page for popular platforms already integrating Icecat data,
which we regularly update.
Are you interested in adding product reviews? Then check out our reviews manual.
Brand owners that look into pushing data into Icecat, please, look into our Push-API or for
manual data-entry Icecat’s Free Vendor Central.
2. The links and files in the download directories are to be downloaded with scripts (from
server to server). If you try to download them via your browser, you may sometimes
experience memory problems. This does not mean that on our side the data is not accessible
or correctly available. In case you experience any problem, contact us via the contact form.
3. We show by default a daily index file when you only enter the directory name. This,
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 1
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
because it is more efficient for testing and viewing in a browser. To get the complete index
file with products for Open Icecat users and all described products for Full Icecat users,
include in the path the full file name of the complete index: files.index.xml (in Open Icecat
or Full Icecat).
After a rip of our general site, an IP address may be temporarily blocked. A rip is not
acceptable as it may hurt site performance and can be a copyright infringement.
However attractive the free Open Icecat database is, the Full-Icecat database still contains
many, many more brands: 10,000s (see https://icecat.biz/menu/vendors/index.htm). If you
want specific brands to be included into free Open Icecat, please, recommend Icecat to your
manufacturer contacts.
Read more about the main advantages of Full Icecat over free Open Icecat and see the
Icecat Susbcription Plans.
In case you want to upgrade to Full Icecat, please contact us via the contact form, or click
on the “request Full Icecat access” link after the User Login.
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 2
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
to mention explicitly the “Specs Icecat” on all product datasheets, with a link to the
Icecat.biz web site
to include explicitly an “AS IS” disclaimer, an example of which can be found
at https://www.icecat.biz/menu/disclaimer/index.htm (available in several languages).
Coverage analysis
To decide to make use of Full Icecat or Open Icecat it may be important to analyze the
coverage of your portfolio or the portfolio of your supplier. You can always ask us to give
you an overview of the coverage of (stock) portfolio of your suppliers (i.e., distributors) at
any given moment. We strive to cover at least 90% of the stock portfolio (stock > 1) of
mainstream distributors and work together with partners to monitor this coverage daily.
1) Register and upload your catalog to Icecat via the Login. In the match analysis you get
the match percentages back for Full Icecat and Open Icecat. And you can get a list of the
unrecognized products. An advantage is that you can also optionally display your offers -
using this method- on the Icecat.biz price comparison sites. NOTE: if you follow this method,
Icecat also provides you automatically with your private Icecat index.
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 3
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
2) If you do in-depth coverage analyses, please use the following matching file, which is a
plain dump of everything in our catalog: https://data.Icecat.biz/prodid/prodid_d.txt.gz (a
very large file with all known products in the Icecat database, described and standardized
or not in gzipped archive). NOTE: Market presence=Y in the matching file indicates that we
notice that the product is currently present in at least one distributor’s catalog.
Quality=Icecat means that Icecat editors have standardized the data-sheet.
Quality=SUPPLIER means that the data-sheet comes from a manufacturer’s CMS (or a
brand editor created a product data-sheet manually via Icecat PIM), and might be queued
for standardization. Quality=NOEDITOR means that our editors did not create a data-sheet
yet, nor did we auto-import manufacturer data; so, it is only the raw data as we received it
from channel partners.
TIP: discuss with the Icecat team a service level for your portfolio.
1. Icecat Repositories
The Full Icecat XML repository is a set of product data files and an index file, which includes
all pointers to individual product files and gives some basic information as timestamp
updated, content quality level, etc. The Icecat XML repository contains product information
for products of all brands, and categories that subscribers want us to support. The current
content can be best reviewed on the public Icecat website
The Open Icecat repository is the part of the Icecat XML repository that is sponsored by
manufacturers and is provided for free to the channel. Open Icecat is distributed under the
Open Content License Agreement. Which brands are included in Open Icecat? The most
recent list is here. Open Icecat is also useful during development and testing.
2. Directories
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 4
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
2.1 Access
To obtain access to the repository, you will need a login/password, provided to you by
Icecat. If you don’t have these, please register online for free Open Icecat. NOTE: During
the registration you need to choose for the Live (url, JS) or Data (xml, json, csv) version. If
you want to make use of XML, make sure that you choose the Open Icecat Data
subscription. You can always change this afterwards, or create a second subscription for the
Live version (URL links to hosted data-sheets) method. For Full Icecat access, a separate
contract is needed. You can request this contract here.
Take care that you provide us with the right IP addresses of your content servers, as the
access is secure. You can test the login/password combination by entering it by hand in the
BASIC HTTP authorization pop-up screen, which appears automatically when the directory
URL is entered. In this case, we need to have added the IP address of your workplace as
well to your profile. If you don’t know this IP address, just type in your browsers
https://www.whatismyip.com/ or https://www.getip.com/ and fill it in the IP addresses field
after Login via the Icecat website (multiple IPs need to be entered space separated).
Of course, for professional use, server access need to be set up. In case of little experience
in this field, we advise you to make use of the URL versions (links to pictures and
datasheets) or ask us for assistance.
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 5
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
https://data.Icecat.biz/xml_s3/xml_server3.cgi?prod_id=<prod_id>;vendor=<ve
ndor_name>;lang=<lang>;output=<output>
or
https://data.Icecat.biz/xml_s3/xml_server3.cgi?ean_upc=<EAN or
UPC>;lang=<lang>;output=<output>
Where:
prod_id – the manufacturer product code of the requested product, vendor_name – name
of the manufacturer of the requested product, ean_upc – the EAN or UPC code to identify a
product,
lang – the language code (INT – for the international/standardized repository, EN, NL, FR…
– for local ones),
output – the type of the response. It can be productxml (the product xml file) and metaxml
(the index or meta information on the Brand + ProdID/M_Prod_ID or EAN/UPC).
For example, if you need to get an xml file of product ‘WM527 mouse Bluetooth Laser 1600
DPI Right-hand’. Manufactured by DELL, with product number WM527-BK, in English, the
call will be:
https://data.Icecat.biz/xml_s3/xml_server3.cgi?prod_id=WM527-BK;vendor=dell;lang=en;ou
tput=productxml
In case you want to retrieve data on the basis of an EAN or UPC code:
https://data.Icecat.biz/xml_s3/xml_server3.cgi?ean_upc=5397063929863;lang=en;output=p
roductxml
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 6
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
https://data.Icecat.biz/xml_s3/xml_server3.cgi?prod_id=WM527-BK;vendor=dell;lang=en;ou
tput=metaxml
XML Server 3 access works via the basic HTTP authorization, also used for other HTTP
requests.
TIP 1: Use the Presentation_Value parameter in localized data-sheets for display of a data-
sheet, and use the standardized Value parameter for search & compare on standardized
values.
Please, note, where QUALITY = SUPPLIER only the original product data as imported from
a manufacturer’s CMS is available, the respective data-sheet is not yet standardized by our
editors. When standardized, the parameter changes to QUALITY = Icecat.
Please, note that this language-specific content contains the standardized content (same as
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 7
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
in INT), which is entered by our editors. Or, if no standardized data is yet available, it
contains just the original data (Quality=Supplier) imported from the CMS of a manufacturer
or entered manually via the Icecat PIM by the manufacturer.
In general, we follow the ISO 639-1 two-character country codes, except when a code can
not be unambiguously tied to a language. So we use for example EN_SG, and EN_IN to
identify Singapore English and Indian English respectively
TIP: we advise the language-specific product content (Presentation_Value) purely for the
display of product data-sheets, not for search & comparison. For search & comparison, we
advise the standardized product content of the Value parameter from the /INT or local
language directory. We advise LocalValue for country-specific search & compare
applications.
TIP: Open Icecat is also useful for testing purposes before you upgrade to Full Icecat. The
technical structures are 100% compatible with those of Full Icecat.
Please, note, where QUALITY = SUPPLIER only the original product data as imported from
a manufacturer’s PIM is available, the respective data-sheet is not yet standardized by our
editors. When standardized, the parameter changes to QUALITY = ICECAT.
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 8
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
In general, we follow the language codes as given by the ISO 639-1 standard. However, in
some cases, we choose the simpler two-character country codes. For example, when it’s
obvious to which language code it relates (e.g., BR for Brazilian-Portuguese, or US for US
English).
About mod_deflate: mod_deflate transports our interface files as compressed gzip data, but
will show it transparantly in its original format in modern browsers.
To benefit in your server scripts from gzip, you must use in any HTTP request this additional
parameter:
Accept-Encoding: gzip
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 9
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
Example:
Host: prf.Icecat.biz
Accept-Encoding: gzip
User-Agent: Firefox/1.0
Here is example code in C# to download files. You can use it also in ASP.NET code. We
tested it and it works ok:
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 10
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
req.Credentials = cache;
return fileData;
In Classic ASP, MSXML2.ServerXMLhttp can be used Some details are present here
3.1 Purpose
The purpose of the index files is to find the right product XML files for a given product. The
matching between any portfolio and Icecat catalog has to be based on unique product
identifiers and Icecat support few options:
Brand + Product code – The “key” is always Brand name + Product code. The
combination of these identifiers should give a perfect hit.
GTIN (EAN / UPC) – These numerical identifiers are also available in XML index files
at Icecat and can be used as a unique identifier for matching a product. Indication
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 11
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
“IsApproved=1” means that we received a GTIN either from the manufacturer or from
three independent distributor feeds. In all other cases, we tag a GTIN as
“IsApproved=0”.
Brand + Distributor Product code – Icecat Catalog has an extensive and
dynamically updated database of Product code mappings between product identifiers
that comes from different distributors all over the world and the original manufacturer
product codes. It opens an ability to make a matching by using a distributor Product
code (M_Prod_Id) in combination with a brand name.
daily.index.xml or .csv– a smaller index file with only references to the new or changed
product data-sheets of the respective day
on_market.index.xml or .csv – an index file with only the products that are currently on
the market, as far as we can see that based on 100s of distributor and reseller price files
Tip: use the on_market index file, in case that you think the full index file is too big for you
to process or too inefficient. DISCLAIMER: if our monitor for your market is incomplete, the
on_market index file is probably too incomplete for your purposes.
nobody.index.xml or .csv – an index file with the products that are or were on the market
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 12
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
for which we only have basic market data, but no complete data-sheet.
Tip: In case that you want to recategorize your products, for which we have no data-sheet
(QUALITY=NOEDITOR), make use of the nobody index file
Important note:
The structure of complete files.index.xml and daily.index.xml are identical for a user’s
convenience.
Files include the index information for each product and gives the path to the XML product
file. Below is an example of a file tag in the index file:
The On_Market attribute indicates that a product is still on the market (On_Market=“1”) or
not seen on the market anymore (On_Market=“0”).
Per product, Icecat provides also the GTIN (EAN or UPC) codes, used for logistical
purposes. It is possible that there are multiple GTINs (EANs or UPCs), as this varies with
packaging or importers.
Further, we provide the Product code variants, used in the market, in a sub-list
(M_Prod_ID).
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 13
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
Finally, the <Country_Market Value=”US”/> indicates that we see the product at the
moment in the catalogs of US channel partners. This tag can be useful to limit your view on
the index file, in case you don’t have distributor imports to do that.
The Model_Name attribute contains the product name. The Product_View attribute
indicates how many times the current product is requested.
We refresh files.index.xml and daily.index.xml every day. You can use daily.index.xml to
update only the data in your local database that has changed or is new in the Icecat
database. files.index.xml, you can use the first time for a full import of needed product
information of active data-sheets for Open Icecat users and all existing data-sheets for Full
Icecat users, and for the regular analysis of coverage of your product database.
Historic and obsolete products can be downloaded via the real-time interface.
TIP 1: download ONLY the xml files that you actually need for your shop or application by
using a daily catalog export from your ERP or shop (or actually the daily imports from your
distributors/suppliers) as a filter. It is useless to import all the global product data files if
you only operate in one geographic area or have an assortment limited to a certain
category.
TIP 2: check the timestamps to only download files that have changed or check only for new
files, to improve update performance cycles.
TIP 3: the daily index file also includes the list of product codes that are removed the day
before (QUALITY=REMOVED). We remove product data-sheets, for example, when editors
map them away. You can use it for cleansing your catalog.
Each product has its own data file in XML format. Its location is defined by the attribute
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 14
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
“path” in the *.index.xml.gz. The file content includes all the available product information
for the respective product.
The “Prod_ID” attribute is a manufacturer’s unique identifier for a product, often called
Brand Product Code (Manufacturer Part Number, code de fabricant, Produkt-Kode,
artikelnummer, etc). In combination with the “supplier_id” (i.e., the brand identifier) it is the
unique key to select a product’s XML file via Icecat’s internal identifier (“Product_id”).
M_Prod_ID consists of one or more manufacturer product code(s) that are present in
distributor or partner feeds but are mapped to the original (correct) Prod_ID.
The EAN_UPC sub-list lists the GTIN codes (EAN or UPC) that refer to this product. There
can be multiple codes, as these are logistical codes depending on packaging types and
geography.
On_Market (1=Yes, 0=No) indicates whether Icecat sees a product somewhere on the
market or not. The Country_Market lists the individual markets (countries), where Icecat
sees that the product is present. This indication can be used to limit the view on our index
file.
DISCLAIMER: Our market view is always imperfect as we depend on the data of connected
channel partners and distributors.
Catid is Icecat’s internal category identifier. For example Catid=”151” refers to Notebooks.
For every category there is a second, external, category identifier based on UNSPSC. In the
reference files or the product XML the meaning of categories can be found.
SUPPLIER – This content is received from a brand’s PIM or entered manually by a brand
editor via the Icecat PIM, and is not yet standardized by an Icecat editor. The language-
specific directories are likely to contain the full (unstandardized) data-sheet.
ICECAT – This content is entered and/or standardized by Icecat editors. Such standardized
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 15
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
NOEDITOR – The content is received from a merchant (in most cases one of the 100s of
distributors we are daily “polling”) and may be parsed. Editors haven’t described this
product yet. Icecat doesn’t export the NOEDITOR data in XML to 3rd parties.
4. Additional data
Within this file you’ll find the match between different manufacturer names variations and
the original manufacturer name.
The format is self explanatory. The DTD for the message format is is available at:
https://data.icecat.biz/dtd/ICECAT-supplier_mapping.dtd
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 16
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
To help you correct the MPNs from different distributors and to help you to get the best
coverage, we are publishing the available MPN mappings to the official MPN. In every
repository directory, the file with product mappings is published in the index files
(files.index.xml, etc.).
INT/product_mapping.xml
EN/product_mapping.xml
etc…
4.3 References
In product XML feeds, you will find a lot of structures, which are included as references in
the product data files, e.g. categories, features, measures (units), etc.
The reference files are present in the /refs/ subdirectory. For example, for Full Icecat, at
https://data.Icecat.biz/export/level4/refs/ and for Open Icecat at
https://data.Icecat.biz/export/freexml/refs/
CampaignsList.xml
CategoriesList.xml.gz
CategoryFeatureIntervalsList.xml
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 17
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
CategoryFeaturesList.xml.gz
DistributorList.xml.gz
FeatureGroupsList.xml.gz
FeatureLogosList.xml.gz
FeaturesList.xml.gz
LanguageList.xml.gz
LogisticsDataList.xml
MeasuresList.xml.gz
RelationsList.xml
SupplierProductFamiliesListRequest.xml.gz
SuppliersList.xml.gz
Almost all files are present in Gzipped format. The typical use of the files is in combination
with data from the INT (international) directory, not the localized directories as there the
language-specific values are already automatically applied to the product XML files
Brand products can have campaigns for specific period. CampaignsList.xml has the list of all
assigned campaigns.
Example:
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 18
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
This chunk provides information on Icecat categories. Each category may be either
searchable or not. Searchable categories may be used for product lookup by product feature
values, as Icecat editors systematically maintain data integrity to allow smart searches.
Searchable=”1″: This category may be used for product lookup in product list lookup
request
Searchable=”0″: This category is not made searchable (in our own product finder tools)
“Score” attribute in the response reflects the category usage statistic. The higher number
means the higher the usage level.
Example:
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 19
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
This chunk provides information about features that are assigned to every single category
and which are used for describing products in the category.
Each searchable feature will have LimitDirection attribute defined, which will give advice
for finding a “better” feature value, either < or >. E.g. for feature “Hard disk capacity” it
would be > relation, and for the feature “Write seek” it would be < relation.
Possible feature values may be restricted to a limited set of values. In that case possible
values for the combination Category-Feature are listed in container RestritedValue.
Example:
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 20
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
Example:
Example:
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 21
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
Icecat has dynamic relations (x-sells). They based on the product sets, manually defined.
One product set relates to another one.
Example:
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 22
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
This chunk provides the list of supplier product families. Each product family may have
information about it’s Name, Description in different languages, and category.
From Supplier list users can identify if brand belongs to Full or Open Icecat.
Example:
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 23
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
We log all Icecat product requests (real-time requests by end-users and xml requests). This
data gives a product popularity indication, useful for sorting products and analyzing market
trends. This popularity data is daily published on:
https://data.icecat.biz/export/level4/data_prod_stat.xml
https://data.icecat.biz/export/level4/<lang>/product_overview.txt
https://data.icecat.biz/export/level4/<lang>/product_overview.txt.gz
Where <lang> stands for the language code. Column 1 is the Icecat ID, and column 2 and 3
give a short and a long standardized product summary description.
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 24
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
TIP: Think about the guarantees that you get from your suppliers (distributors), based on
errors made in their product descriptions. It may still be smart to make use of these supplier
(distributor) texts in the ordering process for this reason.
https://data.icecat.biz/dtd/ICECAT-interface_response.dtd
Please, note that the tag “No=” (example: No=”100090″) is a Priority indicator. The higher
the number the more important the feature or feature group is considered to be for buyer
orientation. Thus, we advise you to sort features and feature groups by priority, to get a
more customer-friendly user presentation.
Product code is a deprecated field, earlier it used as product response status. Now, product
code always equals 1.
You can find the basic information about XML Schema here. Icecat XSD sources are located
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 25
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
Notes:
LocalValue is a local transformation of our international Value attribute. It applies the local
dictionary to Value and transforms values according to local standards. As in Value attribute
LocalValue contains value of the feature and measurement unit as a separate attributes.
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 26
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full
Icecat XML
See for a full overview: Digital Asset Types as Represented in Icecat’s Open Catalog
Interfaces.
Vazha Abramishvili
Open Catalog Interface (OCI): Manual for Open Icecat XML and Full Icecat XML | 27