Data Transfer report files provide non-aggregated, event-level data from your ad campaigns. This data is essentially raw content from the Ad Manager ad server logs, with a separate file generated for each type of event. Data Transfer files contain event data that is accurate to the second, and you can choose to include other information in the files to see device, geography, and other information related to the event. Partners need to approve ad units in Ad Manager to show in their Data Transfer files.
Get started
Data Transfer report files come at an additional cost. Contact your account manager to enable the feature. After the feature is enabled, you can set up your Data Transfer file types.
If your organization is not able to manage extract, transform and load (ETL) processing, support large files, manipulate text files, design and administer a mid-sized data store, and design and implement scripts, consider working with an approved Google Marketing Platform partner.
Available Data Transfer report files
Each Data Transfer file contains information about different events. You can add fields to each file type to see contextual information related to those events.
Backfill file, the Network file doesn't include impressions served from Ad Exchange or AdSense via dynamic allocation. Use the Backfill file for information on dynamically-allocated impressions.About the data contained in Data Transfer files
- Late data
Delays of a few hours are normal, but occasionally Data Transfer files take longer than usual to process. If data is late, it appears in the next hourly batched file with an accurate time stamp. This could mean, for example, that a file has mostly 8 a.m. to 9 a.m. timestamps with a scattering of earlier timestamps if processing was delayed. -
Hours with no activity
If there is no activity during a given hour, an empty Data Transfer file that only contains header data is posted. - Date and day boundaries
The first hourly file for a given day typically contains events from midnight to 1 a.m. Pacific Time, but the event timestamps are in the publisher's network time zone. If, for example, the publisher is set to Eastern Time, they'd see events from 3 a.m. to 4 a.m. in the first hourly file. The three hours before that would actually be in the previous day's file. You might therefore have timestamps from a different date than is represented by the file name. Always refer to the timestamp on events in the file, not the time the file is posted or name of the file. - Master/companion reporting in Data Transfer
Data Transfer files show both master and companion creative impressions but the query tool only counts an impression served for the master/companion creative set (not an impression for each creative) unless information is broken down by master/companion.
IsCompanion is “TRUE” for the Companion creative impression. The CreativeId field contains the individual creative IDs for the Master and Companion creatives and not the Creative Set ID. There isn’t an additional field in Data Transfer for Creative Set ID to associate companion impressions to master impressions. - Discrepancies with Ad Manager reporting
Bad traffic, or spam data, is periodically removed from Ad Manager reports or API-generated reports. Due to the publishing schedule of Data Transfer files, some of this cleanup may not be reflected. This can result in Data Transfer showing slightly more impressions, clicks, or other events. When discrepancies occur, the extent tends to be ~1%.Your
CodeServesData Transfer files might show a higher value than the "Code served count" metric in Ad Manager reports because Data Transfer counts a code serve for every served impression, but reports only count one code serve per request. Some creative formats, such as Google Ads text ads, can contain multiple impressions for a single request. Group your Data Transfer files by theKeyPartfield values to find the unique impressions for comparison.
Download a sample file
You can download these sample files to preview the data and fields contained in each of your Data Transfer report files.
| Event type | File name and type | What it shows | Sample file |
|---|---|---|---|
| Joinable Bids | NetworkBackfillBidsJoinable |
Information about bids. Joinable with other Data Transfer files. Once per hour. |
|
| CodeServe | NetworkCodeServes |
Records every response from Ad Manager, whether downloaded or not. Twice per hour. Note: The Request and CodeServe files are included with the Impression file at no additional cost. |
|
| Impression | NetworkImpressions |
Information about downloaded impressions. Twice per hour. |
|
| Request | NetworkRequests |
Records every ad request received by Ad Manager, whether filled or unfilled. Twice per hour. Note: The Request and CodeServe files are included with the Impression file at no additional cost. |
|
| Active view | NetworkActiveViews |
Information about Active View-eligible Ad Manager-based impressions. Twice per hour. |
|
| Bids | NetworkBackfillBids (Beta) |
Information about Authorized Buyers and Open Bidding auction bids. Once per hour. |
|
| Click | NetworkClicks |
Information about clicks. Twice per hour. |
|
| Minimum bid to win | NetworkMinimumBidToWin (Beta) |
Reports the minimum bid buyers needed to win the auction when at least one remnant line item is competing. Once per hour. |
|
| Rich Media conversion | NetworkRichMediaConversions |
Information about Studio events, including both standard and custom actions (like, play and pause), action duration, and more. Twice per hour. |
|
| Video conversion | NetworkVideoConversions |
Information about video-specific events, including actions (like, play and pause), content IDs, pod positioning, and more. See all video events. Twice per hour. |
How files are delivered
Data Transfer files are pushed to Ad Manager cloud storage buckets on an hourly basis. We advise polling at regular intervals to check for updates. The majority of the data is delivered and available between 5 and 15 hours after the recorded hour, though some delayed events can take up to 8 days after the event occurred. Ad Manager does not deliver data transfer information to third-party servers.
File names include the start hour for events in US Pacific time zone (observing Daylight saving time), but the timestamps present in that file are always given according to the Ad Manager network time zone (which might not observe Daylight saving time). This can lead to empty/skipped files or files containing more than one hour's worth of data, depending on the interplay between these time zone settings.
File names
Data Transfer file names follow a predictable convention: [Type]_[Network ID]_[YYYYMMDD]_[HH].gz
YYYYMMDDis the year, month, and date.HHis the start hour in 24-hour format.The hour number (01, 02, 03) specified in each file name is in Pacific timezone, but publishers get their own network timezone-specific data from the timestamps contained within the Data Transfer files. Be aware of this difference when you calculate file delivery.
Data Transfer files in the ad request process
Review the following graphic to understand the Data Transfer file types associated with various stages of the ad request process.
Use Data Transfer report files
Once you've set up Data Transfer, files are kept in Ad Manager cloud storage buckets. You can access them on the web, with a command line tool, or through an API. Learn more about how to access Ad Manager cloud storage buckets
If you limit your data ingestion and analysis to a specific set of Data Transfer files based on the start hour in the file name, you might overlook data that's provided in a subsequent file because of daylight savings time, late data collection, or other similar scenarios. A better approach is to read all the Data Transfer files into a separate system (such as a data warehouse or query engine) and restrict your analysis based on the timestamp of the events.
Data Transfer files are saved as .CSV files that you can convert using a spreadsheet editor. Here are some examples of typical ways you may choose to apply Data Transfer information:
- Dimensional user aggregation of events and activities (for example, per creative)
- Calculate unique conversions across multiple days
- Match users against a customer database
- Report on user geographic and demographic information
You can use match tables to provide a name-to-ID lookup for values contained within Data Transfer files, allowing you to match ad serving information (such as ad unit or line item) to pre-assigned values stored in the database.
Store files locally
A good rule of thumb is that each event uses between 25 and 35 bytes in a compressed file. As such, 10 million impressions would require about 300 MB of disk space in a compressed file. Keep in mind that these are estimates, and your file size could be somewhat larger. Also, because this is the size of the data in compressed form, you need additional space to decompress and use the files.
Data Transfer files older than 60 days are purged from Ad Manager. If you would like to store your files for longer than the allotted 60 days, we recommend that you either store the files locally or move to a permanent cloud storage solution, which can include an independent Google Cloud Storage account over which you have full control.
Learn about the BigQuery Data Transfer Service
Make large Data Transfer files easier to process
Google Code has released CRUSH (Custom Reporting Utilities for Shell), an open-source toolkit for processing delimited-text data from the command line or in shell scripts. The CRUSH tools have been extensively developed and tested, and they work best on Linux or Unix operating systems. Support for CRUSH is available through the open-source community.
A non-open-source alternative is DMX, a data integration software developed by Syncsort.
