ESSNET on SDMX II

WP2: PC-Axis SDMX Integration

Tutorial of the process using PX-file to SDMX for CensusHub

2012-05-21

Type of Document Reference: Issue: Created by: Updated by: Approved by:

Project deliverable WP 2; 2d Presentation and information for

other countries
Revision: Lars Nordbck Status: Date: Draft 2 2012-05-26

Document Change Record

Analysis and Design of the Test Client v 0.1

Date Updated: 6/7/2013 2:34:14 a6/p6 Prepared by: TC, IB

Page ii of 23 Page ii Reviewed by:

[Title]

Table of contents

Page

1 Introduction........................................................................................................................................ 4 1.1 Purpose............................................................................................................................................ 4 1.2 Reference Documents and Standards.............................................................................................. 4 2 Description of process for CensusHub reporting from PC-Axis files...........................................6 2.1 Workflow........................................................................................................................................... 6 2.2 Interaction with SDMX_RI modules.................................................................................................. 7 3 Creation of the example.................................................................................................................... 8 4 Preparing the input for the Census HyperCube 6...........................................................................9 5 Preparing to handle the output of HyperCube 6...........................................................................13 6 Using Census HyperCube 6 in the PC-Axis file based process for Eurostat CensusHub........15

List of figures

Page

Figure 2-1: Process for CensusHub reporting from PC-Axis files...................................................6

List of tables

Page

Table 1-1: Terms and Abbreviations................................................................................................... 5

Analysis and Design of the Test Client Version 0.1

Date Updated: 6/7/2013 2:34:14 a6/p6 Prepared by: IB, TC, SL

Page 3 of 23 Page iii Reviewed by: SLI

[Title]

1
1.1

Introduction
Purpose

In this paragraph will be described the purpose of the document and how the document is structured The purpose with this document is to give the participants in the Essnet for SDMX Workshop in Rome 25-27 a documentation of the processes involved when using PC-Axis files as the base for the use of Eurostat Reference Architecture in the CensusHub project.

1.2

Reference Documents and Standards

Insert all references to documents or convention or standard used in the document. This part can be divided in: Standards and Conventions Documents or standard named in the document Statistics Sweden documents PC-Axis file format document: http://www.scb.se/Pages/List____314011.aspx Attributes in the PX-file; New keywords for attributes on observation values, Petros Likidis PX-Web Systems Diagram, Mikael Nordberg PX-Web powerpoint presentation, Mikael Nordberg Eurostat RI documents On CIRCA
SDMX - training material SDMX Self-learning tutorials

Mapping Assistant Installation Guide of the Mapping Assistant.doc Mapping Assistant Design and Implementation Guide for Programmers.doc Mapping Assistant User Manual and Tutorial.doc Test Client No instructions found NSI Web Service Installing_NSI_Web_Service_(.NET).doc
Analysis and Design of the Test Client Version 0.1 Date Updated: 6/7/2013 2:34:14 a6/p6 Prepared by: IB, TC, SL Page 4 of 23 Page 4 Reviewed by: SLI

Web Client Analysis of Web application for exposing browsing_dissemination environment of an NSI v0.7.doc Installation Guide of the Web Application for exposing_browsing dissemination env of NSI (.NET) v0.8.doc User Manual of Web application for exposing browsing_dissemination environment of an NSI v0.4.doc Terms and abbreviations Terms or abbreviations used in the document Acronym SCB RI CIRCA SSD PX-file PX-Edit Nordic Data Model PC-Axis file Definition Statistics Sweden Eurostat Reference Infrastructure Eurostat website for access to documents and software Statistics Sweden Statistical Database on the Website PC-Axis file Software for creating PC-Axis file from different input file formats Data model used by the countries using PC-Axis SQL based software Open file format based on keywords for metadata and data

Table 1-1: Terms and Abbreviations

Definitions: Definitions used in the document

Description of process for CensusHub reporting from PC-Axis files

In this chapter will be described the example providing a description of the whole workflow. This chapter should be a sort of short User Manual in which is described how to use the example.

2.1

Workflow

In this paragraph should be described the workflow of the example. In a picture should be shown the modules used of the SMDX-RI and the new modules created for the purpose of the example.

Statistics from a material made of the PC-Axis file with the program PX-Edit version 3. 1. Statistical production produces statistics including attributes on cell level to be included in the PCAxis file database. 2. Subsequently, the statistical product load statistics in PX-Edit and enter metadata including Metadata for Attributes on data cell level. 3. Check of the entered data in the database using PC-Axis SQL or the SSD Web interface. 4. Based on Eurostats description of the statistics, Data Structure Definition (DSD), which is stored in the local database Mapping Store, a mapping of the SCB material, can be made against what is in the DSD for the codes, etc. using the program Mapping Assistant. The resulting mapping is then stored in the Mapping Store. 5. To make sure the material is correct you can use the Test Client that creates an SDMX file of the statistics. 6. Publishing is executed based on normal copying to the external PC-Axis directory This routine is not using RSS for notifying because Census data is only posted once. 7a and 7b. Eurostat software downloads then the statistics that dynamically is transformed into SDMX format. In this case the Statistical database is outside the inner Firewall and security of the database has to be monitored.

Figure 2-1: Process for CensusHub reporting from PC-Axis files

Explanation of the staining of components in the image above: Yellow: Applications and databases developed by Eurostat called Eurostat Reference Infrastructure, Open Source software. Green: Program developed by Statistics Finland, freeware. Gray: Existing standard applications and databases for the dissemination of statistics by Statistics Sweden. Pink: Programs that are used by the product to produce the file for loading the statistics in the Statistical Database. Brown Pink: Product production database.

2.2

Interaction with SDMX_RI modules

In this paragraph must be indicated the modules that interact with the SDMX-RI No software module is interacting with the SDMX-RI. The interface is the PC-Axis file.

Creation of the example

In this chapter will be described the step done in order to realize the example. The example can use one or more modules belonging to the SDMX-RI, some of this modules can be modified and new code can be added in order to realize the example.

The example is the CensusHub Hypercube number 6.

Preparing the input for the Census HyperCube 6

In this chapter a description of the input must be provided

The input in the process is a txt file for Hypercube 6 in the CensusHub project. A subset follows below:

The Attributes on Cell level are found after the pipe character | in the end of the lines that contains such an attribute. In this case P,P;,Source: SCB on the fifth data line and NULL|CNI,P on the ninth data line. On this fifth line there are also marked two textual attributes expressed with the extra pair of -characters. This file is put into the PX-Edit software to make it a PC-Axis file.

In PX-Edit the table is exposed as above where you can find the red corners in data line 5 and 9 corresponding to the cell attributes in the txt file above. Beneath the Table Metadata parts concerning the attributes are found on the three bottom lines.

10

On the following cell note metadata screen you also can find the attributes as they are picked up in PX-Edit from the txt-file.

11

The resulting PC-Axis file looks like this:

Notice the attribute keywords ATTRIBUTE-ID and the cell related keywords ATTRIBUTES including the cell position for the expressed attributes.

12

Preparing to handle the output of HyperCube 6

In this chapter a description on how to handle the output must be provided

In PX-Web it is possible to see the tables including the Attributes on cell level, this development is ongoing. See http://groups.google.com/group/pcaxis/topics?hl=en
Attributes on cell level in PX - Solution proposal.pdf 98K View Download Attributes in the PX file format.pdf 216K View Download

13

14

Using Census HyperCube 6 in the PC-Axis file based process for Eurostat CensusHub
Text-file, Census Hyper Cube 6, see chapter 4 above. PX-Edit, see chapter 4 above. PC-Axis file, see chapter 4 above. PX-Web including Attributes on cell level for visual checking, see chapter 4 above. Mapping Assistant mapping a PC-Axis file, see below. Test Client, see below. Web Client, see below.

This chapter should be a sort of short User Manual in which is described how to use the example.

15

Mapping Assistant mapping a PX-file

Data Flow

16

DDB Connections

Data Set

17

Mapping Set

Header Info

18

View Mapped data

Test Client

19

Web Client Web Client main window

Selecrtions

20

View Results

View result as a table

21

22

View results download in HTML

23