Data Management Challenges

Our society is good at collecting data, but managing is very poor to make enough sense from it.

Simply collecting data is not enough; enterprises and organizations need to understand from the

start that data management and data analytics only will be successful when they first put some

thought into how they will gain value from their raw data. They can then move beyond raw data

collection with efficient systems for processing, storing, and validating data, as well as effective
analysis strategies.

Another challenge of data management occurs when companies categorize data and organize it

without first considering the answers they hope to glean from the data. Each step of data collection

and management must lead toward acquiring the right data and analyzing it in order to get the
actionable intelligence necessary for making truly data-driven business decisions.

Data Management Best Practices

The best way to manage data, and eventually get the insights needed to make data-driven decisions,

is to begin with a business question and acquire the data that is needed to answer that question.

Companies must collect vast amounts of information from various sources and then utilize best

practices while going through the process of storing and managing the data, cleaning and mining
the data, and then analyzing and visualizing the data in order to inform their business decisions.
 It’s important to keep in mind that data management best practices result in better analytics. By

correctly managing and preparing the data for analytics, companies optimize their Big Data. A few
data management best practices organizations and enterprises should strive to achieve include:

 Simplify access to traditional and emerging data

 Used data to infuse quality into existing processes

 Shape data using flexible manipulation techniques

 Linkability the data can be linked to wider scope of areas of human need

 Discoverability can be discover by many data users

 Trust infuse quality

 Interoperatability

 Processability

It is with the help of data management platforms that organizations have the ability to gather, sort,

and house their information and then repackage it in visualized ways that are useful to marketers.

Top performing data management platforms are capable of managing all of the data from all data

sources in a central location, giving marketers and executives the most accurate business and
customer information available.

Benefits of Data Management and Data Management Platforms

 Managing your data is the first step toward

handling the large volume of data, both structured and unstructured, that floods businesses daily.

It is only through data management best practices that organizations are able to harness the power
of their data and gain the insights they need to make the data useful.

In fact, data management via leading data management platforms enables organizations and
enterprises to use data analytics in beneficial ways, such as:

 Personalizing the customer experience

 Adding value to customer interactions

 Identifying the root causes of marketing failures and business issues in real- time

 Reaping the revenues associated with data-driven marketing

 Improving customer engagement

 Increasing customer loyalty

Further Reading on Data Management

For further information on data management, visit our blog. For your convenience, we have linked
to three of our most recent data management posts below:
 Big Data Analysis Learning Resources: 50 Courses, Blogs, Tutorials, and More For Mastering Big

Data Analytics

 Business Analytics Learning Resources: 50 Top Resources for Getting the Most Out of Your

Analytics

 Top Customer Analytics Blogs: 50 Awesome Blogs on Gaining Deeper Customer Analysis

Insights

Molly Galetto March 31, 2016 What is Data Management?

https://www.ngdata.com/what-is-data-management/

Many datasets are essentially short-term thus market data, production, consumption data, and
weather data are good examples. These data are meaningless unless their timeframe for them and
they are updated regularly. Soil data, for example, are similar, even if not updated as often as
weather and market data. They change by location and during the course of – and between –
seasons.
In order to build and maintain trust in any data obtained for use by both indivual and organsizations
such data must necessary have stable data management principles and best practices in place. Good
data management principles help to ensure that data produced or used are registered, stored,
made accessible for use and reuse, managed over time and/or disposed of, according to legal,
ethical, funder requirements and good practice. Therefore data consumers, trust depends the
following factors:

 Knowing the source. Trust in data begins with knowledge of its source. That is who is the
data provider, where is the data obtained, at what period does the data was obtained, what
method was used to generate the data?
  Trusting the source. If you know that data comes from a trusted source, then you can rely
on it, and on the conclusions, you draw from it.
 Timeliness of the data. Even when from a trusted source, data is not useful if it is outdated.
 Data quality. Trusted data must accurately and precisely reflect what it measures.
 Sustainability. A trusted dataset must have some guarantee of availability.
 Discoverability. Like documents, data is only useful if it is straightforward to find.
 Documentation and support. Data users should be able to access support for data if needed.
 Interaction. Data users should be able to provide feedback if there is a problem with data.
Data management therefore, is a process involving a broad range of activities from administrative
to technical aspects of handling data in a manner that addresses the factors listed above. A sound
data management policy will define strategic long-term goals for data management across all
aspects of a project or enterprise.

A data management policy is a set of high-level principles that establish a guiding framework
for data management. A data management policy can be used to address strategic issues such as
data access, relevant legal matters, data stewardship issues and custodial duties, data acquisition,
and other issues. As it provides a high-level framework, the data management policy should be
flexible and dynamic.

Quality as applied to data has been defined as fitness for use or potential use. Many data quality
principles apply when dealing with species data and with the spatial aspects of those data. These
principles are involved at all stages of the data management process, beginning with data
collection and capture. A loss of data quality at any one of these stages reduces the applicability
and uses to which the data can be adequately put.
All of these affect the final quality or fitness for use of the data and apply to all aspects of the
data. Data quality standards may be available for:
● accuracy
● precision
● resolution
● reliability
● repeatability
● reproducibility
● currency
● relevance
● ability to audit
● completeness
● timeliness.
Data quality is assessed by applying verification and validation procedures as part of the quality
control process. Verification and validation are important components of data management that
help ensure data is valid and reliable.

Components of a data management plans

1. Administrative information
 Name and ID of the project
 Project Description
 Funding body/bodies
 Project Data Contact
 Related Policies
 Date of First Version
 Date of Last Update

2. Data collection
 Data description, including anticipated type, format and volume
 Existing datasets to be re-used
 Methods by which data will be collected or created
 Structures, naming and versioning system for folders and files
 Quality assurance processes

3. Documentation and metadata

 A list of information you expect will be needed for the data to be read and interpreted in
the future
 How you plan to collect or create this documentation and metadata
 The metadata standards you will use
4. Some examples of data documentation:
 Laboratory notebooks & experimental protocols
 Questionnaires, codebooks, data dictionaries
 Software syntax and output files
 Information about equipment settings & instrument calibration
 Database schema
 Methodology reports
  Provenance information about sources of derived data Add detailed descriptions for
collections or files, e.g. what is in a file, where did it come from, how could it be
retrieved if needed, any existing problems etc.
5. Ethics and legal compliance Ethics
 Details of consent needed for data preservation and sharing
 Steps to be taken, if needed, to protect the identity of any participants
 Steps to be taken, if needed, to ensure sensitive data is stored and transferred securely
6. Copyright and Intellectual Property Rights
 Name(s) of the owner(s) of the data
 Licence(s) for re-use which will be applied (e.g. one of the licences available from
Creative Commons or Open Data Commons)
 Restrictions on third party use
 Any expected delay to data sharing e.g. pending a patent application or embargo related
to publication in a journal

7. Storage and backup

 Where (physically) data will be stored
 Backup provision
 Person or team responsible for backup
 Recovery procedures
8. Security
 Risks, and how they will be managed
 Access arrangements
 Any arrangements, if needed, for safe and secure transfer of data collected in the field
9. Selection and preservation
 Details of which data should be retained, shared and/or preserved, with particular
reference to contractual, legal or regulatory requirements
 Foreseeable research uses for the data
 Length of time for which data will (or should) be kept beyond the life of the project
 The repository or archive where the data will be held, and any associated charges
 Time and effort needed to prepare data for preservation / data sharing
10. Data sharing responsibilities and resources
 Named person responsible for implementation of the data Management plan
 Named person responsible for each data management activity
 Hardware and software required (any that is additional to existing institutional provision)
 Additional specialist expertise or training required
 Charges to be applied by data repositories
Data organisation and Documentation
Data files and folders need to be labelled and organised in a systematic way so that they are both
identifiable and accessible for current and future users. The benefits of consistent data file
labelling are:
 Data files are distinguishable from each other within their containing folder
 Data file naming prevents confusion when multiple people are working on shared files
 Data files are easier to locate and browse
 Data files can be retrieved not only by the creator but by other users
 Data files can be sorted in logical sequence
 Data files are not accidentally overwritten or deleted
 Different versions of data files can be identified
 If data files are moved to another storage platform their names will retain useful context.

There are three main criteria to consider regarding the naming and labelling data files, namely:
 Organisation: important for future access and retrieval, and needs to take into account the
file naming constraints of the system where the file is located
 Context: this could include content specific or descriptive information, independent of
where the data are stored
 Consistency: choose a naming convention and ensure that the rules are followed
systematically by always including the same information (such as date and time) in the
same order (e.g. YYYYMMDD).

Common elements of a file naming strategy

 Version number
 Date of creation
 Name of creator
 Description of content
 Name of team/department/unit associated with the data
 Publication date
 Project number.

Open Data
Management in
Agriculture, Nutrition
and Land November 2018

An image is a visual representation of something (a picture). An audio file makes a sound when
played. A video combines multiple images with audio to make a moving picture. Data is… difficult
to define and thus provide search for.
Data is the lowest level of abstraction from which information and knowledge are derived. These
are abstract terms and thus data could be an image, or spreadsheet, or audio file. Additionally, if
the web is an information space then data is something which is of a lower level?
Traditionally, data is thought of as a spreadsheet or set of numbers that can be analysed in some
way. On the web, such data is often shared via data portals simply as a file that can be downloaded.
Some portals provide YouTube-like functionality where the data can be explored without
downloading, however the data itself is still a static resource, uploaded ready to be downloaded by
someone else.