Roma +39 0644254074 - Milano +39 0240031289 info@eurokleis.com
Menu

Data Lake: new data store scenario

The Challenge

The Big Data revolution redefined how business data is managed. On the one hand the revolution exponentially increased information by opening up the scope for new analytical skills, but on the other it led to a technological acceleration, driven above all by the open source world, towards tools such as Apache, Hadoop and Spark and technologies that handle vast amounts of data.

The new data-storage scheme which is cloud-based structures data in a scalable way and allows you to carry out an endless amount of experiments on data.

In October 2010, James Dixon, CTO of Pentaho, a business intelligence company, introduced for the first time the term data lake on his blog with the following definition:

If you think of a datamart as a store of bottled water – cleansed and packaged and structured for easy consumption – the data lake is a large body of water in a more natural state. The contents of the data lake stream in from a source to fill the lake, and various users of the lake can come to examine, dive in, or take samples.

The fundamental difference between traditional storage structures such as data silos or data warehouse and a data lake is that a data warehouse stores data in a structured and rigid way, whereas data lake uses a flat architecture. Each element receives an identifier and a set of metadata to fit it. The data lake can therefore be interrogated in the search of relevant data, which can then be analysed for specific results.

The data lake allows you to overcome the problem of the rigidness of the Data Silos and DWHs design and prevents the proliferation of structured databases. Instead of multiple data sources which manage themselves independently, you can opt for data lake whose use facilitates and speeds up sharing of information while simultaneously promoting the reduction of server-licensed costs.

The Solution

Eurokleis has a long-standing consulting experience for customers who handle large amounts of data and operate in the telecommunications, ICT and air transport sectors.

Eurokleis has the skills to guide the customer in choosing the best solution through every scenario: Data Silos, ODS (Operational Data Store), DWH, or Data Lake.

The Eurokeis team offers the following services:

  • Feasibility studies
  • Assessment
  • Analysis of requirements and features
  • Technological audit
  • Design of technical architectures
  • Structured database design
  • Realisation of data lake
  • Change management and knowledge transfer

Digital Business: new data store scenario

In a context where data flows continuously from countless sources, information systems must allow them to archive and make information available at any time necessary.

This requires technological efficiency that eliminates latency between data demand and availability as well as infrastructure efficiency that allows quick reconfiguration to ensure prompt access to data using new real-time variables.

All the Big Data figures

source Gartner

%

organisations who are unsure of how to get value from the Big Data

%

believes that defining a strategy for Big Data is one of the main obstacles

%

states that Big Data skills are rare and expensive

Data Warehouse vs Data Lake

Data

Data Warehouse: Structured data, organised and managed batches of information while applying business rules

Data Lake: Structured, semi structured, non structured and raw data

Processing

Data Warehouse: Schema on write

Data Lake: Schema on read

Archiving

Data Warehouse: Expensive for large quantities of data

Data Lake: Designed to maintain low costs

~

Security

Data Warehouse: Mature

Data Lake: Maturing

Agility

Data Warehouse: Not very agile, initial configuration is difficult to customise

Data Lake: Very agile, possibility to modify and re-modify configuration as needed

Author of the article

Luca Satolli

senior partner & founder - Technology

IT Manager with transverse skills acquired over time

Background Experience Passion

You may also be interested in the following articles:

Facilitated Finance: The New Sabatini Law 

Access to credit is critical to the company’s competitiveness. New in financial leasing for SMEs and micro-enterprises for the purchase of new machinery, plant and equipment

Business Model Development
A company is only successful if it creates value. Business modeling is therefore essential to effectively manage your business idea.