Название: The Digital Journey of Banking and Insurance, Volume III: Data Storage, Data Processing and Data Analysis
Автор: Volker Liermann, Claus Stegmann
Издательство: Palgrave Macmillan
Год: 2021
Страниц: 278
Язык: английский
Формат: pdf (true), epub
Размер: 27.8 MB

This book, the third one of three volumes, focuses on data and the actions around data, like storage and processing. The angle shifts over the volumes from a business-driven approach in “Disruption and DNA” to a strong technical focus in “Data Storage, Processing and Analysis”, leaving “Digitalization and Machine Learning Applications” with the business and technical aspects in-between. In the last volume of the series, “Data Storage, Processing and Analysis”, the shifts in the way we deal with data are addressed.

Three major trends—especially in external digitalization—are driving the data handling process: Trend A: increase in available data, Trend B: accelerated speed in data processing, Trend C: special structures for optimized storage and querying of complex and unnormalized3 data structures. These three trends are mirrored in technologies: Trend A is reflected in the new cluster databases like Hadoop (or AWS-S3 , Google Bigtable ), making the handling of high data volumes possible and affordable. Trend B is shown in streaming technology (like Kafka , see Steurer) making real- or near-time data provision possible. Streaming technology was in place long before the digital transformation sped up, and the first steps with Kafka Standalone faced several challenges. The modern architecture concepts like Lambda , Kappa and Delta architectures (see Kratz and Morawski, Data Infrastructures—Lambda Architecture and Other Architectures) combine traditional architecture patterns providing stability with the dynamism and speed of streaming technology. Other facets of Trend B are in-memory databases (IMDB ), making it feasible to handle huge data volumes in the blink of an eye. Trend C materializes in different specialized databases (like document-based databases and graph databases) as well as in distributed ledgers.

Data availability and data technology stimulate each other continuously. The internet has made mass data available for almost every important (and unimportant) subject. The volume forced Google to develop a concept to deal with such amounts of data: the BigTable and the MapReduce concept as a part of the Google File System. The availability of technology (especially as a cost-efficient open-source implementation) then opens up for other Big Data processing use cases, such as customer clustering analysis or (when including the time dimension) the prediction of a customer journey.

Driven by the business requirements, topic-specific database variants like graph databases or other NoSQL databases (document store, key-value store, object database, …) have been established in the market. There is no perfect NoSQL database. Every type of database has advantages and disadvantages depending on the subject it is applied to. The evolution of specific types of database shows the demand for application-specific database types (in-memory DB, cluster DB, graph DB, document DB, …). Once the technology is implemented and available (ideally as open source) new use cases are mapped, and sometimes surprising applications can arise from a tool in the right hands. For example: graph databases are used in the context of anti-money laundering (AML) to analyze connected persons and accounts. This application is certainly not the most obvious application for graphs (nodes and edges), but formulation of the challenge AML as a graph delivers stable and reliable results.

Contents:
Part I. Big Data and Special Databases
Data Lineage
Digitization and MongoDB - The Art of Possible
Graph Databases
Data Tiering Options with SAP HANA and Usage in a Hadoop Scenario
Part II. Streaming
Kafka: Real-Time Streaming for the Finance Industry
Architecture Patterns - Batch and Real-Time Capabilities
Kafka—A Practical Implementation of Intraday Liquidity Risk Management
Part III. data: A View of Meta Aspects
Data Sustainability - A Thorough Consideration
Special Data for Insurance Companies
Data Protection - Putting the Brakes on Digitalization Processes?
Part IV. Distributed Ledger
Digital Identity Management - For Humans Only?
Part V. Machine Learning and Deep Learning
Overview Machine Learning and Deep Learning Frameworks
Methods of Machine Learning
Summary