Big data architecture and infrastructure(EN/NL/FR)

This 1 day ABIS course gives a general picture of big data and what it represents: an overview is given of the technologies on which it is based, and the frequently heard technological terms which we need to get acquainted with are placed in context and perspective. Open to anybody interested in Big Data - to attend, elementary knowledge of database management systems is an advantage.

Remark: Course description in English; Dutch and French versions are available on the ABIS website. Courses are planned in Dutch, English, and French. Consult the ABIS website for alternate course formats.

Description:

This course gives a general picture of big data and what it represents: an overview is given of the technologies on which it is based, and the frequently heard technological terms which we need to get acquainted with are placed in context and perspective.

Main topics:

• Introduction: about data, databases, and data warehouses - and now big data
• What is big data?
  • Perspective: problem formulation - why big data?  (1) data centric management (2) the 4 Vs: volume, variety, velocity, variability - types of data - examples (3) data quality, consistency, and reliability (veracity)
  • Big data architecture - components - technologies - towards an integrated data architecture
• Overview of new data sources: web statistics ("click streams"); social media; Twitter feeds; Google Maps; sensor data (e.g. surveillance cameras) ant the Internet of Things (IoT); ...
• NoSQL databases versus relational databases - types and use - and popular today: MongoDB, Cassandra, ...
• Big Data Frameworks
  • The "divide & conquer" model: Hadoop and MapReduce - distribute data and analyse it through massively parallel algorithms
  • Spark: in-memory hence speed - supporting a plethora of data sources
  • Machine learning
• Performance considerations
• Big data analytics - know your data -- or: the role of the data scientist!
  • How to judge data quality; risk analysis - and the importance of statistics
  • Use of programming languages: Python, R, Scala, ...
  • Use of visualisation tools in order to keep an overview and to estimate the relative importance of the different data sources
• Overview of often used (open source) products/technologies on the market

Intended for/Audience: The course is designed for everybody who wants to learn about big data: IT personnel, people confronted with big data technologies. Also for non (IT) technical collaborators.

Background/Prerequisites: Elementary knowledge of database management systems is an advantage.

Training Method/Didactics: Classroom training.

Duration: 1 day.