WEB MINING НА ОСНОВІ ЛЯМБДА АРХІТЕКТУРИ

Фінчук О. В.

doi:10.31435/rsglobal_ws/12072018/6004

WEB MINING НА ОСНОВІ ЛЯМБДА АРХІТЕКТУРИ

Фінчук О. В. associate professor, Україна, м. Київ, Національний технічний університет України «Київський політехнічний інститут ім. І. Сікорського», ФІОТ, кафедра ОТ, студент

DOI: https://doi.org/10.31435/rsglobal_ws/12072018/6004

Keywords: web mining, web scraping, lambda architecture, big data, batch processing, stream processing, cluster, apache kafka

Abstract

Amount of data in the web is growing rapidly from day to day and for normal human it's hard to realize what is actually happening in the world at the current moment, so it is required to have a system which allows collect and analyze huge amount of variety data from different web resources. The system should be able to process a lot of data fast and be capable to store all collected data for future processing. How to achieve this with use of lambda architecture is described in this article.

References

Big Data: Principles and best practices of scalable realtime data systems 1st Edition, 2015. -328с.

Patterns of Enterprise Application Architecture 1st Edition, 2002. – 560c.

lambda-architecture-with-apache-spark [Електронне джерело] – режим доступу: https://speakerdeck.com/mhausenblas/lambda-architecture-with-apache-spark?slide=1

A repository dedicated to the Lambda Architecture [Електронне джерело] – режим доступу: http://lambda-architecture.net/

High Performance Spark Best Practices for Scaling and Optimizing Apache Spark, 2017-175c.

Hadoop: The Definitive Guide, 4th Edition, 2015. -235c.

Testing with JMeter [Електронне джерело] – режим доступу: https://jmeter.apache.org/

Views:

264

Downloads:

186

Published

2018-07-12

Citations

How to Cite

Фінчук О. В. (2018). WEB MINING НА ОСНОВІ ЛЯМБДА АРХІТЕКТУРИ. World Science, 2(7(35), 4-8. https://doi.org/10.31435/rsglobal_ws/12072018/6004

Download Citation

Issue

Vol 2 No 7(35) (2018): World Science

Section

Computer Science

This work is licensed under a Creative Commons Attribution 4.0 International License.

All articles are published in open-access and licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0). Hence, authors retain copyright to the content of the articles.
CC BY 4.0 License allows content to be copied, adapted, displayed, distributed, re-published or otherwise re-used for any purpose including for adaptation and commercial use provided the content is attributed.