Hadoop ve Spark sistemlerin karşılastırmalı çalışması

Thumbnail Image

Files

449314 A comparative study of Hadoop and Spark frameworks.pdf (1.54 MB)

Date

2016

Authors

Journal Title

Journal ISSN

Volume Title

Publisher

Research Projects

Organizational Units

Thumbnail Image
Organizational Unit
Software Engineering
(2005)
Department of Software Engineering was founded in 2005 as the first department in Ankara in Software Engineering. The recent developments in current technologies such as Artificial Intelligence, Machine Learning, Big Data, and Blockchains, have placed Software Engineering among the top professions of today, and the future. The academic and research activities in the department are pursued with qualified faculty at Undergraduate, Graduate and Doctorate Degree levels. Our University is one of the two universities offering a Doctorate-level program in this field. In addition to focusing on the basic phases of software (analysis, design, development, testing) and relevant methodologies in detail, our department offers education in various areas of expertise, such as Object-oriented Analysis and Design, Human-Computer Interaction, Software Quality Assurance, Software Requirement Engineering, Software Design and Architecture, Software Project Management, Software Testing and Model-Driven Software Development. The curriculum of our Department is catered to graduate individuals who are prepared to take part in any phase of software development of large-scale software in line with the requirements of the software sector. Department of Software Engineering is accredited by MÜDEK (Association for Evaluation and Accreditation of Engineering Programs) until September 30th, 2021, and has been granted the EUR-ACE label that is valid in Europe. This label provides our graduates with a vital head-start to be admitted to graduate-level programs, and into working environments in European Union countries. The Big Data and Cloud Computing Laboratory, as well as MobiLab where mobile applications are developed, SimLAB, the simulation laboratory for Medical Computing, and software education laboratories of the department are equipped with various software tools and hardware to enable our students to use state-of-the-art software technologies. Our graduates are employed in software and R&D companies (Technoparks), national/international institutions developing or utilizing software technologies (such as banks, healthcare institutions, the Information Technologies departments of private and public institutions, telecommunication companies, TÜİK, SPK, BDDK, EPDK, RK, or universities), and research institutions such TÜBİTAK.

Journal Issue

Abstract

Bu çalışma, büyük veri sistemlerden ikisi olan Hadoop ve Spark'ı karşılaştırarak incelemektedir. Bu çalışmaya dahil edilen sistemler ideal performanslarını gösterebilecek şekilde çeşitli parametrelerle ayarlanmış; performans, donanım kullanımları ve kullanılan donanım sayısının arttırılmasıyla oluşan hızlanma oranları dört farklı uygulamada değerlendirilmiştir. Değerlendirme sonuçları incelendiğinde Spark'ın WordCount ve Sort uygulamaları için daha iyi performans gösterdiği, Hadoop'un Naive-Bayes uygulamasında Spark'tan daha başarılı olduğu gözlenmiştir. Bununla beraber, Grep uygulaması için Hadoop ve Spark arasında göze değer bir fark görülmemiştir.
In this thesis, Apache Hadoop and Apache Spark are compared with respect to performance, resource usage, and speedups of four different Big Data workloads. The comparison is done by tuning the parameters with various combinations for the optimum performance of each framework. The evaluations show that, Spark outperforms Hadoop for WordCount, and Sort workloads. However, Hadoop outperforms Spark in Naive-Bayes workload, and there is no a remarkable difference in the performances for Grep workload.

Description

Keywords

Bilgisayar Mühendisliği Bilimleri-Bilgisayar ve Kontrol, Computer Engineering and Computer Science and Control

Turkish CoHE Thesis Center URL

Citation

WoS Q

Scopus Q

Source

Volume

Issue

Start Page

0

End Page

75

URI

https://hdl.handle.net/20.500.14411/4300

Collections

Master Tezler / Master Thesis