Software engineering issues in big data application development
No Thumbnail Available
Date
2017
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Institute of Electrical and Electronics Engineers Inc.
Open Access Color
OpenAIRE Downloads
OpenAIRE Views
Abstract
Big Data has become one of the most important concepts that is being studied in Computer/Software Engineering. The data produced in recent years have increased rapidly and exponentially, necessitating the solution of major problems such as the collection, processing and storage of huge volume of data. Big Data Frameworks are developed specifically to solve these problems that facilitates application developers by providing opportunities to collect, process, manage, monitor and analyze these data. A few examples of these frameworks are Hadoop, Spark, Storm, and Flink, which are developed by Software Engineers as open source projects. Although the challenges raised from coordination of IT resources such as huge amounts of computation power, storage area, memory, and network bandwidth in a distributed manner solved by these frameworks, there still remains many Software Engineering problems in application development phase, even if they based on these frameworks. High scalability, fault tolerance, flexibility, reliability and testability can be listed as the main issues need to be carefully considered in terms of Software Engineering. In this paper, we first clarify the terms Framework-Application, and then the overview information about Big Data and related frameworks are given before emphasizing the problems arising in terms of Software Engineering. Nevertheless, we tried to provide guidance to the people who would develop software for Big Data and tried to give the further research guidance. © 2017 IEEE.
Description
Keywords
Big Data, Characteristics, Data engineering, Data science, Frameworks, Problems, Software Engineering
Turkish CoHE Thesis Center URL
Fields of Science
Citation
4
WoS Q
Scopus Q
Source
2nd International Conference on Computer Science and Engineering, UBMK 2017 -- 2nd International Conference on Computer Science and Engineering, UBMK 2017 -- 5 October 2017 through 8 October 2017 -- Antalya -- 132116
Volume
Issue
Start Page
851
End Page
855