Software engineering issues in big data application development

No Thumbnail Available

Date

2017

Authors

Karakaya, Ziya

Journal Title

Journal ISSN

Volume Title

Publisher

Institute of Electrical and Electronics Engineers Inc.

Open Access Color

OpenAIRE Downloads

OpenAIRE Views

Research Projects

Organizational Units

Organizational Unit
Computer Engineering
(1998)
The Atılım University Department of Computer Engineering was founded in 1998. The department curriculum is prepared in a way that meets the demands for knowledge and skills after graduation, and is subject to periodical reviews and updates in line with international standards. Our Department offers education in many fields of expertise, such as software development, hardware systems, data structures, computer networks, artificial intelligence, machine learning, image processing, natural language processing, object based design, information security, and cloud computing. The education offered by our department is based on practical approaches, with modern laboratories, projects and internship programs. The undergraduate program at our department was accredited in 2014 by the Association of Evaluation and Accreditation of Engineering Programs (MÜDEK) and was granted the label EUR-ACE, valid through Europe. In addition to the undergraduate program, our department offers thesis or non-thesis graduate degree programs (MS).

Journal Issue

Abstract

Big Data has become one of the most important concepts that is being studied in Computer/Software Engineering. The data produced in recent years have increased rapidly and exponentially, necessitating the solution of major problems such as the collection, processing and storage of huge volume of data. Big Data Frameworks are developed specifically to solve these problems that facilitates application developers by providing opportunities to collect, process, manage, monitor and analyze these data. A few examples of these frameworks are Hadoop, Spark, Storm, and Flink, which are developed by Software Engineers as open source projects. Although the challenges raised from coordination of IT resources such as huge amounts of computation power, storage area, memory, and network bandwidth in a distributed manner solved by these frameworks, there still remains many Software Engineering problems in application development phase, even if they based on these frameworks. High scalability, fault tolerance, flexibility, reliability and testability can be listed as the main issues need to be carefully considered in terms of Software Engineering. In this paper, we first clarify the terms Framework-Application, and then the overview information about Big Data and related frameworks are given before emphasizing the problems arising in terms of Software Engineering. Nevertheless, we tried to provide guidance to the people who would develop software for Big Data and tried to give the further research guidance. © 2017 IEEE.

Description

Keywords

Big Data, Characteristics, Data engineering, Data science, Frameworks, Problems, Software Engineering

Turkish CoHE Thesis Center URL

Fields of Science

Citation

4

WoS Q

Scopus Q

Source

2nd International Conference on Computer Science and Engineering, UBMK 2017 -- 2nd International Conference on Computer Science and Engineering, UBMK 2017 -- 5 October 2017 through 8 October 2017 -- Antalya -- 132116

Volume

Issue

Start Page

851

End Page

855

Collections