Template-driven Real-time Data Acquisition Technique with Big Data

Abstract

The development of Big Data (BD), which is used to obtain numerous data from various domains, is brought about by technological advancement. However, managing the information and extracting knowledge from it is the most challenging and problematic. Thus, this paper proposed a template-centric new Data Acquisition (DAQ) methodology. The stock market data is gathered from several structured or unstructured data sources. After the DAQ criterion, templates are created for the gathered data. The stock market data is collected grounded on its Application Programming Interface (API) and transmitted via the transmission protocols during the DAQ process. To effectively remove redundant data, the transmitted data is pre-processed and stored efficiently in the network for further real-time analysis. Finally, the proposed technique’s performance is evaluated. As per the experimental and empirical evaluation, the proposed system surpasses the other methods.