Skip to main navigation Skip to search Skip to main content

A density-aware similarity join query processing algorithm on mapreduce

  • Miyoung Jang
  • , Youngho Song
  • , Jae Woo Chang*
  • *Corresponding author for this work
    • Jeonbuk National University

    Research output: Contribution to conferenceConference paperpeer-review

    Abstract

    Recently, the amount of data is rapidly increasing and thus MapReduce has attracted much interest as a new paradigm for such data-intensive applications. Similarity join is an essential operation for data analytics, including record linkage, near duplicate detection, document clustering. However, the performance of MapReduce is limited when applied on complex data analytical task involving joins of multiple datasets. Hence, workload-aware data partitioning techniques are required, which ensure the balance of computation of each machine. In this paper, we propose a similarity join algorithm using MapReduce that provides scalability and high performance by using grid-based data mapping technique for joining datasets. From the experiment analysis, we prove that our algorithm outperforms the existing algorithm under various data size and similarity thresholds.

    Original languageEnglish
    Title of host publicationAdvanced Multimedia and Ubiquitous Engineering - FutureTech and MUE
    EditorsHai Jin, Young-Sik Jeong, Muhammad Khurram Khan, James J. Park
    PublisherSpringer Verlag
    Pages469-475
    Number of pages7
    ISBN (Print)9789811015359
    DOIs
    StatePublished - 2016
    Event11th International Conference on Future Information Technology, FutureTech 2016 - Beijing, China
    Duration: 2016.04.202016.04.22

    Publication series

    NameLecture Notes in Electrical Engineering
    Volume393
    ISSN (Print)1876-1100
    ISSN (Electronic)1876-1119

    Conference

    Conference11th International Conference on Future Information Technology, FutureTech 2016
    Country/TerritoryChina
    CityBeijing
    Period16.04.2016.04.22

    Keywords

    • Bigdata analysis
    • Cloud computing
    • Grid-based partitioning
    • MapReduce
    • Similarity join

    Quacquarelli Symonds(QS) Subject Topics

    • Engineering - Mechanical

    Fingerprint

    Dive into the research topics of 'A density-aware similarity join query processing algorithm on mapreduce'. Together they form a unique fingerprint.

    Cite this