Programming Assignment 8
Using Map-Reduce

Due on Thursday May 5 before midnight


Description

The purpose of this project is to develop a simple Map-Reduce program on Hadoop to analyze data.

This project must be done individually. No copying is permitted. Note: We will use a system for detecting software plagiarism, called Moss, which is an automatic system for determining the similarity of programs. That is, your program will be compared with the programs of the other students in class as well as with the programs submitted in previous years. This program will find similarities even if you rename variables, move code, change code structure, etc.

Note that, if you use a Search Engine to find similar programs on the web, we will find these programs too. So don't do it because you will get caught and you will get an F in the course (this is cheating). Don't look for code to use for your project on the web or from other students (current or past). Just do your project alone using the help given in this project description and from your instructor and GTA only.

Platform

You will develop this project on Omega. After you login on Omega, do the following:

export HADOOP=/home/u/ux/uxg4406/public_html/project9/hadoop-1.2.1
export CLASSPATH=.:$HADOOP/hadoop-core-1.2.1.jar
export JAVA_HOME=/opt/jdk1.6.0_20
Hadoop can be run from your Hadoop directory with the following command in STANDALONE MODE.
$HADOOP/bin/hadoop jar YourJarFile.jar YourMainClass YourProgramArguments1 YourProgramArguments2

Documentation

Project Requirements

You will use the orders.tbl with the following schema:

Orders ( OrderKey, CustKey, OrderStatus, TotalPrice, OrderDate, OrderPriority, Clerk, ShipPriority, Comment )

You need to implement the following SQL query in the Map-Reduce Framework:

SELECT CustKey, SUM(TotalPrice) FROM orders GROUP BY CustKey

What to Submit

Use the form below to submit your java file:

Submit Project #8:

Last modified: 04/25/2016 by Leonidas Fegaras