© Dong Ma and Mahbub Hasan 2018
Individual Project Asignment for COMP4336/9336 Mobile Data Networking
Semester 2, 2018
Due: 11:59pm Friday 12 October
Weighting: 20%
Version 280818 – Released on 28 August 2018
Title: Mobile Gesture Communications using Light Sensor
Background and Motivation
With advancements in Internet of Things (IoT), we need to interact with many objects around us.
Conventional methods for interacting with mobile devices, such as touch scren, verbal commands,
etc. wil not always be feasible, convenient or privacy-preserving. Instead, hand gesture is expected
to become the prefered mode for communicating with many IoT devices. Researchers are looking
to exploit various sensors such as camera, acelerometer, microphone, and light sensor to recognize
gestures. Tech giants such as Microsoft Kinect and Leap Motion have already implemented some
form. of gesture recognition in their products. In this asignment, you are required to design a
mobile gesture communication system for Android devices using light sensors.
Learning objectives
Upon completing this asignment, students wil:
1. Master the aces and manipulation of light sensor in mobile devices,
2. Design and implement a real-time gesture recognition system for mobile devices, and
3. Develop an application that takes input from hand gestures.
Asignment Tasks
Task 1 Graphical Display of Light Sensor Data [5 marks]
A light sensor measures the light intensity of the environment. The brighter the environment, the
higher the sensor readings, and vice versa. The key idea of light-based gesture recognition is that
specific hand movements (gestures) near the device block ambient light in specific ways, leaving
their unique signatures on the light sensor readings. Hence, the first step is to obtain the time series
of light sensor data from your Android device. In this task, you should develop an App that can
continuously read light sensor data and display a real-time graph of the sensor values (only display
the latest five seconds values).
Task 2 Gesture Counter [5 marks]
In this task, you should define a basic gesture, Down-Up, which represents moving a hand towards
the light sensor and then moving away from it (asuming the device is lying on a table). In the real-
time graph, you can expect a trough as the sensor value would decrease when the hand is
© Dong Ma and Mahbub Hasan 2018
approaching the device, and vice versa. Your aim in this task is to count the number of troughs
within the last five seconds and display the trough counter in a TextView. To achieve this, you wil
have to design an algorithm to detect troughs in light sensor time series data.
Task 3 Gesture Recognition [10 marks]
By exploiting the counts, as detected in Task 2, you can define a set of gestures with diferent
counts. For example, one count (only one trough detected) represents Gesture1, two counts (two
troughs detected) represents Gesture2, and so on. In this task, you are required to develop a
practical App that can recognize count-based hand gestures and react to them. Examples of
practical apps can be a gesture-controlled music player that alows the user to control the
functions of the player, such as play, pause, turn up the volume, turn down the volume, etc. using
hand gestures. Another example app can be a gesture-assisted photo sharing via Bluetooth. In
Lab7, you wil learn Bluetooth based device-to-device communication. However, al the actions in
that lab experiment are based on your typing inputs (e.g., click buttons). In this example app, you
are expected to share a photo with a nearby Bluetooth receiver (another smartphone or laptop) by
relying only on your gesture inputs (without typing). You are alowed to design any other similar
apps (should be a practical app) that exploit count-based gestures. Your apps should recognize at
least 3 gestures.
Some Useful Links
Gesture recognition
https:/ww.cs.dartmouth.edu/~trdata/reports/TR2016-797.pdf
Android music player (if you are developing a gesture-controlled music player)
https:/developer.android.com/guide/topics/media-apps/audio-app/building-an-audio-app
Bluetooth file transfer (if you are developing gesture-asisted device-to-device photo sharing)
https:/ww.intorobotics.com/how-to-develop-simple-bluetooth-android-application-to-control-a-robot-remote/
https:/developer.android.com/guide/topics/connectivity/bluetooth
https:/dzone.com/articles/bluetooth-data-transfer
Moodle Discusion Forums
You can discuss any isues with the asignment specs in Moodle Forum.
Asignment Submision and Marking
Asignment wil be marked based on a report as wel as a demo that you wil have to perform.
during Wek 13 (demo slots wil be advertised later). You are required to submit your code and a
maximum of 10-page (single column, 10 pt.) report explaining your algorithms and designs by
11:59pm 12 October. Late penalty at the rate of 10% per day late wil apply for the report
submision. No submisions wil be acepted after 19 October.
The End (We hope you enjoy doing this asignment)