SEMINAR REPORT

On

Sixth Sense Technology

Wear your world

Devsthali, Ambala

(Affiliated to Kurukshtera University, Kurukshetra)

(2008-2010)

Submitted to: Submitted by:

Mr. Arvind Selwal
Prarthana Wahi
(HOD)
2308015
CSE Dept. CSE
3rd yr.

1 |Page
 ACKNOWLEDGEMENT
I would like to take this opportunity to show gratitude towards Mr.
Arvind Selwal (HOD CSE Department) who helped in bringing a seminar
to a success. He has been a motivator & a source of inspiration to carry
out the necessary proceedings for the seminar to be completed
successfully.

Finally I would like to take the opportunity

to thank the organization ACE & AR which helped me to acquire proper
knowledge and a success in the B.Tech (CSE)

Prarthana Wahi
20308015

INDEX
2 |Page
 1. ABSTRACT
2. INTRODUCTION
3. THE MASTER MIND
4. INITIAL EXPERIMENTS
5. SIXTH SENSE:THE DEVICE
6. WORKING

7. APPLICATIONS

8. CONCLUSION
9. REFRENCES

3 |Page
ABSTRACT
'SixthSense' is a wearable gestural interface that augments the physical world around us with digital information
and lets us use natural hand gestures to interact with that information.
All of us are aware of the five basic senses – seeing, feeling, smelling, tasting and hearing. But there is also
another sense called the sixth sense. It is basically a connection to something greater than what their physical
senses are able to perceive. To a layman, it would be something supernatural. Some might just consider it to be
a superstition or something psychological. But the invention of sixth sense technology has completely shocked
the world. Although it is not widely known as of now but the time is not far when this technology will change
our perception of the world.
Pranav Mistry, 28 year old, of Indian origin is the mastermind behind the sixth sense technology. The device
sees what we see but it lets out information that we want to know while viewing the object. It can project
information on any surface, be it a wall, table or any other object and uses hand / arm movements to help us
interact with the projected information. The device brings us closer to reality and assists us in making right
decisions by providing the relevant information, thereby, making the entire world a computer.
The SixthSense prototype implements several applications that demonstrate the usefulness, viability and
flexibility of the system. The map application lets the user navigate a map displayed on a nearby surface using
hand gestures, similar to gestures supported by Multi-Touch based systems, letting the user zoom in, zoom out
or pan using intuitive hand movements.The user can stop by any surface or wall and flick through the photos
he/she has taken. SixthSense also lets the user draw icons or symbols in the air using the movement of the
index finger and recognizes those symbols as interaction instructions. For example, drawing a magnifying glass
symbol takes the user to the map application or drawing an ‘@’ symbol lets the user check his mail. The
SixthSense system also augments physical objects the user is interacting with by projecting more information
about these objects projected on them.
The current prototype system costs approximate $350 to build

4 |Page
INTRODUCTION
We've evolved over millions of years to sense the world around us. When we encounter something, someone
or some place, we use our five natural senses to perceive information about it; that information helps us
make decisions and chose the right actions to take. But arguably the most useful information that can help us
make the right decision is not naturally perceivable with our five senses, namely the data, information and
knowledge that mankind has accumulated about everything and which is increasingly all available online.
Although the miniaturization of computing devices allows us to carry computers in our pockets, keeping us
continually connected to the digital world, there is no link between our digital devices and our interactions
with the physical world. Information is confined traditionally on paper or digitally on a screen. SixthSense
bridges this gap, bringing intangible, digital information out into the tangible world, and allowing us to
interact with this information via natural hand gestures. ‘SixthSense’ frees information from its confines by
seamlessly integrating it with reality, and thus making the entire world your computer.

Sixth Sense technology is the science of tomorrow with the aim of connecting the digital world with the
physical world seamlessly, eliminating hardware devices. Mistry’s flirtations with the digital world began in
the early 2000s when he pieced together four mouse rollers with pulleys and springs to give shape to a
motion sensing device.

It is similar to Telepointer, a neck worn projector/camera system developed by Media Lab student Steve Mann
(which Mann originally referred to as "Synthetic Synesthesia of the Sixth Sense"). What the researchers have
done is combine a number of standard gadgets including a webcam, projector, and mobile phone, to form a
brand new interaction experience. In its current form the battery-powered projector is attached to a hat, the
webcam is hung around the neck (or also positioned on a hat), and the mobile phone provides the connection to
the Internet.

The wearer uses hand gestures combined with the gadgets to perform actions. So, for example, they could make
a circle over their wrist with the fingers of one hand. WUW would recognize this action and project a clock face
on to their wrist. Make a picture frame with both your hands and WUW will take a picture as if it was a camera.
Stand near a wall and you can get a projected desktop allowing you to open applications, view the pictures
you’ve taken, or surf the ‘Net.

With the invention of intuitive computer interfaces, the digital and physical worlds came together closer than
ever. One just has to place an object (anything from a flight boarding pass to a key) on a Tangible Public Map
(TaPuMa) on its horizontal smart screen to get inside-out information about it. Keep a coffee cup on a particular
spot on the map and a whole range of cafés in the area is brought to you on a platter. All you have to do to get
seamlessly connected with the digital world is wear a simple pendant-like equipment consisting of a camera and
a portable battery-powered projection system with a mirror (a more modish version is just round the corner).
The device (now known as the Sixth Sense Device) when connected to a cell phone acts as a computation and
communication tool. The camera tracks hand gestures and helps gather “meta information” (information from
the surroundings) and articulates it with the digital domain. Wearing marker caps on ones fingers (a more
stylish option is to paint the said fingernails in a different colour each) and making gestures with them, one can
use any interface (Yes! Any interface! No longer is the human race tied to the bulky world of computer screens)
to access and modify data. Clicking a picture is as easy as conjuring up a rectangle in the air aimed at the object
of visual desire with the thumb and index fingers. A few finger motions help edit and resize pictures and
another set of gestures later, the pictures find themselves E-mailed to recipients.

5 |Page
THE MASTER MIND
Pranav Mistry, 28 year old, of Indian origin is the mastermind behind the sixth sense technology. Born in
Palanpur, Gujarat he received his Bachelors of technology in computer science and engineering from India
and later received masters in design from MIT Boston. He invented ‘ Sixth Sense / WUW ( Wear UR World)
‘ which is a wearable gestural , user friendly interface which links the physical world around us with digital
information and uses hand gestures to interact with them. Pranav’s eagerness to pop pixels out of the digital
world into the real world led to the birth of Sixth Sense technology.
Currently he is a PhD student at MIT and he won the ‘Invention of the Year 2009 ‘- by Popular Science. The
device sees what we see but it lets out information that we want to know while viewing the object. It can
project information on any surface, be it a wall, table or any other object and uses hand / arm movements to
help us interact with the projected information. The device brings us closer to reality and assists us in making
right decisions by providing the relevant information, thereby, making the entire world a computer.

6 |Page
SIXTH SENSE:THE DEVICE

Sixth Sense is a mini-projector coupled with a camera

and a cellphone—which acts as the computer and your
connection to the Cloud, all the information stored on
the web.

The hardware components are coupled in a pendant

like mobile wearable device.

Both the projector and the camera are connected to the

mobile computing device in the user’s pocket.

Software:

Software program processes the video stream data captured by the camera

Pranav Mistry says the SixthSense software will be open source. As far as this seems to be a little set of items,
there will not be user interfaces or much advanced programs for the users. There will be much harder and
secured coding inside the device to make sure the security of the software. It will be interesting to know the new
language for coding for a SixthSense device.

Hardware:
In order to control the SixthSense, it requires some advance hardware as it appears to be. In some of the
public presentations, the presenter was wearing some controlling devices including color markers, Camera and
projector. They has to be compact and easily controllable. However the hardware integration of SixthSense
technology is quite innovative since they have manage to develop camera and pen like day to day objects.

Projector projects visual information enabling surfaces, walls and physical objects around us to be used as
interfaces

Camera recognizes and tracks user's hand gestures and physical objects using computer-vision based
techniques.

Colored markers (visual tracking fiducials) placed at the tip of the user’s fingers using simple computer-vision
techniques helps the webcam to track the movement of fingers. The movements and arrangements of these
fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces

Mobile device may be a laptop, PDA, smart phones etc. These are connected to other hardware devices and
sends information to projector for projection. The important thing is that the device is a mobile device. Means,
it is so light that we can take it with us where ever we want to. It is as small as a cell phone and is so simple to
use.
7 |Page
WORKING
Sixth sense device analyses what user sees and visually augments the surfaces and physical objects user is
interacting with. What the researchers have done is combine a number of standard gadgets including a webcam,
projector, and mobile phone, to form a brand new interaction experience. The key here is that Sixth Sense
recognizes the objects around you, displaying information automatically and letting you access it in any way
you want, in the simplest way possible

The technology in itself is nothing more than the combination of some stunning technologies, but the idea of
combining those technologies is really great. The technology is mainly based on hand gesture recognition,
image capturing, processing, and manipulation, etc. The camera is used to recognize and track user’s hand
gestures and physical objects using computer-vision based techniques, while the projector is used to project
visual information on walls or on any physical thing around us. Other hardware includes mirror and colored
caps to be used for fingers. The software of the technology uses the video stream, which is captured by the
camera, and also tracks the location of the tips of the fingers to recognize the gestures. This process is done
using some techniques of computer vision.
Basically it is a device which is a mini projector and which can be projected on any surface, it carries the
information stored in it and also collects information from the web. It is the one which obey hand gestures of
yours and gives you what you want to see and know.

It is the combined technology of computer along with cell phone. It works when a person hang it on his neck
and start projecting through the micro-projector attached to it. Your fingers works like the keyboard as well
as the mouse.

8 |Page
POTENTIAL APPLICATIONS
In Pranav Mistry’s demonstration, he showed several applications that can be made using the SixthSense
technology. It would be a great achievement for the Information Technology as well. Since it is the
beginning of the SixthSense applications, there will be lot of tough problems ahead the SixthSense
application developers.

Motion Capture

The innovative camera is something that got applauds of the crowd in the demonstration. Four rubber rings in
the four fingers two in each can be used do draw a rectangular. It will be the area covered by the photograph.
Then the photos can be taken to any interface for editing, managing the galleries and sharing. This is much
controversial. There is an inbuilt storage device where the data can be stored. When using a surface, the whole
data is taken into that surface and stored in the device after the editing. And also the capturing device being so
small like 4 rubber rings can be used to capture the photos of any location. Example using 2 fingers to send 5
seconds, 3 fingers to send 10 seconds likewise it can be done.

There are so many new applications to be developed in the near future. A remote controller for media devices
can be implemented to be worked with fingers using this technology. Except having the remote controller to
send fast forward, rewind or next or previous options, a SixthSense device can be used. The current process of
sending fast-forward or rewind with advanced options.

Many games can be made using this technology. A car race using an imaginary steering wheel will be an
attractive application. The player will have to drive the vehicle as a real one using his arms. The SixthSense tool
kit will be a good replacement for the XBOX game kit and it will not be much expensive as it is said.Other
potential applications include:

Capture photos with fingers: Why to take camera on your holiday and no tension for the photo space as
this Sixth Sense computer will work like your camera. It captures the photo, when you make a square with
your fingers, highlighting which one you want to frame.

Phone Call: You can call to your friend by typing the numbers on your hand. It displays the keypad of the
phone over your palm and the keys appear on the four finger. Use your other hand's finger to press the keys.

Check the brand of the Product: It helps you to choose the best brand product from the super market.

Read Books easily: Check out the ratings of the Book you are going to buy, it checks the ratings from the
internet. And another amazing thing is that it reads the book for you.. :)

Newspapers: Did you saw the moving pictures of the Newspaper in the movie Harry Patter, it is quite
similar to it. It Searches the most appropriate video from the web by seeing the headlines or the caption of
the News report.

Check your Flight Status: You can check the status of the flight while you are on Taxi. Just place the ticket
in front of the projector and it checks its status from the internet.

SEE time without wearing wrist watch: Watches are ancient history now with the prevalence of cell
phones. With sixth sense technology, it is just “drawing a circle on the wrist” away from catching the current
time.
9 |Page
CONCLUSION
Using this technology

“Finally one gets a taste of the world of Harry Potter.”

Sixth Sense technology is the science of tomorrow with the aim of connecting the digital world with the
physical world seamlessly, eliminating hardware devices. Mistry’s flirtations with the digital world began in the
early 2000s when he pieced together four mouse rollers with pulleys and springs to give shape to a motion
sensing device. This device interprets gestures made in the physical world and replicates them in the digital
world. Next he gave the omni-present sticky note a digital makeover. Digitized sticky notes (or Quickies) set as
reminders can jog your memory regarding tasks through an SMS or E-mail. Jotting down a question on a
quickie prints you the answer. All it takes to message someone on their cell phone is write a note on a quickie.
Next in tow for Mistry was a pen that can draw in 3D (a technology popular by the name “Inktuitive”).

With the invention of intuitive computer interfaces, the digital and physical worlds came together closer than
ever. One just has to place an object (anything from a flight boarding pass to a key) on a Tangible Public Map
(TaPuMa) on its horizontal smart screen to get inside-out information about it. Keep a coffee cup on a particular
spot on the map and a whole range of cafés in the area is brought to you on a platter.

Pranav’s eagerness to pop pixels out of the digital world into the real world led to the birth of Sixth Sense
technology. All you have to do to get seamlessly connected with the digital world is wear simple pendant-like
equipment consisting of a camera and a portable battery-powered projection system with a mirror (a more
modish version is just round the corner). The device (now known as the Sixth Sense Device) when connected to
a cell phone acts as a computation and communication tool. The camera tracks hand gestures and helps gather
“meta information” (information from the surroundings) and articulates it with the digital domain. Wearing
marker caps on ones fingers (a more stylish option is to paint the said fingernails in a different colour each) and
making gestures with them, one can use any interface (Yes! Any interface! No longer is the human race tied to
the bulky world of computer screens) to access and modify data. Clicking a picture is as easy as conjuring up a
rectangle in the air aimed at the object of visual desire with the thumb and index fingers. A few finger motions
help edit and resize pictures and another set of gestures later, the pictures find themselves E-mailed to recipients

10 | P a g e
REFRENCES
http://www.pranavmistry.com/projects/sixth-sense-technology/

http://gizmodo.com/5167790/sixth-sense-technology-may-change-how-we-look-at-the-world-forever

http://www.ixibo.com/2009/03/sixth-sense-technology/

http://www.youthkiawaaz.com/2010/06/pranav-mistry-%E2%80%93-innovator-of-sixth-sense-technology-
%E2%80%93-technology-that-makes-sense/

http://www.geek.com/articles/gadgets/mit-develop-wear-ur-world-6th-sense-hand-gesture-gadget-2009026/

http://notesofgenius.com/multimedia-applications-sixthsense-technology/

11 | P a g e