SIXTH SENSE TECHNOLOGY

-LIFE BEYOND THE FIVE SENSES

 1.INTRODUCTION:

1.1 What Is Sixth Sense Technology:

Everyday there is a new technology in the world. And how much of you remember the
last one. But there are some technology which changes the thought of each and every
person who are fond of new technology and accept it very well with its uses in their life.
In the same way, recently, I got to know about the new technology and want to share with
you all. It is not only unique but amazing technology which will change the entire world.
It is none other than 'Sixth Sense'. This technology is very unique and who ever will
know about it, will proud to be an Indian and the person who had developed it. The Sixth
Sense technology is a wearable gestural interface that augments the physical world with
digital information around us and we can use natural hand gestures to interact with that
information. Doesn't it sound so interesting!!

1.2 Need For Sixth Sense Technology:

We've evolved over millions of years to sense the world around us. When we encounter
something, someone or some place, we use our five natural senses to perceive
information about it; that information helps us make decisions and chose the right actions
to take. But arguably the most useful information that can help us make the right decision
is not naturally perceivable with our five senses, namely the data, information and
knowledge that mankind has accumulated about everything and which is increasingly all
available online. Although the miniaturization of computing devices allows us to carry
computers in our pockets, keeping us continually connected to the digital world, there is
no link between our digital devices and our interactions with the physical world.
Information is confined traditionally on paper or digitally on a screen. Sixth Sense
bridges this gap, bringing intangible, digital information out into the tangible world, and
allowing us to interact with this information via natural hand gestures. ‘Sixth Sense’ frees
information from its confines by seamlessly integrating it with reality, and thus making
the entire world your computer.
 1.3 Invention of Sixth Sense Technology:

This Sixth Sense technology is developed by Pranav Mistry who is 28 year old and the
most interesting part is that he is of Indian origin. He invented ‘ Sixth Sense' which one
can also say WUW ( Wear UR World) in short. Actually, it is named so because you can
operate this with a wearable gestural , it is user friendly interface which links the physical
world around us and uses the hand gestures the same way you use other things like
reading newspaper, turning pages, etc and in the same way you can interact with them.
Well, this device will bring us closer to reality and assists us in making right decisions by
providing the relevant information, thereby, making the entire world a computer. Isn't so?
Augmented Reality enhances user perception by overlaying real world information with
virtual computer-generated information. The aims of the 6 th Sense project are to improve
real-time interaction between the real environment and the virtual world and to maximize
the user experience in mobile Augmented Reality. To achieve these objectives a generic
framework constituted of two main layers is proposed. The End-to-End Adaptation Layer
adapts in real-time the parameters of the Augmented Reality system to provide the user
with the best possible experience despite the varying operating conditions such as the
transmission link and user head motion. The Generic Augmented Reality Layer
encompasses solutions to the problem of overlaying adequate information in the real
scene and manages multimodal interaction with the virtual environment.

Basically, Sixth Sense is a mini-projector coupled with a camera and a cellphone—

which acts as the computer and your connection to the Cloud, all the information stored
on the web. Sixth Sense can also obey hand gestures, like in the infamous Minority
Report.

However, instead of requiring you to be in front of a big screen like Tom Cruise, Sixth
Sense can do its magic—and a lot more—everywhere, even while you are jumping
hysteric over Oprah's sofa.
 The camera recognizes objects around you instantly, with the micro-projector
overlaying the information on any surface, including the object itself or your hand.
Then, you can access or manipulate the information using your fingers. Need to make
a call? Extend your hand on front of the projector and numbers will appear for you to
click. Need to know the time? Draw a circle on your wrist and a watch will appear.
Want to take a photo? Just make a square with your fingers, highlighting what you
want to frame, and the system will make the photo—which you can later organize
with the others using your own hands over the air.

But those are just novelty applications. The true power of Sixth Sense lies on its
potential to connect the real world with the Internet, and overlaying the information on
the world itself. Imagine you are at the supermarket, thinking about what brand of soap is
better. Or maybe what wine you should get for tonight's dinner. Just look at objects, hold
them on your hands, and Sixth Sense will show you if it's good or bad, or if it fits your
preferences or not.

Now take this to every aspect of your everyday life. You can be in a taxi going to the
airport, and just by taking out your boarding pass, Sixth Sense will grab real time
information about your flight and display it over the ticket. You won't need to do any
action. Just hold it in front of your and it will work.

The key here is that Sixth Sense recognizes the objects around you, displaying
information automatically and letting you access it in any way you want, in the simplest
way possible. Clearly, this has the potential of becoming the ultimate "transparent" user
interface for accessing information about everything around us. If they can get rid of the
colored finger caps and it ever goes beyond the initial development phase, that is. But as
it is now, it may change the way we interact with the real world and truly give everyone
complete awareness of the environment around us.
2.System over view:
2.1 TED(Technology,Entertainment,Design):

TED (Technology, Entertainment, Design) is one of the most interesting conferences that
happen every year in the US where extraordinary people from all over the world come
and share their insights and knowledge about topics which are as varied as they are
interesting.
For a long time these talks were for the privileged few who attended, however quite
recently, these phenomenal talks were made freely downloadable from their website
www.ted.com and made available to the public at large. This article is based on a talk that
I downloaded from the TED website featuring Pattie Maes and a mesmerizing gadget that
she, her student Pranav Mistry and her team are developing. They call it the Sixth Sense.
And no, it’s got nothing to do with some deep spiritual stuff or Bruce Willis, but the end
product is such that I wouldn’t blame you if you said it was akin to magic.
SixthSense Technology: Life Beyond the Five Senses

The world has shrunk. Distances have dissolved. Communication lines and interaction
with countless systems have been rendered feasible. However this technological overhaul
has been peripheral and not so much related to the human body; researchers and
innovators have constantly grappled with the issue of bridging the gaps which limit the
human-environment contact. Well, looks like we finally may have stumbled upon an
answer to that quagmire.
 Pranav Mistry, a student at the Media Lab of Massachusetts Institute
of Technology (MIT), has developed a gestural interface device which enables
enrichment of the physical world with knowledge that is digital and allows a person to
use natural motions to act together with this information so received. This device,
tentatively name as the Sixth Sense, is a wearable machine that assists unexplored
interactions between the real and the virtual sphere of data. It consists of certain
commonly available components, which are intrinsic to its functioning. These include a
camera, a portable battery-powered projection system coupled with a mirror and a cell
phone. All these components communicate to the cell phone, which acts as the
communication and computation device. The entire hardware apparatus is encompassed
in a pendant-shaped mobile wearable device. Basically the camera recognises individuals,
images, pictures, gestures one makes with their hands and the projector assists in
projecting any information on whatever type of surface is present in front of the person.
The usage of the mirror is significant as the projector dangles pointing downwards from
the neck. To bring out variations on a much higher plane, in the demo video which was
broadcasted to showcase the prototype to the world, Mistry uses coloured caps on his
fingers so that it becomes simpler for the software to differentiate between the fingers,
demanding various applications. The software program analyses the video data caught by
the camera and also tracks down the locations of the coloured markers by utilising single
computer vision techniques. One can have any number of hand gestures and movements
as long as they are all reasonably identified and differentiated for the system to interpret
it, preferably through unique and varied fiducials. This is possible only because the ‘Sixth
Sense’ device supports multi-touch and multi-user interaction.

Now you might be thinking how does it work . Well, this Sixth Sense prototype is
comprised of a pocket projector, a mirror and a camera and the hardware components are
coupled in a pendant like mobile wearable device. To use this both the projector and the
camera are connected to the mobile computing device in the user’s pocket. On the base of
an information, I would like to share that projector projects visual information enabling
surfaces, walls and physical objects around us to be used as interfaces while the camera
recognizes and tracks user's hand gestures and physical objects using computer-vision
based techniques. The video stream data captured by the camera and processed by the
video strea. After that they tracks the locations of the colored markers at the tip of the
user’s fingers using simple computer-vision techniques, thus Sixth Sense also supports
multi-touch and multi-user interaction.
You can also zoom in and zoom out using intuitive hand movements. This technology
also recognizes user’s freehand gestures or postures you can say.

2.2 Sixth Sense Commands Computer with Gestures:

Pranav’s Website defines this gadget/experience as “Sixth Sense is a wearable gestural

interface that augments the physical world around us with digital information and lets us
use natural hand gestures to interact with that information. By using a camera and a tiny
projector mounted in a pendant like wearable device, ‘Sixth Sense’ sees what you see and
visually augments any surfaces or objects we are interacting with.”
Sixth Sense is the opposite of virtual reality. That is, virtual reality submerses a person
in the digital world. Sixth Sense turns the tables and enables the digital world to shine a
light on the real world. Communication in this computing environment happens with
everyday human gestures.
It projects information onto surfaces, walls, and physical objects around us, and let’s
interact with the projected information through natural hand gestures, arm movements, or
our interaction with the object itself. ‘Sixth Sense’ attempts to free information from its
confines by seamlessly integrating it with reality, and thus making the entire world
your computer.”

Wearing sensors on your fingertips that understand gestures and a camera equipped,
image projector system hanging around your neck. With Sixth Sense, you can draw on
any surface, including your hand, check email by making an "@" symbol with your
finger in the air, take pictures simply by framing an image with your hands, watch a
video or read news displayed on a regular piece of paper, among other amazing feats.
 'Sixth Sense' is a wearable gestural interface that augments the physical world around
us with digital information and lets us use natural hand gestures to interact with that
information.

2.3 Sixth Sense Technology Using RFID:

RFID is widely used to track the movement of goods through a supply chain. RFID
platform is extended by presenting Sixth Sense here. Sixth Sense combines mobility
information obtained from RFID-based sensing with information from enterprise systems
such as calendar and presence, to automatically draw inferences about the association
and interaction amongst people, objects, and workspaces.
Radio Frequency Identification (RFID) is an electronic tagging technology that allows
the detection and tracking of tags, and consequently the objects they are affixed to. Sixth
Sense focuses on applying RFID to an enterprise setting, such as a corporate office or
university department. In short Sixth Sense represents a form of mobile computing
applied to non-computing entities.
Sixth Sense: RFID-based Enterprise Intelligence

The vision of the Sixth Sense project in the MNS group at Microsoft Research India is
the workplace or home of the future where computing is extended to encompass non-
computing entities such as people, objects, and spaces to enable rich user experiences.
For instance, we would like users to be able to search the physical world for objects they
may have misplaced or use physical events to index their experiences (e.g., "was my
laptop with me when I received a phone call from person X?").
The key technology underlying Sixth Sense is Radio Frequency Identification, or RFID.
The technology comprises inexpensive tags that are attached to objects and readers that
are able to read these tags from some distance. RFID is widely used to track the
movement of goods through a supply chain. In a typical setting, a reader installed at the
entrance of a warehouse can track pallets as they are moved in or out of the warehouse.

In Sixth Sense, we extend the domain of RFID to settings where there is a rich interaction
amongst people and between people and objects. To this end, we consider an enterprise
setting of the future where people (or rather their employee badges) and their personal
objects such as books, laptops, and mobile phones are tagged with cheap, passive RFID
tags, and there is good coverage of RFID readers in the workplace. See the figure below
for an illustration of such a setting.

Sixth Sense provides a platform for RFID-based enterprise intelligence applications.

Sixth Sense combines mobility information obtained from RFID-based sensing with
information from enterprise systems such as calendar and presence, to automatically draw
inferences about the association and interaction amongst people, objects, and workspaces.
For instance, Sixth Sense is able to automatically distinguish between people and objects,
learn the identities of people, and infer the ownership of objects by people. Sixth Sense
also infers when an object has been interacted with by a user, for example, when a user
picks up their mobile phone.
Sixth Sense is a platform in that its programming model makes the inferences made
automatically available to applications via a rich set of APIs. To demonstrate the
capabilities of the Sixth Sense platform, we have prototyped a few applications using
these APIs, including a misplaced object alert service, an enhanced calendar service, and
rich annotation of video with physical events. We also discuss the issue of safeguarding
user privacy in the context of Sixth Sense.
3 Architecture:

The figure below shows the Sixth Sense architecture. The key components of the system,
including the databases, inference engine, and applications, are run centrally by the
enterprise. This provides the (trusted) inference engine access to the complete set of
sensed data across all users, objects, and zones, allowing it to make effective inferences.
Likewise, the (trusted) application is allowed the flexibility of working with a complete
set of inferences (i.e., inferences pertaining to all users and their objects), yet control
what processed information is presented to the users to ensure privacy. In contrast, if the
inference engine or the application were run by individual users on their own desktop
machines, privacy consideration would restrict the set of information made available to
these, and hence limit their functionality.

The main components of Sixth Sense are:

RFID Monitor: The RFID Monitor issues a read command every 500 ms to the RFID
reader. The reader reports the EPC and the signal strength (RSSI) of the tags read via
each of its antennas. This data then gets pushed into the raw database.
Other Enterprise Monitors: These monitor the information listed below and push their
updates into the raw database:

• Calendar Monitor: This resides on each users desktop machine, and monitors the
time and location of the user's appointments.
• Presence Monitor: This monitors the status of each user's interaction with their
desktop. A machine is said to be idle when it receives no user input for 2 minutes.
Transitions from idle to active state are detected and reported.
• Login Monitor: This is similar to the presence monitor except that in general login
is a much stronger indication of a user being present than simply a change in their
machines idle.
• Cameras: Office buildings are often equipped with cameras for security reasons.
The camera feed is stored in a video database for future analysis, if the need
arises. We show in applications how we combine the camera system in an
enterprise with other sensors to build useful applications.

Inference engine:
The inference engine operates on the raw database to draw inferences about people,
objects, and workspaces. It makes the following inferences:

• Person-Object Differentiation: The goal of person-object differentiation is to take

an undifferentiated mass of tags and classify each tag as either belonging to a
person or to an object. A “person tag” refers to a tag affixed, say, to a person's
employee badge, which is (almost) always carried by him/her.
• Object Ownership: Having classified tags as people or objects, the goal of this
module is to infer the owner (obviously a person) of each object.
• Zone Identification: We automatically classify RFID zones into individual
workspace, shared workspace, reserved shared workspace and common areas.
• Person Identification: After the nature of tags (person vs. object) has been
identified, we then infer the identity of the person corresponding to a person tag.
 • Person-Object Interaction: Finally, we examine events within an RFID zone.
Specifically, we infer that an object has been interacted with, e.g., picked up by a
person.

Processed database:
The processed database is populated by the inference engine with its inferences, making
these available to applications built on top of the Sixth Sense platform.

Raw database: The RFID monitor and the other enterprise monitors push data into the
raw database.

API:
Sixth Sense provides a set of APIs for applications to lookup the inferences stored in the
processed database or to receive callbacks when new inferences are made.

3.1 Applications using RFID:

We have prototyped a few applications using the APIs exposed by the Sixth Sense
platform.

Misplaced Object Alert:

The misplaced object alert service tracks the movement of users and objects, and notifies
a user when it thinks that he/she may have misplaced an object. An object is said to be
misplaced when it is in a shared workspace or a public area, and the object’s owner, who
was also in the same zone, moves away to a different zone. When such a situation is
detected, the system suspects that the object may have been misplaced and alerts the user
via email or phone.

Annotating Video with Physical Events:

We have built a simple application that the annotates the video feed from a camera with
RFID events corresponding to inter-zone movement and object interaction. The video
could, for instance, be from the security cameras in a building. With the video thus
annotated with physical events, a user who is searching for a misplaced objected but
suspects that it went missing in the lobby could issue a query like: "find me all segments
of the security video recording corresponding to times when I passed through the lobby".
This would then return the desired segments of video, relieving the user of the burden of
sifting through hours of irrelevant video recording. A screenshot of the application we
have built is shown below.

Automatic Conference Room Booking:

Sixth Sense automatically identifies a shared space as a reserved space (e.g., a conference
room) using the calendar information of users. While reserving such a room may be the
norm, people may sometimes occupy it without reservation, if it is not already occupied
and they are looking for space to hold an unplanned, last-minute meeting. Despite the
barging in, if the room has in fact not been reserved for the time in question, it would be
desirable to reserve it for the group that has barged in, to avoid the possibility of someone
else trying to reserve the apparently free but in fact occupied room. We have built an
automatic conference room booking application to accomplish precisely this.

3.2 Devices or Components of Sixth Sense:

The hardware components are coupled in a pendant like mobile wearable device.
1. Camera
2. Projector
3. Mirror
4. Mobile Component
5. Color Markers

Camera
Captures an object in view and tracks the users hand gestures. It sends the data to
smart phone. It acts as a digital eye, connecting you to the world of digital
information.
Projector
The projector projects visual information enabling surfaces and physical objects to
be used as interfaces. The project itself contains a battery inside, with 3 hours of
battery life. A tiny LED projector displays data sent from the smart phone on any
surface in view object, wall, or person.
Mirror
The usage of the mirror is significant as the projector dangles pointing downwards
from the neck.

Smart Phone
A Web-enabled smart phone in the users pocket processes the video data. Other
software searches the Web and interprets the hand gestures .
Color Markers
It is at the tip of the users fingers. Marking the user fingers with red, yellow, green,
and blue tape helps the webcam recognize gestures. The movements and
arrangements of these makers are interpreted into gestures that act as interaction
instructions for the projected application interfaces.

The Sixth Sense prototype is comprised of a pocket projector, a mirror and a

camera. The hardware components are coupled in a pendant like mobile wearable
device. Both the projector and the camera are connected to the mobile computing
device in the user’s pocket. The projector projects visual information enabling
surfaces, walls and physical objects around us to be used as interfaces; while the
camera recognizes and tracks user's hand gestures and physical objects using
computer-vision based techniques. The software program processes the video
stream data captured by the camera and tracks the locations of the colored markers
(visual tracking fiducially) at the tip of the user’s fingers using simple computer-
vision techniques. The movements and arrangements of these fiducials are
interpreted into gestures that act as interaction instructions for the projected
application interfaces. The maximum number of tracked fingers is only constrained
 by the number of unique fiducials, thus Sixth-Sense also supports multi-touch and
multi-user interaction.

The prototype device as it is right now, is assembled from available separate parts
which together don’t cost more than USD$ 350. The cost is bound to become lower and
lower as they refine it and mass-produce it, and it cannot cost more tomorrow than it
would cost today.
Right now, it looks very clunky and quite heavy. Since you need to wear it around your
neck, it might be quite uncomfortable to lug around. But again, it’s still very much in the
research stage and as they get closer to a (possible) commercial release, they will for sure
make much sleeker, lighter and cooler designs

3.3 HOW IT WORKS:

The hardware that makes Sixth Sense work is a pendant like mobile wearable
interface. It has a camera, a mirror and a projector and is connected wirelessly to a
 blue tooth smart phone that can slip comfortably into ones pocket. The camera
recognizes individuals, images, pictures, gestures one makes with their hands
Information is sent to the Smartphone for processing. The downward-facing
projector projects the output image on to the mirror. Mirror reflects image on to
the desired surface. Thus, digital information is freed from its confines and placed
in the physical world.

4. Sixth Sense Technology will Revolutionize the World:

The Sixth Sense prototype consists of a pocket projector, mirror and a camera. The
device is pendant shaped like mobile wearing devices. Both the projector and the camera
are connected to the mobile computing device in the user’s pocket. The projector projects
visual information enabling surfaces, walls and physical objects around us to be used as
interfaces; while the camera recognizes and tracks user’s hand gestures and physical
objects using computer-vision based techniques. The software program processes the
video stream data captured by the camera and tracks the locations of the colored markers
(visual tracking fiducials) at the tip of the user’s fingers using simple computer-vision
techniques. It also supports multi touch and multi user interaction.
 The device has a huge number of applications. Firstly, it is portable and easily to carry
as you can wear it in your neck. The drawing application lets user draw on any surface by
observing the movement of index finger. Mapping can also be done anywhere with the
features of zooming in or zooming out. The camera also helps user to take pictures of the
scene he is viewing and later he can arrange them on any surface. That’s not it. Some of
the more practical uses are reading a newspaper. Imagine reading a newspaper and
viewing videos instead of the photos in the paper. Or live sports updates while reading
the newspaper. The device can also tell you arrival, departure or delay time of your air
plane on your tickets. For book lovers it is nothing less than a blessing. Open any book
and you will find the Amazon ratings of the book. To add to it, pick any page and the
device gives additional information on the text, comments and lot more add on features.
While picking up any good at the grocery store, the user can get to know whether the
product is eco friendly or not. To know the time, all one has to do is to just gesture
drawing circle on the wrist and there appears a wrist watch. The device serves the
purpose of a computer plus saves time spent on searching information. Currently the
prototype of the device costs around $350 to build. Still more work is being done on the
device and when fully developed, it will definitely revolutionize the world.

Games Gamut
“All work and no play makes Jack a dull boy” – Jack grew older with this proverb and
has no penchant for play but loves to see his son play. Jack Jr however does not play but
is not dull either; he “games” on his PlayStation 3. This form of entertainment has
spawned many fads; video game culture has evolved in time, particularly in connection
with internet culture. Gaming in all forms has got all the attention be it console based or
web-based. Technological advances and commercial success helped the gaming concept
take a leap farther than anticipated, the gaming interface has emerged from a mocked and
rejected oscilloscope view to a 3-D user interactive view. The PlayStation, X-
Box, Wii and et al have become a genre in gaming and redefined the art and passion
involved. Apart from these LAN games, social networking games, online portals for
games, PC based games, smart phones and a gamut of other categorically identifiable
games have become a trendy frenzy.
 Would you believe that as of July, 2008 “The PlayStation” has sold a 102 million units
and to add to the trivia PlayStation 3 sold approximately 81000 units within 24 hours of
its release. The ongoing hoopla in the entertainment industry is not about Michael
Jackson’s 3D comeback or the 2012’s breathtaking CGI but the launch of a video game
“Call of Duty: Modern Warfare 2″; a record 4.7 million copies of the game were sold on
the first day raking up a whooping 186 million pounds. According to a report published
in The Telegraph, UK, Mike Griffith of Activision admits that this can now be seen as a
cultural phenomenon. The record shattering orders for launch of video games have
proved that these games have emerged as a new medium for entertainment. The social
networking websites are no exception on these numbers, Facebook’s Farmville has a
staggering 63.6 million users logging in almost every day across the globe and learning
the art of farming; ironically the plight of the farmers all over is not cared or understood
but a lost sheep or grown vegetable adds to the curiosity.
People across all age groups are smitten by this gaming bug and left them in a frenzy
over the blown-out-of-proportions technological advancements of these games. Five year
old Mike plays as one of the cheeky monkeys in the “Buzz! Junior: Jungle Party” along
with his other friends to figure out who is more cheekier. The Nielsen Active Gamer
Study in November 2005 revealed that the average age in a group of 2000 regular gamers
is 30 years and this has been increasing ever since. The gender divide is more or less
equal between males and females with the female percentage increasing. Apart from fun
there are games that are thought provoking such as treasure hunts, castle quests and
educational ones such as math blasters, word building which help the children build their
mental math and vocabulary. From what started off as fun has over the ages transformed
into passion and blossomed into art and culture.
As we take a peek to learn about the effort that goes into making this ground
breaking technology we surprise ourselves with the knowledge amassed. A multi billion
dollar industry with big players like EA sports, DMA design etc have a wide range of
products seeing the light of the day. Games like Halo, Age of Empires, Need for Speed,
Grand theft Auto and Moto GrandPrix are on top of the charts. These games have become
more user interactive with generations, A joystick to maneuver your car or swing your
sword or take a shot gives you a real-time experience.
The sixth sense concepts, real time interactions, internet, computer programming
advancements and game developer communities have ganged up to produce innovative
games. Though technically savvy and financially attractive this industry has its own share
of controversies; like related forms of media, computer and video games have been the
subject of frequent controversy and censorship, due to the depiction of graphic violence,
sexual themes, advergaming (a form of advertising in games), consumption of drugs,
consumption of alcohol or tobacco, propaganda, or profanity in some games. Critics of
video games include parents’ groups, politicians, organized religious groups, and other
special interest groups, even though all of these can be found in all forms of
entertainment and media. One of the most common criticisms of video games is that they
allegedly increase violent tendencies among youth. games like “death Race”, “Mortal
Kombat” are proponents of physical violence and have been banned for their content.

Controversies are bound to be a part of any realm but the big picture this industry airs is
more glamorous and attractive. A generation awakens to a new genre of entertainment
and indulges in a non-existent world of exhilaration. Video Games in all formats are the
future for fun and entertainment.
5.CAPABILITIES:
 Recognizes hand movements
 Examples from online demonstrations:
 Make a call
You can use the Sixth Sense to project a keypad onto your hand, then use that
virtual keypad to make a call.
 Call up a map
With the map application we can call up the map of our choice and then use
thumbs and index fingers to navigate the map.
 Check the time
Draw a circle on your wrist to get a virtual watch that gives you the correct
time.
 Create multimedia reading experiences
Sixth Sense can be programmed to project related videos onto newspaper
articles you are reading.
 Drawing application
The drawing application lets the user draw on any surface by tracking the
fingertip movements of the userâ„¢s index finger .
 Zooming features
The user can zoom in or zoom out using intuitive hand movements.
 Get product information
Sixth Sense uses image recognition or marker technology to recognize
products we pick up, then feeds us information on those products.
 Get book information
The system can project Amazon ratings on that book, as well as reviews and
other relevant information.
 Get flight updates
The system will recognize your boarding pass and let you know whether your
flight is on time and if the gate has changed.
 Feed information on people
The system will project relevant information about a person such as what they
do, where they work, and so on.
 Take pictures
If you fashion your index fingers and thumbs into a square ("framing"
gesture), the system will snap a photo.
After taking the desired number of photos, we can project them onto a surface,
and use gestures to sort through the photos, and organize and resize them.
 6.Applications of Sixth Sense Technology:

The Sixth-Sense prototype implements several applications that demonstrate the

usefulness, viability and flexibility of the system. The map application lets the user
navigate a map displayed on a nearby surface using hand gestures, similar to gestures
supported by Multi-Touch based systems, letting the user zoom in, zoom out or pan using
intuitive hand movements. The drawing application lets the user draw on any surface by
tracking the finger-tip movements of the user’s index finger. Sixth-Sense also recognizes
user’s freehand gestures ( postures ). For example, the Sixth-Sense system implements a
gestural camera that takes photos of the scene the user is looking at by detecting the ‘
framing ’ gesture. The user can stop by any surface or wall and flick through the
photos he / she has taken. Sixth-Sense also lets the user draw icons or symbols in the
air using the movement of the index finger and recognizes those symbols as
interaction instructions. For example, drawing a magnifying glass symbol takes the
user to the map application or drawing an ‘@’ symbol lets the user check his
mail. The Sixth-Sense system also augments physical objects the user is interacting with
by projecting more information about these objects projected on them. For example, a
newspaper can show live video news or dynamic information can be provided on a
regular piece of paper. The gesture of drawing a circle on the user’s wrist projects an
analog watch.
 Imagine being able to walk into a super market and having the ability to decide which
of the different varieties of products to buy based on preset personal preferences, simply
by holding the product in your hand. Or, while you are taking a stroll, you see this
fantastic photo opportunity and you don’t need to fumble around for a camera, simply
hold your hands up making a rectangular frame and the photo get taken… And if you
want to see how the photo looks like, you can project it onto any surface… the road, a
wall, even someone’s back, and zoom, resize and rotate the photo all with natural hand
gestures.

You are in a bookshop or a computer games store and want to know what other people
think about the book or game you want to buy? You don’t need to go searching for a
computer, or even use google in your clunky blackberry, simply hold the book (game) up
and you will get the amazon.com rating as well as excerpts from reviews (which you can
expand with a gesture, in case you want to read the entire thing), even notes your other
friends may have made about this particular product.
 You would never have to read a newspaper again, you could watch the up to date,
relevant news projected right on top of the paper. And you could get a tag cloud of
keywords pulled out of any individual’s blog or website while you are talking to them to
know more about them and their interests.

Hear in the above figure the person checks out his image dump, resizes the images and
arranges all the pictures using the sixth-Sense technology using figure tips.
Dialing a call using sixth-sense technology, by projecting on palm and using cap for
finger to dial a call.

Hear in the above figure the person checks out the road map using sixth sense
technology.
 Hear in the above figure the user of Sixth-Sense, projects the watch on his waist.

You are driving to the airport and want to know your flight status and which gate you
need to board from? No problem! Simply hold your boarding card up and the device will
tell you if it’s delayed or on time, and which gate you need to board from! You will not
ever need to wear a watch again or even carry your mobile… You just (gesture) draw a
circle on your wrist and voila, you will have the time. And you can project the number
pad of a phone right on the palm of your hand and use the other hand’s finger to dial a
call… Much better than the iPhone!

7.The advantages of Whirlpool sixth sense technology:

Whirlpool sixth sense technology offers a wealth of benefits, from giving you perfect
results on every cycle, to 50% savings in energy, water and time.

Personally I love the idea of bunging my clothes in the washing machine, pressing go,
and letting the washing machine do the rest – and saving energy (and money) in the
process!

7.1 How does Whirlpool sixth sense technology work:

In Whirlpool washing machines the intelligent sixth sense technology sensors firstly
measure the load size, then they adjust the water, the temperature and the time in order to
save energy and natural resources.

After this the machine continues to monitor and adjust the programme to ensure perfect
results and maximum efficiency.

The Whirlpool AWOE9558B and Whirlpool AQUASTEAM9769 washing machines are

some of the models which possess this sixth sense technology.

In addition, the AQUASTEAM9769 has the advantage of steam cleaning.

Steam cleaning works by injecting steam into the drum to relax fibres, allowing for the
deeper penetration of detergent and offering a superior level of cleanliness!

The beauty of steam cleaning is that it is and gentle on fabric tough on stains. This means
your clothes will look newer for longer. In addition a 20 minute steam clean will allow
you to refresh clothes, without having to perform a whole wash cycle. This is great for
removing odours like cooking smells or smoke.

7.2 BENEFITS:
 Cost: ~300$
 Software will be open source
 Many possible medical applications using the technology
 Surgical procedures (using tracking capabilities)
 Rehabilitation techniques
 8.Conclusion:

With this exciting technology just around the corner, the world will suddenly become
more meaningful for everyone as they get relevant information about what they are
looking at and interacting with right away. And maybe we will all know the Latin (and
common) names of trees and flowers just by looking at them.
The current prototype system costs approximate $350 to build. He said it will be open
source so that every one can use it at mass level without any problem. This is really a
very big achievement and still more work is being done on the device and when fully
developed, it will definitely revolutionize the world.
The current prototype WUW sixth sense design is not sleek or somewhat attractive to
be wearing around all the time but hopefully they can make a more sleeker design in the
near future. Wearing a camera system around your neck or attaching it to your cap on a
day to day basis would get annoying sooner than later.
Today, we are all so proud of Pranav Mistry for developing this unique technology
called 'Sixth Sense'
Sixth Sense recognizes the objects around us, displaying information automatically and
letting us to access it in any way we need. The Sixth Sense prototype implements several
applications that demonstrate the usefulness, viability and flexibility of the system.
Allowing us to interact with this information via natural hand gestures. The potential of
becoming the ultimate "transparent" user interface for accessing information about
everything around us.
9.Future scope:

It has been a few years now since Microsoft announced their Surface technology. Touch
technology, access to data, game functionality, etc., etc. It all sounded interesting if only
it were not so terribly expensive and equally removed from being anything that could be
confused with being portable. Of course since then mobile based technologies led by the
ubiquitous iPhone abound, but a team over at MIT is attempting to take things to the next
level with what they are calling the "Sixth Sense." The team points to the Surface-like
touchpanel that Tom Cruise's character fingers throughout "Minority Report" to explain
it, but since I don't recall much of that I'll stick with the MIT company line on Sixth
Sense instead. They describe it as,

"A wearable data device that paves the way for data-rich interaction with the world."

Now this sounds an awful lot like a modern cell phone, but the concept of Sixth Sense is
different from this. What the working prototype demoed at this year's Technology,
Entertainment, Design (TED) Conference displayed was an attempt at direct tactile
interaction with data in real life situations. The portable set up consisted of a collection of
devices, including a mini projector, a camera, a cell phone and color-coated finger tip
covers. With the right image recognition or marker technology attached to items users
want to know more about, Sixth Sense could project set pieces of information anywhere
using cell phone's access to the Internet for data gathering, and/or through the
combination of projection and input recognition via finger tip attachments, allow users to
input commands to search for a variety of related information. The examples used in the
demo included product resources accessed and projected on the products themselves in
the supermarket and content, like Amazon.com book reviews, called up through contact
with a physical book copies. The device also will/would associate certain programmable
hand gestures to specific tasks, such as taking a picture of the contents of a user's framed
hands and projecting the current time in a specific spot. Nice, no more watch.

Interesting stuff this, and because of its portability and price -- estimated at not much
more than $300 or so -- much more achievable on a large scale than Surface. But at the
same time, it would also be very reliant on image recognition and marker technology,
which the Surface uses and involves chip insertion into all items in use. Also, at the end
of the video below the MIT speaker, Pattie Maes, makes a comment about a brain
implant being another step in the unit's development. Whether she was kidding or not,
myself not being quite ready for the hive mind, I don't think that would ever be a wise
move, but if this technology ever became a reality at any level just think how it might
alter personal electronics as we know them

The Future of Mobile Technology: The Sixth Sense?

Although the challenge today is integrate current technology in today’s classroom in a

way that is useful and practical for personal productivity and for learning, it is always
interesting to see what the future of technology will be and the potential uses for it. In
this newly released TED video, Pattie Mae and Pranav Mistry demo their wearable
technology nicknamed “Sixth Sense” and discuss what may be possible in the near
future. If we can access all of the information we need from wherever we are, what is the
need for memorization?
10.Bibliography:
 Sources:
 “About (Pranav Mistry)” pranav::mistry. 2010. Apr 1 2010.
http://www.pranavmistry.com/
 Mistry, Pranav. “About (SixthSense)”. 2010. Apr 2, 2010.
http://www.pranavmistry.com/projects/sixthsense/
 “Pranav Mistry’s: The Thrilling Potential of SixthSense Computing-Future of
Interface”. Boxesandarrows. Feb 2010. Apr 1 2010.
http://www.boxesandarrows.com/idea/view/51321
 http://www.google.com
 http://www.slideworld.com
 http://www.seminarprojects.com/Thread-sixth-sense-technology--7186?
pid=18932#ixzz1BpRihoou

 Publications:
 P. Mistry, P. Maes. SixthSense – A Wearable Gestural Interface. To be appeared
in SIGGRAPH Asia 2009, Sketch. Yokohama, Japan. 2009
 P. Mistry, P. Maes. SixthSense – A Wearable Gestural Interface. To be appeared
in SIGGRAPH Asia 2009, Emerging Technologies. Yokohama, Japan. 2009
 P. Mistry. The thrilling potential of SixthSense technology. TEDIndia 2009.
Mysore, India 2009
 P. Mistry, P. Maes, L. Chang. WUW - Wear Ur World - A Wearable Gestural
Interface. In the CHI '09 extended abstracts on Human factors in computing
systems. Boston, USA. 2009