Welcome Tomas Gecevičius portfolio

Share this Project:

"Smart ARbnb"

Augmented reality smart home interface with VLC


The smart home system is the next stage of home evolution. Currently, an additional trend of peer-to-peer rental has already spread in our society, for example, Airbnb. Smart homes bring peace of mind and resources savings for Airbnb hosts and a unique experience for the guests. Nevertheless, current solutions for controlling smart home systems are not suitable for shared-home context, regarding privacy, accessibility, and ease to use. In addition, problems like unknown smart home system automations can bring guests unpleasing experiences during the stay.
SmartArbnb is a web application that combines augmented reality and visual light communication to control smart home devices. The tool uses a mobile camera to scan IoT communicators, let users change connected devices' settings, and visualizes the current set automation. Additionally, guests can use presets to change the atmosphere, creating a wonderful experience

Design process

Huge designs like the final master project require to be broken down into manageable steps. In order to tackle this project - Constructive design research [1], inspired process is used. It helped to untangle the project, by defining the design space, performing the research, and finding the main problems. After synthesizing the research and concept creation, a prototype was created that helped to evaluate the project, create iterations, and deliver the design.

Design context

To start the project, firstly, the design context needs to be defined. In the final master project, the union of the smart home system and peer-to-peer accommodation, in this case, Airbnb is selected. The main stakeholders are the hosts that provide the accommodation which has smart home system and traveling guests who lives in provided home.

Research and findings

To dive deeper into this design context and understand what kind of problems stakeholders encounter with smart home systems or with the stay, for what kind reasons guests choose Airbnb and what they expect from it, research was conducted. It consists of three main parts:
• interviews with the hosts
• interviews and customer journeys with Airbnb guests
• and literature research.

After conducting the research with the guests and hosts main pains, problems, and expectations were found. The main benefits to use the smart home system for the hosts are peace of mind that the property is secure and monitored and saving costs from optimizing the electricity and heating. From the interviews with guests dominant issues with the smart home system like usability, accessibility was raised up. In addition, research showed that guests use Airbnb not only for sleeping but also for entertainment, like watching movies or cooking together.

The concept

The proposed system is called “Smart ARbnb”, a human-system interface that uses the web application with augmented reality to interact with the smart home and communicate the smart home behavior through light patterns. “Smart ARbnb” is a product that consists of two parts - a small IoT communicator and a Web application

Web application

The first part of the prototype is a web application. The website is a tool that uses a mobile camera to detect smart appliances in the environment by reading IoT communicators’ light patterns. In the Airbnb context, the guests should have a quick and simple way how to interact with the smart homes.

Easy accessibility

It's just a website! Open the link or tap the IoT indicator with your phone with NFC and enjoy the possibilities. No apps, no hassle.

On-spot interface

Quickly change the individual device settings or select one of the predefined scenes to create perfect atmosphere for the memorable holiday!

Full security

The interface can be accessed only in your home WiFi network and after physically detecting IoT indicator. So no checked out guests can interfere with next guests!

Automation overview

Simple way to show and control the existing smart appliances, upcoming automations or routines, to ensure stable and understandable stay.

Atmosphere selection

Create the environment that fits your activities or mood, for example, "Movie Night", which could dim the lights in the living room down, pause playing music, lock the door, turn on the TV, and open Netflix on it.

IoT communicator

The second part is the IoT communicator which is a battery-powered coin size device that is attached to each smart appliance in the home. IoT communicator has an integrated LED light and WiFi module that is connected to the smart home system. This device uses visual light communication, a specific, morse code inspired, light pattern to broadcast the identification number of the smart appliance.

Light patterns

Showing the upcoming automations

Letting know about the coming automation or routine beforehand can save the guests from some uncomfortable and confusing moments.

Indicating problems with the device

Even sturdiest appliances brakes. When it happens IoT indicator can tell what exactly is wrong and show how exactly guests could fix it.

Leaving a message

Create an amazing stay by sending a surprising note to your guests or tell about other important or relavant information.

How it works?

1. Look for the IoT indicators

Find the blinkink IoT indicators through your mobile phone camera

2. Control the device

Toggle the device by tapping the icon or go to the device menu and access additional settings

Change the atmosphere

Select the scene which would make the perfect atmosphere for your stay.

For this project to work, web application needs to be able to read the blinking pattern of IoT communicator through the mobile camera. It is achieved by taking each frame of camera footage, scanning for pixel clusters - blobs, selecting the ones that fits in the specific thresholds and finally, recording the pattern. The pattern later is decoded and the smart appliances information and settings are taken from the smart home system.

Concepts evaluation

In this section, I briefly summarize the concept evaluation. More detailed information can be found in the report. A user study is conducted to evaluate the design concept and the usability of the prototype. The study consists of five parts - learning about participants’ Airbnb experience, evaluation of interactions usability and experience, UI control usability, and ease to use, IoT communicators’ light pattern recognition and business validation. During the study, to learn about the usability of the “Smart ARbnb” system, the participants were asked to perform a list of tasks, and their actions were observed. Furthermore, to understand if the user experienced the interface as simple to use and useful, the modified “Perceived usefulness and ease to use” [2] questionnaire was given to fill in. Next step, the evaluation of the light patterns' perceived meanings is done based on the Y.Chuang study process [3]. Finally, the desirability of the concept in the Airbnb context is understood from interviews with the participants.

Findings and improvemets

After performing the user study, the main problems and insights are found. By observing the participants, it was visible that some parts of the interface are not so clear and understandable. Nevertheless, the users emphasize that it is easy to use application and it has an exciting and fun experience. The most highlighted feature was atmosphere selection. In Don Norman's book - “Emotional design”, the author mentions that when the product is fun and pleasing to use, the user overlooks the products’ imperfections [4]. In this project, user study participants were surprised and excited when trying out the web application, which could explain why the user interface problems did not influence the overall positive experience. More insights can be found in the report. Nevertheless, based on the user study, new improvements are proposed:

Atmosphere routine

Participants couldn’t find the atmosphere overview button, so the scene automation routine is brought up in front to increase the visibility

Icon redesign

Furthermore, from the behavior of the participants, it was noticed that they were not aware that the icon is also a toggle button, so to improve that, it is also redesigned

Interface adjustments

To reduce clutter on the screen, the interface would expand only when it is centered. Nevertheless, participants often would move the mobile phone when trying to reach the slider, thus loosing the IoT communicators view on the camera and making the interface of that device dissapear. To improve that, the controls of the appliances will be put on the separate pop up which can be accessed when pressing the menu button.

Automation overview

The smart home automation overview tab is changed to a separate more visible button and a blue outline signifiers are added to the linked smart appliance icons for users to find the connected devices faster.

Onboarding redesign

The user study showed that the current onboarding form is not clearly understood and it was not engaging enough for participants to keep the attention to it. To solve this issue, a new onboarding process is presented. The new tutorial will be overlaid on top of the web application main screen and would indicate where each important element is located and what exactly is for.

IoT communicator change

Based on the finding of the user study, the IoT communicator design needs to be changed to make the LED light on mobile cameras more visible from a wider angle. To improve readability and visibility, the additional design concept exploration is done and a proposed new version is selected.This physical design should have an outer semi-transparent plastic case that would diffuse the LED color and would be visible from all 360-degree angles.

Business case

Based on IDEO [5], when building the business it is important to have a harmony of three aspects - desirability, viability and feasibility . In this Airbnb and Smart home context at first it is important to understand the desirability part. At the beginning, to be sure that “Smart ARbnb” is positioned around the host needs and values, the Value proposition canvas and business model canvas are made which can be seen below:

Furthermore, it is important to check if Airbnb hosts are seeing the benefits of the concept and if they are willing to implement this system in their accommodations. To do so, it is essential to create a proposition. To clearly convey the message in an accessible, engaging, visual way a website https://smartarbnb.com is created. In the website it was tried to clearly write what is the “SmartARbnb” and what kind of values it can bring to the Airbnb hosts. To learn about the desirability of the “Smart ARbnb”, Airbnb hosts are approached through the Airbnb community page and are asked what they think about this product. Overall desirability is contrasting, while some hosts would love to implement “Smart ARbnb” system, others are afraid that it might bring more confusion for their guests or system can malfunction bringing more problems. To increase the desirability for the hosts, the interface needs to be as easy to use as possible and more clear value proposition created.


In this project I conducted a research about the Airbnb and smart home systems context, created, prototyped, and evaluated smart home interfaces with augmented reality and visible light communication. The design concept and prototype were developed which would use a mobile web browser to detect light blobs and patterns through mobile camera to receive information from the IoT communicators and show augmented smart appliances control on the screen. While the user study showed that the feeling being around IoT communicators blinking is slightly negative and the user interface of the web application has some understandability issues, the overall experience of the interaction and system appears to be positive and users would like to see “Smart ARbnb” system inside the next Airbnb accommodation place. In future work, IoT communicators design needs to be prototyped and its feasibility checked out, the web application user interface should be revised and evaluated regarding the understandability of the automation overview and onboarding. In addition, the business case should be evaluated with more Airbnb hosts and additional benefits should be found to increase the value proposition.

To see next project, scroll down more or press a button:

Next Project

Early diagnostics for hearing loss

Are you sure that you can hear everything?