project premium

Sketching Hybrid Interactions

Sketch-a-TUI is a toolkit for sketching tangible user interfaces on capacitive screens. Physical objects of almost any material, also everyday objects like bottle caps, can be used as user interfaces. Markers consisting of conductive ink and tape make it possible to track physical objects.

A video demonstrating the marker creation and use of the iPad application can be watched below:

The iPad application provides basic output elements (2D shapes, controls, sound), which behave depending on the movement of the physical objects they were assigned to.

Sketch-a-TUI provides a way to explore tangible interaction in a quick, cheap and easy way. It could be part of an interface lab in a design studio, providing an opportunity to become familiar with different forms of tangible input modalities.

In our interviews with potential users (interactions designers, usability engineers with an average of seven years of professional experience), we could observe that Sketch-a-TUI was used in a playful way, just like a toy. It supports communication and stimulates ideation, facilitating the exploration of form and interaction. Therefore, it serves as a tool for sketching interactions.

Download the source code of the Sketch-a-TUI application.

Have a look at our tutorials and templates for cardboard objects.

Posted 19.03.2012 by Fabius Steinberger | no comments

Creating the toolkit

It’s the year 2020. Car sharing is big. Image you’re an industrial designer trying to improve the in-car experience. Many cars have a central information display, kind of like an integrated iPad for controlling navigation, contacts and other features.

Your task is to come up with a safer and more personalized way of interacting with these features. So let’s try out tangible user interfaces that are attached to the screen with magnets. They don’t distract you as much from driving and they can be assigned to the current user’s favorite features.

We had this scenario in mind when we were asking ourselves who would use our toolkit.

Then we got to work.

Designers love using random everyday objects when trying things out. But how do you make a capacitive screen recognize these everyday objects? We tried capacitive ink, but found aluminum foil attached to double-sided tape in some ways even better.

With aluminum foil it’s easier to bridge the gaps at the edges of objects. It’s easier to create precise touch points for the capacitive screen (these act as IDs, so this is important). It doesn’t wear off quickly.

So what does it do so far? Watch this video.

It shows how two random objects were chosen as tangible user interfaces, attached to aluminum foil and put on an capacitive screen. The toolkit recognizes different objects and lets you assign behavior to a tangible.

Posted 27.11.2011 by Fabius Steinberger | no comments

Into the wild

When creating a new system it is always valuable to ask potential users about their needs and expectations. We are aiming to create a prototyping toolkit for interaction designers that lets you quickly try out different tangible user interfaces on capacitive screens.

So we went into the wild and asked around. What do you expect from a prototyping toolkit? What tools are you currently using and why do you love them? What problems occur when art and technology meet? What’s important in the early stage of the design process?

It was important to us to get qualitative input from experts. So we asked people from the industry and met with usability engineers, industrial designers, game developers, interaction designers and user experience experts of Designaffairs, Google, Ravensburger, Oxxid and Hochschule München.

This way, we found out that designers need their freedom and want to be flexible in the early stage of the design process. They enjoy having standard UI elements available to get started, but also want to be able to modify them. They like to imitate system feedback for the user without programming efforts. And many interdisciplinary teams provide their creative people with a collection of the most exciting electronics and materials so they know what’s possible and what could be included in their current project.


Posted 26.11.2011 by Fabius Steinberger | no comments

Microsoft Surface Tutorial – Creating a custom TagVisualization

Aim of this tutorial is to demonstrate a basic example of building a TagVisualization with self-designed visual appearance. Therefore we want to realize an object triggered equivalent to the touch enabled version of a control-dial, presented in a tutorial on the Microsoft Internet platform.


The control-dial on Image 2 is controlled by a physical object, which can be placed above the black pointer (1). As a problem, a TagVisualization always moves with the tagged object, which means in our case, that a rotation of the tag causes a rotation of the whole object (1+2). To avoid that effect, we import two separated PNG files for the pointer (1) and the menu (2) to adapt the characteristics:

First of all, we don’t want the TagVisualization to rotate automatically. To make the pointer rotate and achieve the menu to stand still, we set in a first step the UseTagOrientation attribute to false. Additionally we have to add a method, which tracks the movements of the User and renders the rotation only for the Pointer. All in all, the XAML will include the following code:

:Name=”controlDial” Height=”x” Width=”y” …/>

Per default, the middle of the graphic is covered by the tag. In our case, we want to move our center on the Pointer. Therefore, we have to adapt the Container – for example a grid – of the TagVisualization (cf. Image 2).In the body, there will be to references for the PNG-Files, which are positioned within a Grid. The margin is important, to assign the center of the rotation for the Pointer.

Image Source=”/Framework1;component/Resources/Pointer_Menu.png” …/>
Image x:Name=”pointer” Source=”/Framework1;component/Resources/Pointer.png” Margin=”x” …/>

The last step is to render the Pointer manually in the code behind and to trigger actions, when the angle is in a particular degree. The center of the rotation has to be adapted to the margin and position of the pointer.

private void TagVisualization_Moved(object sender, TagVisualizerEventArgs e){

Double orientation = e.TagVisualization.TrackedContact.GetOrientation(this);
render = newRotateTransform(orientation);

render.CenterX = x;
render.CenterY = x;
pointer.RenderTransform = render;

int angle = (int)orientation;

   if (angle > a && angle < b){

     //trigger method



    //trigger method


Posted 24.07.2011 by Christian Orth | no comments

Prototyping Concept

Motivated by the outcomes of our study with the NEFF design course, we thought about possibilities to create mutable understanding among team members and to accelerate the proceedings by focusing on the actual prototype instead of abstract ideas and opinions. As soon as you have got something graspable, you also have a starting point. Something you can talk about. So it might be desirable to come up with something graspable as soon as possible!

We developed a prototyping concept, which – after the first ideas of a product are recorded in a scenario – starts with a low fidelity 3D-paper prototype. That prototype will be improved iteratively until finally a functional and well designed prototype is built.

During a pilot project together with an industry partner we will try to find out whether this approach is useful for real product development. After a prototype is finished we will conduct a constant questionnaire with professional designers and ask them about their opinion on the prototype. Furthermore we are conducting user studies to find out whether external persons can understand and use a low fidelity 3D prototype – as sometimes a good ability to abstract is necessary.

Posted 23.06.2011 by Hanna Schneider | no comments

A User Study With An Adaptive Prototype

To find out, if we are able to enhance user participation with an adaptive prototype we arranged a participatory design session. One challenge according to the recruitment was, that the video editor has been developed, to help people with a minum of computer experience to design a crossover between two videos. Therefore we had to recruit a special group of users, which we reached by means of personal contacts and lectures for seniors. Especially the senior students were very interessted and open minded.



For our research, we want to find out whether our framework can improve communication between designer and user. Our key questions:

1.) Is communication happening?
2.) Is it a creative communication?
3.) Does the creative process influence the choice of favorite controls on the side of the user?

Furthermore it seems interesting that all those questions concern high-fidelity, and NOT low-fidelity prototyping, which has its main focus on communication during the design process.

Posted 14.06.2011 by Christian Orth | no comments

A Digital/Physical Video Editor

Inspired by Jamie Zigelbaums “Tangible Video Editor”, we decided to pick up the concept of collaborative video editing.  As a result, the application which is accessed by the framework depicts a very basic video editor. According to the idea of assisting less experienced users with video editing, we prepared a simple use case: two videos can be merged with two different crossovers. The control for the selection will be specified by the framework – in the example beneath, the video can be manipulated with a control-dial.

Posted 08.06.2011 by Christian Orth | no comments

A Framework For Adapting Prototypes During Participatory Design Sessions

Hybrid user interfaces offer a lot of new possibilities for interaction. During the process of developing a new user interface, the interaction designer has to consider a wide range of alternatives. If the designer worked with a single static prototype, he would has to evaluate each possibility on it’s own. Additionally a programmer would has to implement the prototype over and over again.

To gain higher efficiency concerning time and decision making, a multiple prototype can solve those problems: during a participatory design session, the prototype can be adapted to individual requests of participants. As a result the possibility to try and to compare different interface approaches will enhance communication between user and designer.

In our research we will aim for a multiple prototype. Therefore, we will realize following tasks:

  1. Implement a framework which offers the possibility to choose the control of an application.
  2. Implement a very basic application, which is accessed by the framework.
  3. Evaluate the application and the framework.

As a working environment we have chosen the interactive table “Microsoft Surface”.

Posted 01.06.2011 by Christian Orth | no comments

Based On Experience

During the last weeks we met almost all students who participated in the NEFF Design Workshop for an interview about their team work experiences. Every group of students consisted of two Design students from the ‘Hochschule München’ and two Media Informatics students from the ‘Ludwig-Maximilians-Universität München’. Two students from the same field of expertise who have been working in the same team were interviewed together. And indeed many teams encountered related difficulties.

The biggest differences between the designers and the developers approach to product design appeared in many teams during the ‘brainstorming phase’. On the one hand three out of five designers felt slowed down by developers who tried to check the practicability of ideas already when they came up. On the other hand developers were unfamiliar with brainstorming -  in two teams they felt like it was too much of it. When designers came up with ideas that were not realizable in the developers opinion the developers sometimes felt like the designers had expectations they could not come up to. The sticking point is that in the designers point of view practicability is not important during the early brainstorming phase – it is all about gathering as many ideas as possible and thinking out-of-the-box!

Posted 23.05.2011 by Hanna Schneider | no comments

Key Challenges Designing Hybrid Interaction


Systems with hybrid user interfaces are getting more relevant for industries and researchers. Hybrid interactions are those which involve physical as well as digital interaction.

In this work we will examine the difficulties when researchers and developer work closely together with designers in order to come up with innovative developments – especially when it comes to hybrid interactions, which have barely been discussed in the past. Previous works have pointed out the difficulties of communicating in interdisciplinary team projects from the designers perspective. We want to add the developer perspective in order to complete the list of challenges, which are currently faced during the design process.

1. As a start we will evaluate the experiences of an interdisciplinary team work course which was conducted by Andreas Butz and Alexander Wiethoff at the LMU in cooperation with the industrial design department of the Design FH Munich.

2. Based on the key learnings of this study, we try to find procedures and methods to improve communication and cooperation during the development process. Previous have already stated that there is a need for innovative practices to facilitate shared understanding.

3. The suggested strategies will be evaluated by applying them to a design project which I will conduct in cooperation with an industry partner who aims to develop an innovative Tangible User Interface.

The prototyping platform during that project will be the iPad.

Posted 09.05.2011 by Hanna Schneider | no comments