The latest work we’ve been doing has revolved around the design of the TUI (tangible user interface), the software (mobile and Surface), and our test plan. For now, I want to write a bit about the TUI design and save the other topics for later.
The TUI has undergone many iterations, on paper and as foam cutouts. The challenge we kept having was form factor with regards to the human hand, the tabletop, and the content. We knew we didn’t want just a simple brick, but we also didn’t want to do something crazy just to be different. After generating several ideas that weren’t that good, we finally hit upon something with promise.
Sketch of the original design
I started off by doing a number of sketches around the concept of a loop based TUI…something like a monocle, but with a handle. When I had something that I thought might work, I created a more detailed sketch with measurements and then sent off to a contact we had with expertise in 3D modeling.
TUI 3D Model
The model looked fantastic and, as luck would have it, we had someone with access to a 3D printer who was willing to help us out. We tried two different printing techniques but since this was being done as a favor, I didn’t want to take advantage. However, if I had access to my own 3d printer I could see spending quite a bit of time iterating this design to perfect it. It’s usable right now, but I think the feeling and size could be improved.
Printed TUI
TUI Test Video
The video above is from our first live test this afternoon after we took possession of the printed TUI, and the photo is simply a closeup view of the same TUI in the video. We’re currently aiming to perform user testing next week, but still have lots of work to do to reach that goal.
Tuesday I was able to complete a rear diffused illumination prototype. It took about a week and a half for my IR LEDs to arrive from the U.S., but that gave me some time to tryout different mounting positions for the projector and mirror combination as well as continue work on the gesture designs I’ve been putting together around multitouch and TUI.
A front view of the multitouch tabletop prototype
Manipulating content
CCV interface and calibrating blob sizes
Calibration Screen
Now that I have a functional multitouch tabletop my next step is to generate fiducial markers and start tracking them. While this prototype will likely iterate a little bit, as needed for performance reasons, there will be very little of the physical components that make it into the next version. However, the knowledge encoded in this prototype is what is important for the next version. What works, what doesn’t, and how it can be improved…aside from that, it’s mostly just the IR LEDs that will be recycled.
I do not like most Microsoft products as I find them, on the whole, poorly designed, uninspired, and difficult to use; however, Microsoft produces some killer research. Take OmniTouch for example:
That’s just killer. What amazed me most is that they’re able to calibrate the computer vision software (it looks like OpenCV but I’m not sure) to actually register “touches” or “clicks” on buttons and other gui assets. Now they just need to shrink the electronics down to the size of a Jawbone and we can all walk around utilizing any surface as a multitouch interface.
Naturally, I see this idea fitting very well with my work on dynamic fiducial markers. This system could easily detect and identify a blob, or marker, and then project content based on that marker on the surface next to it. Users could then utilize the marker as a handle for their content and interact with it using a variety of gestures.
Last week I really scored. I was able to salvage an old prototype from a different tabletop project and set it up near my office in Kuggen. It was originally a prototype for a finished tabletop computer that is centered around emergency response scenarios, and employed pulsed IR tangibles to control the system. It had no multitouch capability, so to get it working for my purposes I’ve ordered 5m of IR strips (850nm) to place around the outside of the acrylic to create a Frustrated Total Internal Reflection (FTIR) multitouch surface.
Tabletop Prototype 1 - Rear View
Side View
The projector is an FL32 by Projection Design, and is seriously hardcore. It has about the best looking image I’ve seen on a projector to date. I have no idea how much they cost but would definitely like to have one at home to watch movies with.
As you can see, the set up is fairly open and adjustable…which has good and bad points to it. The good is that it can be moved around easily and it’s easy to access to work on and adjust, tweak, etc. The bad is it’s easy for everything to get out of alignment and the mirror needs to be a good distance out from the table to reflect a wide enough image on the tabletop. This makes using it from all four sides impossible, but not really a huge deal for my purposes at this point.
I was hoping to include my Google Form but that is one seriously buggy and unusable piece of software. I like most of Google’s stuff, but wow…their support for online forms has fallen through the cracks. Now, I must find a suitable (free) replacement for my survey questions. What I’m hoping to start building data around is the use of smartphones and what applications people use, and also the ways in which people routinely share and collaborate with content on their smartphones (e.g., taking a photo with your phone and then showing it to other people).
Since this post is about prototyping in parallel, I want to mention the other path that is still in consideration is the Microsoft Surface. I did some extensive reading about the development environment and, for me, it seems like too much overhead for what I want. Additionally (and I may be very wrong about this), it feels like the Surface is good if you want to write applications that the Surface is more or less ready for out of the box but isn’t that flexible when it comes to something completely new; whereas, if I go the open source route I may have more flexibility in that area. At any rate, I’m hopeful to bring on a partner with better programming skills than myself and we can hash that out between us.
Google just launched their Google Wallet. You can see an explanation of it below:
One thing that came to mind was the chance I wouldn’t be able to complete the payment myself and have to hand the card to the cashier. I’ve seen lots of systems where this doesn’t happen, and in Sweden I rarely ever have to hand my card over (a Nordea Electron card), but what about when you do encounter those situations. Do you want to open up your digital wallet and then hand your phone over to some stranger just to complete the transaction. I’m less concerned with security than I am with just having someone else’s hands on my phone…which I sometimes I have right next to my face.
One big advantage though is that now, when I lose my credit card, I won’t have to wait two weeks for a replacement. Theoretically, I should be able to remotely erase the data on my lost phone, go to the store and buy a replacement, and synch my phone with all of my backed up data. A couple hours, tops. I guess the only question is: how will I pay for my replacement phone!
Last week our tabletop interaction lab hosted a workshop for visiting professors and PhD students from Europe, US, and Japan. It was great being exposed to all of the interesting work being done elsewhere and being able to bounce ideas off these people. These were long, action-packed days though so it’s also good to now have some time to reflect on last week’s events.
I presented my S.M.A.R.T.T. concept and gathered some tremendously helpful feedback. What I most feared, didn’t happen: someone else having done the exact same project (which has already happened to me a couple times). But I do need to zero in on the specifics of my research and define what is the new information I am adding to the overall body of work that’s been done already. It’s quite different from product development in that I’m not solving for an end user or customer, but am solving for a more abstract quality of novelty and uniqueness.
With that thought, my working research topic is “dynamic fiducial markers for tabletop computing”. So far, this seems to be the unique value I can add to the existing body of knowledge. Under this though, there are several subtopics:
Smartphone/Tabletop interaction
Use of EPD (electrophoretic displays)
Token design
Visual design of token information on a tabletop computer
Interaction framework of the S.M.A.R.T.T. system
To drive my research I need a hypothesis to test against. Right now I have four of them, but again it’s always in progress and working:
Dynamic markers make interacting with a tabletop computer easier and more fun.
Tokens are more secure and power efficient.
Tokens can enable a range of device interactions from smartphone to tablet to ubiquitous computing environments.
Multitasking is easier with tokens acting as handles for digital content.
During the workshop I got turned on to an excellent book on research that I’m reading right now, “The Craft of Research” It offers methods for planning research, making good arguments, and logically connecting your claim to reason based on evidence.
For next week’s workshop I had to submit a short paper and a presentation for peer review. Since I have just started my work, there was little in the way of actual content for me to present; however, I was able to complete a cycle of paper prototyping and put it together into a cohesive story.
Instantiating a blank token with content from a smartphone
The working title is S.M.A.R.T.T.: Simple Multitasking with Assigned Rewritable Tabletop Tokens. It’s an interaction framework for supporting multitasking between a smartphone and a tabletop computing environment.
The token is now instantiated with my twitter credentials.
Utilizing NFC and Electrophoretic Displays, the intent is to have small tangible user interfaces that anyone can use to manipulate content on a tabletop computer. A smartphone happens to be a device that can serve as authentication (who is this user?) and also happens to have a lot of content that has already been curated by the user, which can help us avoid menu complexity on the tabletop. An interesting byproduct of this design is that it requires TUIs with the ability to dynamically change and display state.
With a token placed on the tabletop, my twitter feed is exposed.
Another potential benefit is that we can lessen the demand on the smartphone battery by offloading this to the tokens. There are examples of smartphone/tabletop interaction (mostly via a Windows Mobile device and Microsoft Surface) where the phone is placed on the tabletop and then photos can be swiped into and out of the device; however, if we are to extend this interaction model further we run into two problems right away. First, the phone, by nature of its small screen size, is modal…even if it supports “multitasking”, it’s modal in the sense that I can only really interact with one application at at a time. Secondly, leaving my phone on the tabletop with the screen on and the radio (bluetooth in the Microsoft Surface/Windows Mobile example) will quickly drain the battery.
Multitasking smartphone content on a tabletop with multiple tokens
Next steps are to solicit feedback from next week’s workshop and then continue prototyping work. Once a functional prototype is in hand I can begin iterating the design based on user feedback.
Next week I’m presenting a rough outline of my research work at a workshop centered around multi-modal, mobile, and tabletop user interfaces. My topic is a multimodal framework for smartphone and tabletop interaction. Specifically, I identify four problems with the current interaction model between smartphone and tabletop computing and then motivate a specific solution I have in mind. Essentially, this is laying the groundwork for the rest of my research work.
As part of my presentation (it’s short, only 8 minutes and a very brief paper), I am paper prototyping my design and taking photos or creating a very short video to capture it. Paper prototyping is a design method I’ve used quite a bit, and with great success. I like it because it’s fast but has about the right level of fidelity, especially when working with non-traditional products or design challenges. It also lends itself to quick iteration, quick feedback, and the overall process of sense making when you’re in the early (what I call “thrashing about”) stages of the project. It’s a very inexpensive method as well, with lots of great information potential around usability and design direction for functional prototyping.
Paper prototyping isn’t always suitable, so it must be applied judiciously. But I have a very specific use case that I want to quickly test and this vector of prototyping at this stage will serve me well in creating boundaries and structure to my thinking.
As part of my exploration of contexts of use for tabletop computing, I’ve been studying emergency response and how command-control systems operate. I’ve read “Designing Information Technology for Emergency Response” and also had an opportunity to meet with the author since he is working in a nearby office. I also read “Command and Control in Emergency Services Operations“, which presented some very interesting observations regarding the network topology of emergency systems and responders.
One interesting project I’ve found in this area is from the University of Massachusetts Lowell Robotics Lab. While I like the execution, I find that it fails in some of the same respects that all my ideas thus far have as well. Namely, it seems that emergency response coordination is typically distributed and so having a central tabletop computing system may not actually provide that much enabling value. Secondly, there is a distinction between systems that support sense making in the context of highly dynamic, stressful situations and situations that allow for more time and research (which this system from UMass seems to support better). And finally, these systems need to be portable to be useful. A large tabletop system with a bulky enclosure and sensitive equipment will not work well in the emergency response situations I’ve been reading about.
Recent Comments