Exclusive: Clayton Miller on Interface Design
A while back, many of us in the UI/UX world were blown away by a concept video called “10/GUI”. Predating the iPad and all other large-scale multitouch displays, interface designer Clayton Miller discussed a completely new method of integrating touchscreen on the desktop: a UI designed around gestures that kept some of the best elements of a desktop computer with the flexibility and usability of a tablet computer.
I was so taken by the concept that I contacted Clayton to discuss gestures, multitouch, interaction design, and whether his ideas have changed since the iPad and other tablets have arrived. Fortunately, he was quite forthcoming and a pleasure to chat with! Enjoy the interview.
What’s App Culture Got To Do With It?
APPCULTURE: Due to its windowing system, 10/GUI’s horizontal window system seems inherently designed against multitasking, much in the way that iOS is (despite its underpinnings). I argue that this is a good thing, that single-task focus combined with optimized and specific “views” is the most efficient way to work, but that there are complications. Do you think that your design arose mainly out of frustration with multi-layer window management, or as part of a broader response to App Culture and the mobile world’s system of compartmentalizing personalized tasks?
Excellent question! The idea for Con10uum had its roots simply in the question of “what is the advantage of being able to move and scale windows in two dimensions?” The answer of “to see different information side by side” didn’t seem to require the amount of latitude most window managers have, so that was really my task, to narrow that latitude to only what was needed and to put what we didn’t need to control on rails.
App culture is an undercurrent of that, though. I think mobile has shown the need to see everything as a suite to be much more of an edge case than we’re conditioned to think it is. Sometimes we do need to see things together, but our windowed UIs are built around this old assumption that we need to work that way all the time, and with much greater granularity than we almost ever do.
APPCULTURE: Definitely. In this area I tend to attribute a lot of the “bad” notification design theory to Windows (and Facebook), both of which present a torrent of largely irrelevant information throughout the day… information that you must disable if you DON’T want to read it. It’s ironic that Apple’s notifications are the ones most in need of improvement! (Thankfully, that’s coming soon.)
UI, UX, and You & I
APPCULTURE: At the moment, what’s your favorite UI in an app (desktop or mobile?)
I just love Twitter for iPad. Its gestural navigation really makes me think about button navigation — do we actually need all the buttons that most tablet apps have? Whenever I go between Twitter and FlickStackr (though not perfect, probably the best Flickr app for iPad), I always want to swipe the views back and forth the same way you can swipe panels in Twitter. Though standard, the iOS “Back” buttons feel slow in comparison!
APPCULTURE: In 10/GUI there’s no mention of a systemwide notifications system, or a “widget layer”. Was this a conscious choice, or was it omitted for time? If the latter, what form do you think those features would take in 10/GUI? If it was a conscious choice, what’s your reasoning?
Definitely omitted for time. Notification is one of those things I don’t really feel like anyone is quite getting right yet, with the possible exception of HP/Palm WebOS. This is actually kind of a big, cultural question, though probably more so in mobile: What is our tolerance for notification? When do we want to be distracted, and when don’t we? Outside the computer, there’s this huge continuum of what’s important to us, and depending on what we’re doing, some notifications are going to be over that threshold while a lot won’t be, and on either end of that threshold, there are varying degrees of how hard we want to have our attention diverted. Add in variables of time sensitivity and contextual relevance, and building a good notification system becomes a vast design problem!
APPCULTURE: With internal scrolling functions (such as inside a form on a webpage), the iPad uses a two-finger drag: it’s one of the smartest methods I’ve seen yet. How would you propose handling this ability in 10/GUI without invoking a window command? Or is there a better way to address this in the first place?
So, I read this question, thought for a second, and quickly rushed over to my iPad to try it out… I didn’t realize you could do that!
Obviously I think there’s a bit of a discoverability failure here (ahem!), especially given that the iPad doesn’t really use a finger-hierarchical system in most apps. I’d probably lean more toward the way that scroll-within-a-scroll works on the desktop: Scroll gestures always target the lowest level, but pull the next-highest level in the same direction once the lower level reaches its end. It’s the most physically realistic paradigm (always a good thing for discoverability) and I think would tend to be the least frustrating.
APPCULTURE: What, if anything, would you change about 10/GUI’s implementation given recent advances in software, hardware or services? For example, the iPad launched after 10/GUI’s video: does its featureset (or the capabilities of any new products) alter your thinking about parts of 10/GUI, or do you feel that those products serve different needs?
Direct multitouch has obviously shot way up in terms of mindshare since the introduction of the iPad, but I still don’t think it’s going to displace decoupled touch interaction on the desktop. The two working in tandem is definitely a possible way forward, though: the rise of companion apps, such as TouchOSC or some of Adobe’s concept apps points that way, but I think it will really come into its own once it can be taken for granted and apps designed specifically around it. The Acer Iconia, though awkward, is a fascinating attempt at this.
Touching the Future
APPCULTURE: Imagine it’s 2025 and every computer on Earth is running something like 10/GUI. What problems or limitations do you think people will have noticed after such extended use?
While I did kind of skewer the idea of 3D window managers in the 10/GUI video, I do think that eventually, 3D could very well play a big part in the way we use computers every day. If it does, it won’t be in the way we think of it right now: not silly Hollywood stuff with shapes flying all over, but less-sexy 3D used in a way that actually helps us understand information rather than obscuring it or making it more complex. And we might not need window managers anyway — the window as we know it could die out as UIs become more integrated and APIs blur the lines between app and task, further abstracting the way we use computers.
APPCULTURE: What do you think we’d be giving up in moving to something like 10/GUI? More to the point, what kinds of work, datasets or actions are largely incompatible with 10/GUI’s philosophy?
I don’t think there’s anything that by nature has to be incompatible with 10/GUI — the system should evolve and adapt as other kinds of work outside its initial scope run into any pain points. I think there’s a lot of room for adaptation with the right kind of creative thinking.
APPCULTURE: What do you think is the biggest challenge to hardware and software developers in producing and implementing something like 10/GUI?
To quote [CEO of Microsoft] Steve Ballmer: “Developers, developers, developers!” You could spend hundreds of millions of dollars building a new version of your desktop operating system and a 10/GUI hardware solution to work in tandem, but then you have to convince everyone who has software available for your OS to start over with a new, 10-finger-optimized, Con10uum-compatible version. Not to be snarky here, but… can you imagine how long it would take Adobe? That kind of uncertainty makes the investment into a completely new paradigm a very tough sell. Ultimately, the best way is probably baby steps, gradually weaning users and developers alike off old patterns and onto new ones.
APPCULTURE:Thanks so much for your excellent responses, Clayton. I’m so glad we could get in touch!
As always, I’d love to know what you’re thinking. Let me know in the comments!