The AlpineInker

InkSeine Version 1.2.1720.0 Fixes XP Problem

Ken Hinckley — Thu, 21 May 2009 16:42:00 GMT

Just a quick update that InkSeine version 1.2.1720.0, which fixes our bug with Windows XP, is now available. This build works on XP, Vista, and Windows 7.

See the previous blog post for details on the new features of this build. The only change in this build is the fix for XP. If you grabbed 1.1.1714.0 and you're not on XP there is no reason to reinstall this one.

Also, the install has not yet been pushed out by the OfficeLabs auto updater, so the only way to get it right now is to install it manually. The OfficeLabs guys are helping us out and we will have it available via auto-update as soon as possible.

For the curious, our gaffe with the XP build was a side-effect of a work-around we developed for the Visual Studio 2008 problem with the Windows XP version of Microsoft.Ink.dll that I wrote about way back in January 2008. Our build was forcing use of the Vista / Win7 version of the dll, which has been fixed to avoid the Visual Studio 2008 issue. But if you installed the 1.1.1714.0 build of InkSeine on XP, it goes to look for the Vista verison and it's not there, so poof, it crashes with a failed load of the dll.

Anyway, sorry again for the gaffe... Sometimes I'm amazed that any software ever works...

InkSeine Update 1.1.1714.0 Now Available

Ken Hinckley — Wed, 20 May 2009 13:13:00 GMT

We have an InkSeine update for you.

UPDATE: We have discovered that this build does not work on Windows XP. We are working on a fix (ready tomorrow?).
UPDATE #2: XP problem fixed! The build number on the update install will show up as 1.2.1720.0.

We'll be publishing it shortly via the OfficeLabs auto-updater, so if you have auto-updates enabled, leave InkSeine running for a couple of minutes, and when you exit InkSeine it should ask you if you want to install the update. (Update: the Office Labs auto-update has not yet been pushed out.)

If you do not see the auto-update for any reason, you can install the new version directly from our downloads site : InkSeine version 1.2.1720.0 1.1.1714.0 (but if you do it this way, you must first uninstall your old build of InkSeine).

This build addresses a number of issues that have been on our plate for a while now:

Windows 7 Support. InkSeine's search features now work on Windows 7.
Inking Performance.
We fixed some issues with our ink stroke capture and rendering that produces a much snappier feel for drawing in InkSeine. This is a significant improvement for most tablets; if you have a UMPC like the Samsung Q1 series or an OQO Model 02, the difference is dramatic.
New Page Viewing Options and Scrolling within Pages (see details below)
Hide / Show the Arc
The arc with all your pens and other tools on it now has a small +/- symbol on it. Just tap it to hide or show the tool arc. I find it's great to hide the arc when I'm really concentrating on a full page of notes, or projecting an informal presentation that I've sketched up in InkSeine.
Tool Ring in task tray. The InkSeine Tool Ring now resides in the system task tray. For the many people who run the "ScrollControl.exe" application separately from InkSeine, this will make it easier to manage.
Print margins bug fixed. Printing from InkSeine now always scales the page correctly so that the entire page will be visible on the printout; previously InkSeine would sometimes clip off the margins.
File loading bug fixed. We fixed an obscure bug that on rare occasions would cause opening of certain InkSeine files to fail. Thanks to everyone who helped us track this down!

New Page Viewing Options

Previous versions of InkSeine were limited to always show one whole page of your notes at a time. From the feedback we received, it became abundantly clear to us that this could sometimes be too limiting, particularly when switching between portrait and landscape screen orientations on your tablet.

InkSeine now supports a new Fit Page to Screen viewing option that scales the page to fill the current screen dimensions. For example, you can give this a try by creating a page in landscape format (step 1 below) and then switching your tablet to portrait orientation (step 2). InkSeine now applies the Fit Page to Screen mode by default whenever you switch screen orientations.

You can still view the entire page at once by choosing the Whole Page viewing option instead (step 3). The Fit Page to Screen / Whole Screen viewing option is a toggle found in the InkSeine Options menu, as shown below.

Scrolling within Pages - without a Scroll Bar

The new Fit Page to Screen view implies the need for scrolling. But InkSeine has no scroll bar. That would just be wrong. A concept car for pen computing can't have scroll bars on it.

But now, InkSeine does allow you to scroll within pages. Here's another example where I'm using my tablet in landscape mode to view a page I originally created in portrait mode. I can now circle my pen starting on the InkSeine tool ring to scroll the page up and down.

Note that InkSeine still has an underlying model of discrete pages, so if you continue to scroll, it will not scroll you through all the pages in your note.

You might even notice that InkSeine now adds a subtle effect so you can tell when you've scrolled to the edge of a page. Do you see what it is? What do you think?

Still a Work in Progress...

I had really wanted to get custom pen and highlighter colors (and thicknesses) into this build, but we didn't make it. We believe there are a lot of subtle issues with supporting that well, and we didn't want to include a half-hearted attempt just to check off a new feature. But this is a new capability that I'm desperate to add myself.

There's also a couple basic usability requests that we didn't get to, such as having both Personal Search and Web Search available as top-level options in the context menu which appears after lasso selecting an ink phrase. I have a particular solution in mind for that which we didn't have time to complete.

I currently have three kids at two years old or less, and Raman (our developer extraordinaire) is currently managing our group's entire dev team, so we've both been running on limited cycles this year. But we are still pounding away and looking to introduce more new features and innovations for future InkSeine releases. Definitely be sure to let us know of any new requests, bugs, or reflections that you have on your use of the new features in this update.

Rebecca Renee has arrived!

Ken Hinckley — Fri, 05 Dec 2008 07:29:00 GMT

Things have been quiet here lately, but with good reason. I've been a busy daddy, and I'm happy to announce that Rebecca Renee Hinckley has arrived a bit earlier than expected, but at a robust 9lb 2oz. Consumer consumption is not dropping in this household at least!

She joins Sarah and Alissa (not quite 2yo yet!) and beaming momma Angela. The good lady on the left is Dr Debra Stemmerman who is outstanding and brought a difficult 36hr labor to the happy arrival of Rebecca.

The Microsoft Research Codex: Are Dual Screens the Future of Mobile Devices?

Ken Hinckley — Wed, 01 Oct 2008 10:17:00 GMT

Never buy one of anything. That's advice you should stand by when you're buying unusual gadgets. The advice was good when Randy Pausch offered it to me some 15 years ago, and it's still good now.

Of course, with 18 month old twin girls at home, this has become second nature to me. Two boxes of diapers. Two gallons of milk. Two Elmo plush dolls.

Oh, and yes, of course. Two screens for my tablet computer.

Dual-screen devices have become the subject of increasingly elaborate internet fantasies. Now, rumors about an Apple "Brick" device have stirred up dreams of future tablets, such as the alluring One Laptop Per Child v2.0 concept photos, which now orbit the internet once again.

My interest in dual-screen devices goes back a lot further, though, and was really spurred on by the University of Maryland dual-screen e-book reader project. That effort is led by Francois Guimbretiere, who is a long time collaborator and friend. I had some ideas to build on what his team had done, but also to take things in a different direction.

First of all, I wasn't interested in an ebook reader. I wanted a device that was all about writing. Sure, reading and writing go hand in hand - you encounter cool ideas and search out reference material on the web-but what I wanted to build was a tool for thought. To me that means a tool with writing, sketching, and annotating as the core of the experience.

I'd been thinking for a long time about picking up an OQO Model 02 computer. My team has an extensive code base for pen-and-tablet functionality resulting from InkSeine, and the OQO runs it out of the box. It's got an active digitizer for high fidelity pen input, and it's the smallest slate Tablet PC that money can buy. That's a pretty good start on a small form-factor tool for thought.

So when it came down to it, just how many of those OQO Model 02's do you think I purchased?

Project "Codex" was born.

Twice the Screen at Half the Size

Here's the Codex packed up for mobility. It folds up quite nicely and has a moleskine-style knitted elastic strap to hold it securely shut. There's a loop for the pen and a mesh pocket so you won't lose small accessories, business cards, or receipts that you collect in your travels.

Sure, it's a bit of a brick at present. The whole thing weighs just over 2 pounds. The OQO's are considerably thicker than I'd like. But my goal is to prototype the future as quickly as possible and start living it. The OQO offers a handsome, jet black time-travel machine in that regard.

Don't underestimate the ability to quickly pack up with the screens protected. Folding the Codex in half makes it comfortable to carry and easier to stuff into my gadget bag. It's a self contained kit for ultra-mobility.

The Codex is Not a Container for Dead Trees

A codex is just an archaic term for a bound book. But this Codex is unlike any book that you've ever read. It's not a long linear text that you flip through. To me, there's no use in going to all the trouble to build a dual-screen tablet prototype and write elaborate software just to mimic a traditional book. This is the 21^st century and it's about time we moved past containers for dead trees.

We have lots of information sources. We have multi-tasking. We have hyperlinks. We have split attention. We have a left brain and a right brain and we rarely do one thing at a time any more.

A Separation of Concerns

The Codex has two screens, it's designed to be used that way, and you won't find any half-apologetic demos that try to mash them back together into one big screen. Instead, it's all about the intelligent partitioning of tasks and interface elements across the screens.

Above is one example where I'm working on a blog post that I've had planned for a while. On the left I have a whole bunch of cool photos that I found tagged with moleskine on Flickr. I was browsing through these as inspiration for our InkSeine digital note-taking software. On the right I'm organizing bits and pieces from these photos along various themes. So I just take a snapshot from the collection on the left screen and it appears in my notes on the right screen, where I can arrange it and mark it up as I see fit. I can scroll back and forth on the left screen to find a photo that meets my current needs, while the page that I am authoring on the right screen always remains visible. The two screens are invaluable because I always have the reference material in the context of what I am working on, instead of feverishly flipping between them on a single screen.

Can I do this on a single large screen? Well, sure, I could monkey with the window placements and get everything arranged just so. But that takes a lot of effort and the temptation to expand windows to take over the full screen is hard to resist if I have to expend effort to do so. A dual-screen device that understands the partition between the screens gives a much simpler experience where I don't have to constantly manage the set-up of the windows.

Navigate without Losing the Big Picture

Here's another example. I'm further along with authoring my blog post now, and I have a bunch of material floating around in my notes. I create a page that is a Table of Contents, with links to several themes that I've identified in my Flickr moleskine investigations. If I open a link, such as my Creative Collage page, it opens on the opposite page. I don't even need a "back" command to return to where I was - I still have my navigational structure on the left, side-by-side with my content page on the right.

Unlike a traditional dead-tree book, I have no physical restriction that forces me to view consecutive pages - The Codex lets me follow links or flip through the screens separately to view any two pages together.

Working Big and Small

We can draw some inspiration from traditional media as well. In magazines and books, sidebars are a distinct section of a page that augments the main text with auxiliary information. Well, the Codex has sidebars on steroids. I can take any chunk of my notes, make it into a sidebar, and then arrange a bunch of these on a page. Here, I've made a storyboard page consisting of six sidebars. When I tap on a sidebar in one of the storyboard cells, it expands to full size on the opposite screen. I can plot out the broad sweep of my story on one screen, while maintaining full access to the zoomed-in details on the other.

Your Mother Was Right - Posture is Important!

Perhaps the coolest property of a dual-screen device is that the compact, mobile form-factor encourages shifting the device around. I can orient the screens with respect to one another, stand the device up, look at the screens in portrait or landscape, and so forth. In fact, the Codex supports about a dozen different configurations. We call these postures. (A colleague, Michael Miller, coined this term).

Add two cups of accelerometers, a dash of flex sensors, and bake with some simple software to fuse it all together. Out of the oven pops an intelligent dual-screen display system that configures itself depending on how you arrange it.

For example, if I want to work in landscape, I can flip the device into the laptop posture, as opposed to the book posture that I've been showing so far. Now I have one screen that's angled for easy reading, plus another screen that's horizontal for easy writing.

Or I can use the laptop posture to hook up one of my screens to a projector. I put the public part of my presentation on the top screen, while the controls to drive the presentation and my private notes are confined to the bottom screen.

Now perhaps I meet a friend in a café and I want to show him what I'm up to. I can just lift up on the binding of the Codex to angle the screens so that one is facing me and one is facing my friend (below, left). We call this the battleship posture - as in "You sank my Battleship!" Each person has one screen with a private view. The Codex automatically configures the software for shared whiteboarding so the collaborators can mark up the screens, pass notes back and forth, and other such foolishness.

Next I drop the binding back flat. Now both of us can view the screens, but one screen is oriented towards each person.

Don't Get Too Attached

Something that turned out to be surprisingly useful is that the Codex allows me to pull each screen right out of the binding. A firm pull pops it out, a firm press pops it back in. This is really handy for laying out the screens to suit my work, or to review a video while I jot down some notes. I can even hand the screen to another person to show them something - without actually giving them an electronic copy of the information.

What? You Still Have a Desktop Computer?!?

The Codex is just another wireless device so of course I can connect it to my desktop computer. Now, any screen capture that I take from my desktop screen shows up in my Codex notes right where I left off. For web pages, the snapshot comes across with a hyperlink back to the source, so I can easily revisit it later when I'm reviewing my notes on my Codex. This makes it the perfect companion no matter how I'm working.

A Parting Shot at Text Entry

The Codex is designed around doing most things with a pen, but I'm no idiot. When it comes to text entry, a keyboard is a good thing.

And that, in part, is why I'm puzzled that so many of the dual-display concept designs that I've seen take up a whole screen with a virtual keyboard. I'm sure people in focus groups ask for this. But I still think it's a really bad idea. What is the point of having two screens if you are instantly going to cover one of them with a picture of a keyboard? Your hands immediately occlude it from sight anyway. At that point I'm basically using a single screen again, so I might as well just grab my laptop to get a decent keyboard.

The Codex offers a solution for text entry that I much prefer, by virtue of the OQO Model 02's mechanical keyboard. I just slide the screen up and that reveals the keyboard when I need it. Works great. No dorky touchscreen keyboard. I still enjoy the full benefits of partitioning my work between the two screens. That's the way I like it.

When Can I Buy One?

The Codex is a prototype-- and a rather flaky, cobbled-together one at that. But it uses off-the-shelf devices, and there's nothing magical about the software. So the crass answer is that you can have one now if you are willing to spend some dollars, build yourself a custom binder, and write a little bit of code. That's how I started. My first prototype was a repurposed day-planner with Velcro holding the screens in there. Install a shared clipboard utility and you can start copying and pasting between screens. That will give you just enough of the experience that you will hunger for more.

What would you want out of a dual-screen device? What capabilities would make it most useful to you? How do you see dividing your own work between two screens?

In the meantime, the trajectory of ultra low-power "e-ink" displays bears watching. Check out the recent Plastic Logic device, for example. Right now e-ink is an abomination for anything interactive, but eventually some display technology will get where we want it to go. Low-cost, low-power screens are crucial to make dual display devices a practical consumer device, rather than a research lab curiosity.

The sheer number of concept devices that have popped up in the last 6 months suggests that dual-screen devices are poised to take off in the near future. My hope is that our research on the Codex can help in some small way to unearth the full promise of such devices. I, for one, am convinced that dual-display devices have a well-motivated role to play in a future ecosystem of mobile devices.

In my office, at least, that future is already here. It's just not evenly distributed.

Acknowledgements

I would like to thank the following key contributors to this project:

Morgan Dixon
Raman Sarin
Francois Guimbretiere
Ravin Balakrishnan

A Tribute to Randy

Ken Hinckley — Tue, 26 Aug 2008 10:10:00 GMT

The final grains of sand slipped through the hourglass, as we knew they must. Still the finality of it hit me hard. July 25, 2008. Randy Pausch has died. My friend and mentor are lost.

I worked with Randy, my Ph.D. advisor, at the University of Virginia between 1991 and 1997. He left for Carnegie Mellon shortly after I graduated. Note I did not say that I worked for Randy: I worked with Randy. He always insisted that I say it that way. He was my colleague, not my boss. To this day I am always careful to speak the same way of my colleagues and the team of people that I manage.

As fate would have it, I had a demo scheduled during an executive keynote at the Microsoft Research 2008 Faculty summit on July 29, the Tuesday right after Randy died. Hundreds of faculty from around the world attend the summit every year. Many people who had known Randy would be in the audience, including the legendary Andries van Dam, Randy's undergraduate advisor. I couldn't let the opportunity pass without reflecting on this profound loss. I didn't practice my tribute and I didn't tell anyone I would do it. I just did it.

Randy talks about brick walls and overcoming them in The Last Lecture. It only occured to me just now, but my speech was in the Microsoft Executive Briefing Center, just down the hall from the pièce de résistance of the Microsoft art collection: a graffiti-covered chunk of The Berlin Wall - perhaps the biggest brick wall ever thrown up, in mankind's foolishness, to be torn down by those with greater ambitions.

My slot was only for 10 minutes. I had very little time to say anything substantive, and I still had to do my demo. I thought about it a lot. In the end what I most wanted to say again to Randy was "Thank You!" - thank you for being a great mentor. So I did exactly that, the image below projected on a massive screen twenty-five feet tall. It was literally the biggest Thank You that I could offer to Randy.

I also realized that I couldn't just leave it at that and plow right into my talk and technology demo. I first had to create a sense of closure, where of course the wound was still fresh and there was none. So this is what I said.

I think of Randy's life as an unfinished book. What is there is amazing and touched millions. I know the succeeding chapters would have been brilliant and fantastic. But the next page must remain forever blank.

Another Girl?

Ken Hinckley — Fri, 11 Jul 2008 00:51:00 GMT

It’s been a busy summer – so busy, in fact, that the blog has been starved for attention of late. I have some cool posts in the works but it will be a little while longer before I can get them finished.

The exciting news on the home front is that my wife Angela and I are expecting a new addition to the family in December! The ultrasound suggests it is a girl, but it’s a little too early to say for sure. That would give us three little girls under two years old.

My twins, Sarah and Alissa, are now 18 months old and experiencing an early vocabulary explosion. Words that excite them of late include airplane – after their first plane trip to see their great-grandma and other family- as well as “bull-bo” (bulldozer) after our neighbors had their driveway torn out and replaced.

Some other new words they have picked up recently include truck, hike, mount (mountain), kitty, Cleo (our cat), rabbit, swing, outside, fork, cracker, umbrella, elephant, ear, tongue, and pants. I would guess their vocabulary is probably in the vicinity of 200 words and animal sounds now. Maybe a little more.

They can even tell us when they poop now, although they don’t always like to admit it!

My brother and his girlfriend are visiting this week from Boston, so the latest additions are “Uncle” and “Lori.” Pretty soon he will have them saying “Red Sox” as well, I suspect!

InkSeine blog post featured on OfficeLabs.com

Ken Hinckley — Tue, 17 Jun 2008 19:30:00 GMT

InkSeine is one of five project downloads currently available on the OfficeLabs.com site. I've been having great fun participating in the OfficeLabs.com launch. It's been great to get to know some of the people in the OfficeLabs organization. I've been impressed with how they are running their ship. I suppose it is still too early to declare OfficeLabs a complete success, but I think good things will come of it.

Today OfficeLabs.com is featuring a blog post that I put together about InkSeine: Try InkSeine – Drive the ultimate concept car for pen computing. I give a quick overview of the project and summarize the feedback we have received so far. I close out the post with some fun illustrations of ways to use InkSeine - some from my own notes, some from notes contributed by creative inkers in the InkSeine community!

Check out the blog post, and check out the other downloads available from OfficeLabs while you are there: Community Clips, Search Commands, SharePointPedia, and Task Market. There's lots of great stuff to try out. Your comments and usage of these prototype applications have a real opportunity to influence the future of potential Microsoft product offerings.

JasonJ's InkSanitorium

Ken Hinckley — Wed, 04 Jun 2008 10:07:00 GMT

GottaBeMobile.com forum member JasonJ is a prolific inker. He’s been at the avant-garde of InkSeine for some time now. He’s offered us lots of great feedback and has a flair for illustrating his points. For example, he’d like us to add a sizing tab to make it easier to resize the InkSeine application window. He often uses it like this to make it easier to drag files and links into his notes:

Well, you just can’t make the point any better than this. After seeing a posting like that, how could we not do it? We’ll have to change some things to get this to work, but this kind of feedback gets the feature on the task queue for sure J

He’s also argued against using pressure or additional tablet buttons for pen functionality. As researchers, those are the kinds of additional input channels that we sometimes ponder as routes for tablet innovations, but as JasonJ argues so well, a general tablet and stylus interface can’t require those as building blocks:

I still think zany ideas in this vein are worth exploring as options or alternatives. They can make for good research papers, even if they are not suitable for deployment in InkSeine. I also agree with JasonJ that they need to be approached with caution as they can potentially detract from the pure pen and ink experience.

Another thing JasonJ desperately wants is custom pen and highlighter colors. This is something we’ve been planning to add to InkSeine since well before our initial release, but we haven’t had the opportunity to implement it yet. JasonJ offers another great illustration for how this might work:

This kind of sketch is very interesting for us because it reveals JasonJ’s vocabulary and structure thinking about the task workflow: select the nib type, then select the color. Maybe these could be done as interchangable steps. Even if we don’t follow the exact UI design he’s sketched out, that kind of feedback is really helpful when we are making decisions about how the UI should really work. We do conduct usability tests occasionally to vet our designs and test for problems that we’ve overlooked, but in my experience such tests usually aren’t very helpful to come up with a good design in the first place. But sketches like this from a person who is really using the software to do stuff out there in the really world certainly do!

Jason also has some fun with InkSeine. He experimented with the custom page backgrounds download that we posted. He thought it would be cool to take it one step further and show the pages flipping. Now wouldn’t that be cool?!?

One JasonJ sketch even made the front page of GottaBeMobile. He likes to put hyperlinks to folders and applications in his notebooks so that he can quickly launch them while sketching out his thoughts and taking notes on his ideas. I do this all the time with InkSeine myself – it’s great for things that you use frequently in the context of a project or topic in your notes that you revisit from time to time – but I have to say that JasonJ’s version just looks cooler and more fun than my own versions of these:

Putting them in the canted gold picture frame lends them a wonderful touch of class and personality. It’s certainly more fun to work this way with a tablet than to pull down some soulless drop-down menu with a monotonous list of textual favorites.

JasonJ also kindly sent me a selection of some of the cool note pages that he’s generated in the course of his daily work. This stuff is like solid gold to us – it really shows us what someone is doing with our tool on a daily basis. Even when people write to us that they like InkSeine or that they are using certain features to do fun stuff, we rarely get to see what really happens in those secret journals. This gets us excited all over again about great software for inking on a Tablet PC. It also gets us thinking about more stuff we could do to make this kind of usage more fluid and more expressive by adding new capabilities or by simplifying the program.

JasonJ’s notes are just beautiful and a lot of fun, so I’ll let this selection of pages from the highlights file he sent me speak for themselves:

The final page of this particular notebook is a sketch that JasonJ did that shows how a feature for summing lists of numbers might work in InkSeine. I’m not sure if this is a feature that we will have the cycles to implement, but I love the design he sketches for how it could work. In fact, I often do exactly this sort of ink-plus-screen-capture mashup to sketch out my own ideas for InkSeine (and other projects). It’s a great way to lay it out there and see if the idea really could work, or if it has problems that weren’t obvious at first.

Any way you add it up, JasonJ’s InkSanitorium shows how InkSeine can a fun, productive, and eye-grabbing way to hash out your ideas.

Rough Winter

Ken Hinckley — Tue, 13 May 2008 02:35:00 GMT

I spent the weekend at Snoqualmie pass. Winter has certainly not yet relinquished its grip on the Cascade Crest. Here's some of the sights done AlpineInker style.

Here's a car that someone left parked for the winter. That probably wasn't a good idea. It's been crushed like an empty can of cheap beer. When this much snow piles up, it pancakes down - hard as concrete and twice as heavy.

Despite what these pictures suggest, quite a bit of the snow has melted. I could not see any daylight at all out my shattered kitchen window the last time I was there. The bottom of that window is about 15 feet above grade level. We lost a window on the north side of the house too. First time that ever happened...

It's been a rough winter in the Alpental valley.

Winter 2007-2008
Date of First Measurable snowfall 10/19/07
October snowfall           2
November snowfall      24
December snowfall    180
January snowfall        149
February snowfall        90
March snowfall          101
April snowfall              61
May snowfall               1
              Total         608

That's 50 FEET of snow kiddos!

Even the highway web-cams have suffered! "I fought an avalanche and the avalanche won. This camera was so badly damaged we can't repair it until this summer."

Mother's day was no exception. More snow.

Here's my wife taking the twins for a walk. Today they learned a new word: Snow!

Rule #1: No parking. Rule #2: NO PARKING!

This is, or rather was, my kitchen window...

...But there's no place I'd rather be.

Some Thoughts on Automatic Screen Rotation

Ken Hinckley — Thu, 08 May 2008 08:41:00 GMT

Automatic screen rotation has been popularized by the iPhone but is also available on the OQO Model 02 thanks to Kenrick's Automatic Screen Rotator Utility (executable download). Just hold the device the way you want to use it. The screen flips to the correct portrait or landscape orientation in one second. You don't even have to think about it. What could be simpler? Kudos to Kenrick for putting this great utility together and making it available for free!

This is a program near and dear to my heart. I cobbled together custom sensor hardware, including an accelerometer to support automatic screen rotation, for my old Cassiopeia E105 Pocket PC back in the late 1990's:

Here's our video of the Sensing Pocket PC, with screen rotation and other fun stuff too.

Video: Sensing Techniques for Mobile Interaction

Later, I built sensors for the original slate Tablet PC prototypes that were floating around Microsoft. Many devices now include accelerometers for drop detection, but I'm pretty sure my prototype was the world's first Tablet PC with an accelerometer. It came with an extensive user manual: TILT ME.

That wad of electronics on the top is my sensor module. Here, I'm using the Tilt-a-Sketch application. You could draw on the tablet like an Etch-a-sketch by tilting it back and forth. Yes, it was really hard to sign your name this way, and yes, if you flipped it upside down and shook it, it erased the screen. Accelerometers can be a lot of fun.

But what was the most useful? Like Kenrick's utility, it supported automatic portrait/landscape switching depending on how you held the device. After all that hard work I had to put into building my own sensors, firmware, and software, it's mind-blowing to see this available in a free utility that I can download from the 'net for an off-the-shelf device!

My demo had a few tweaks, some never published before, that might be useful future embellishments to Kenrick's Automatic Screen Rotation utility. In essence these tweaks reduce accidental changes to the display orientation when you're working with your device. They also help to avoid rotation of the screen when you go to set your device down on your desk. Plus there's one bonus idea I tinkered with, described at the end - let me know if you like it or not.

Dead Bands for Increased Stability

Dead bands between the screen orientations made the device tend to stick to the current display orientation. This helped to avoid accidental changes to orientation.

Plot of tilt angles versus inferred instantaneous screen orientation.

To change display orientation, the tilt angles had to pass all the way through the gray ±5° dead bands, and stay within the same display region for 0.5 seconds. No screen rotation occured in the central "Flat" area.

Rotation Preview

Feedback for impending display rotations makes automatic changes to the display orientation more predictable and controllable. My Tablet PC demo displayed a "THIS SIDE UP" arrow at the center of the screen as soon as the tablet was tilted in a different direction. The change to the display format occurred one second after the arrow appeared, but only if the device was still tilted towards the new display orientation. This allowed the user to stop tilting the device to prevent an inadvertent switch.

To foreshadow a change to the display orientation, an arrow appeared immediately when the user rotated the tablet.

Of course, the arrow should not be there all the time. The arrow vanished when:

The screen changed orientation. The arrow remained visible for a couple of seconds after the switch to provide continuing feedback.
The instantaneous screen orientation returned to the current display orientation for a couple of seconds. This case occured if a user acted on the feedback to avoid an accidental change.
The user set the Tablet down flat without changing screen orientation.

I used black color-key transparency (in a layered window) for the actual bitmap used in the code.

Motion Detection

Movement of the device serves as a secondary indicator of when to switch the display format. To avoid accidental changes to the screen orientation, my Tablet PC implementation waited for motion to stop before rotating the screen. For example, this made the device less likely to change screen orientations as you set it flat on a desk.

The OQO Model 02 supports only about a 4 Hz sampling frequency on the accelerometer, so it might not be feasible to implement good motion detection at present. Nonetheless it seems worthwhile to mention it, in the hope that an increased sampling rate becomes possible in the future.

One hack to detect motion is to calculate how much the tilt values are changing, as follows:

Δx = tiltX - prevTiltX

Δy = tiltY - prevTiltY

sampleEnergy = √(Δx² + Δy²)

signalEnergy = signalEnergy*(1-α) + sampleEnergy

In the final equation, α is a decay rate. I used 0.25, with the tilt values in degrees, and signalEnergy initialized to 1.0. Motion "begins" when the signal energy rises above an onset threshold for a few samples and "stops" when the signal energy drops below a termination threshold.

Movement helped to control switching of the display format as follows:

When movement stopped, if the physical screen orientation did not match the inferred instantaneous screen orientation, a 1 second time-out began, after which the software switched the physical display orientation.
If movement began again during this time-out, the time-out for the physical display switch was cancelled.
If the instantaneous screen orientation changed again during this time-out, the time-out was restarted at its full one-second duration.

Orientation-specific Tasks

Here's one other nutty idea I experiemented with. Maybe it's useful, maybe it's not.

I assigned specific applications to specific orientations of the screen. For example, here's a screen shot where I set up Excel to appear in the landscape format, and Windows Journal in the portrait format. Flipping my Tablet PC between the two would switch between the applications, rather than just rotating the screen.

Switching orienations can switch between sets of applications as well...

This offered a simple way to partition applications into task-specific sets for each screen orientation. Unfortunately my prototype of this feature never really worked all that well. You could check off windows as belonging to each screen orientation. The prototype would hide and show the windows as you rotated your tablet. But it had some bugs. Sometimes it would hide the windows permanently, never to be seen again. That's not terribly useful. So I never did usability testing on it, but I found something about it intuitively appealing. What do you think? Would you want this feature on your tablet or mobile devices?

Summary

My experience is that the devil is in the details with sensing techniques. Small touches here and there go a long way to keep the interaction invisible in the background, rather than becoming a focus of attention when things happen that the user didn't intend. Ultimately, the goal should be to create the best possible user interface. What is the best possible interface, you might ask?

The best possible user interface is the one that you don't even notice is there at all.

Resources:

Kenrick's blog: Teaching, Technology, and Learning
Kenricks' Automatic Screen Rotator Utility, download for the OQO Model 02
Ken Hinckley's UIST 2000 conference paper and follow-up journal article on sensing techniques.
Video of the Sensing Pocket PC

My New Favorite Flick Pad

Ken Hinckley — Tue, 29 Apr 2008 19:30:00 GMT

I just received an OQO-style moleskine in the mail!

When I opened it up, it was full of arcane tricks for using InkSeine's Tool Ring as a flick pad. This topic has come up before but now it seems that perfect combination of custom flicks has been discovered.

This is the new set of Flick gestures that I've adopted for the OQO. Show Desktop is especially handy!

Right flick mapped to Alt-Tab - great for flipping between two windows. This one you need to manually configure by choosing the (add) option from the drop down.
Down flick mapped to "Toggle Ctrl" which is one of the standard choices in the drop-down. I use Ctrl with the ToolRing's scroller to zoom.
Diagonal-upper-left flick mapped to Right Click. Also a custom combo, just hit the right-click soft key on your TIP to add this one. (It shows up as "Application" when you press it).
Up flick mapped to Show Desktop - my absolute favorite! A quick flick up on the Tool Ring always gets me to my desktop, which is a handy place to stash all your shortcuts on a tablet. This is also a custom combo consisting of "Windows + D", where Windows is the Windows logo key.

Here are my notes... flick up to Show Desktop:

Voila! There it is! But even better... doing Show Desktop again... puts me right back where I was!

I'm also debating whether I want to replace Undo with Enter instead. It's handy for opening selected files without double-tapping. I guess I'll have to see. But I'm keeping Copy, Paste, and Back for sure - unless I find something better the next time I flip back to this idea!

Here's the return address label on the back cover. I have no idea who this is. And it's been a long time since two cents cut it for postage. It's all a bit mysterious. But whoever you are, keep the tips coming!

Posts in the Tool Ring Shenanigans series:

InkSeine Update, InkSeine Featured on OfficeLabs.com!

Ken Hinckley — Mon, 28 Apr 2008 12:05:00 GMT

InkSeine is one of the projects featured on the new Microsoft Office Labs web site. There are some cool prototypes available there, so I recommend you swing by to check them out, and to learn more about Office Labs. We're honored that Office Labs invited the InkSeine Team to participate in this launch.

If you are new to InkSeine, welcome to the fold! Check out the Twelve Days of InkSeine to see some of the ways that you can use InkSeine to take notes, illustrate ideas, and gather information on your tablet.

This InkSeine update (version 1.1.425.0) is primarily a maintenance release to address a few easy-to-fix bugs. However, we also have our new rotation feature working, so we decided to include that as well. We'll look to tackle many more of the requests and ideas that we've received in future releases.

Our AutoUpdate feature (thanks to Office Labs!) is also now ready to go. If you run InkSeine on your Tablet PC while connected to the internet, you won't even have to grab the download for this update. You'll see an invitation to upgrade to the new version the next time you exit InkSeine (see details below). UPDATE: I may have spoken too soon; it seems that our AutoUpdate will only get applied to subsequent releases, after this one. If you don't see the AutoUpdate invitation, just uninstall InkSeine, head over to the InkSeine download link, and install it the old-fashioned way. We'll try some more stuff tomorrow to see if maybe we can get AutoUpdate working.

InkSeine Version 1.1.425.0 Release Notes

Rotation and Reflection: InkSeine now supports rotation of any lasso selection. Just grab the little green rotation handle and spin away. You can also reflect the selection in any direction by grabbing a resize handle and dragging it through the opposite side of the selection.

Antialiased Page Thumbnails: It's now much easier to recognize pages from their thumbnails. Note: If you load an InkSeine note from a previous version, the page thumbnails only update when you make a change to a page. For example, draw an ink stroke and then erase it to force the page thumbnail to refresh, and you will see the improved version. Here's a comparison showing the improvement, with the old version on the left and the new version on the right:
Search for & Open OneNote sections. Previous builds of InkSeine only handled OneNote pages that were saved into individual .one files. InkSeine now returns OneNote sections with its search results, and you can open them and insert hyperlinks to them in your InkSeine notes.
File association fixed: The association for InkSeine files (.iks extension) now installs correctly. InkSeine files have a little notebook icon, and when you Open them from file folders or shortcuts on your desktop, they now will launch InkSeine.
Saves the last Pen and Highlighter: InkSeine remembers which pen and highlighter you were using so they are ready to go when you next launch InkSeine, or open another note.
Performance improvements, particularly while dragging selections.
Improved Stroke Eraser: It no longer leaves "debris" on the screen on occasion.
Tool Ring bug fix: The Tool Ring will no longer activate the camera or the close icon if you happen to end your pen stroke over them while circling-to-scroll or while using the tool ring as a flickpad on Vista.
AutoUpdate server is online! With the launch of the Microsoft Office Labs site, the Office Labs AutoUpdate server is also now online. We're very grateful to Office Labs for helping us to offer this service for InkSeine.

To get updates, your computer must be on the internet. Start InkSeine and make sure that it has been running for a few minutes. When you exit, you will be prompted to install the update (build 1.1.425.0). ~~Note: Make sure that "Automatically check for updates" is checked in the upper-right corner of the InkSeine options dialog. You can~~ ~~open the options from the check-mark menu. You may disable checks for automatic updates by unchecking this option, or by opting out during your initial installation of InkSeine.~~
~~Alternatively, you may install the new release of InkSeine manually. Uninstall InkSeine, and then~~ ~~download and install the new build~~.
UPDATE: Raman says that AutoUpdate may not yet fire for this release because of the way our installer was configured on our last external release. But it should allow us to auto-deploy subsequent releases. If you don't see the invitation to upgrade, uninstall InkSeine, grab the InkSeine download, and install it the old-fashioned way.

Thanks and be sure to let us know if you find any bugs, or if you have any ideas for improvements and new features. You can also discuss InkSeine and ask questions in the GottaBeMobile forum for InkSeine, or visit their general forums if you have questions about Tablet PC hardware, software, or just want to see some great tips about using your Tablet PC.

-- The InkSeine Team

A Review done InkSeine-style

Ken Hinckley — Fri, 25 Apr 2008 04:01:00 GMT

An InkSeine user, Anthony Chan, posted up a thoughtful review of InkSeine on his blog, written in InkSeine itself! With his permission, I'm reproducing it here. He has a lot of great comments and ideas for features. Let's discuss the points he raises.

Page 1: The table of contents

Anthony starts with a table of contents. It looks nice, doesn't it? Later he mentions that he wishes there were a way to make the entries active hyperlinks. That would be cool.

Page 2: Things That I Like about InkSeine

I'm glad to see InkSeine's search features rise to the top of Anthony's list. We expended a lot of effort on them. The the other features that people often mention include the tool ring, the radial menus, and the clean user interface with nothing but the page and the drawing tools.

Perhaps Bring to Front and Send to Back are trivial additions, but I find them indispensible when I use InkSeine to sketch out designs, draw mock-ups of user interfaces, or create presentations. In those situations I'm typically marking up a lot of screen clippings. It's essential to have some control over the layering. Other presentation / image manipulation features that I'd love to add to the program include non-rectangular clippings, cropping, translucent bitmaps, and possibly brightness/contrast controls. Rotation is coming in our next release.

By the way, in the screen capture above, you can also see an example of the high-fidelity page thumbnails that will be coming in our next release. Our current thumbnails don't look that great.

Page 3: Room for Improvement

InkSeine is a work in progress and we're always looking for ways to improve it as much as possible. It's really helpful when people let us know about areas where it doesn't meet their expectations.

The hover menus do occasionally fail to appear when expected, particularly for icons embedded in the note page. There is probably a bug around this. Also, menus won't pop up if you hold your pen perfectly still; this is an artifact of some special handling that we do for UMPC devices with passive touchscreens, so that menus or other hover information won't activate if the cursor gets left over an icon. We'll have to investigate this further.
A few people have asked for Sections, Tabs, and Folders. We're investigating a bookmark feature where you could create tabs to mark pages within a note. However, full hierarchical organization has a lot of attendant technical complexity so it will be a long while before we could take a crack at that. OneNote handles this kind of organization really well.
I'd dearly love to have custom page backgrounds. It is possible to create custom page backgrounds in InkSeine (samples available to try out). Note that Anthony requests a new background for each page, rather than just having a single custom page that is used for every page of a note. This was my experience too - for example, my OQO sketchbook has a cover page, an interior page style, and a back cover. One custom page template for all pages does not cut it.
The InkSeine installer has a bug which causes the .iks file association for InkSeine files to fail. This will be fixed in our forthcoming release.

Page 4: Ideas for Future Versions

I always love to see people's ideas for future extensions. Even if they're things we've thought of, it helps us to prioritize which things are most interesting. The way that people talk about using new features also suggests how the resulting user interface should be presented.

Hyperlinks within a note and embedding HTML code (for videos and such) both make a lot of sense. I also like how Anthony draws the embedded video with an ink-stroke frame. That would be a nice touch to make it feel like a sketchbook, rather than a blah web browser.

Page 5: Ideas for Future Versions, Continued...

Anthony suggests the tool ring should scoot away if you're writing with the pen and you get too close to it. That's a neat idea, and in fact, that was the very first thing we tried. But it was very annoying to have it keep moving around. Several people have asked for an auto-hide option, where it would shrink down to an icon after a period of disuse, or if you tapped on a little arrow to shrink it. I think that would work.

Anthony also mentions publishing InkSeine pages on a blog. Several people have asked for the ability to export InkSeine pages as HTML image maps. There's a number of interesting ways that could be used, including posting the resulting image maps to create an ink blog entry. So that's a feature I would love to get in there as soon as possible.

Page 6: Ideas for Future Versions, Part 3.

On point (6), to have running section heads, InkSeine would have to know that you were inking an outline to do this. It would be pretty tough to make that happen. The InkSeine user interface avoids handwriting recognition and parsing as much as possible.

For (7) and (8) a system tray icon for InkSeine and/or the tool ring definitely would be handy. You can add the tool ring to your quick-launch area.

Page 7: About

Thanks, Anthony, for all your great comments and I'd be pleased to receive any more thoughts that you, or the other merry inkers out there in the Tabletscape, would care to send my way.

If you've read this far, I've also got one little nugget of info to reward Ye, O Faithful Reader. We are planning to release the InkSeine fixes and enhancements mentioned here, along with a few other things, in an update on Monday! If all goes well I'll put up a post over the weekend confirming this, with a list of the exact features that make the cut. But think of this as a maintenance update - it' won't be a major new release.

Keep on inking and thanks for trying out InkSeine.

Photographic Interlude #1

Ken Hinckley — Thu, 24 Apr 2008 10:23:00 GMT

I snapped this photo years ago near Shaefer Lake in the glorious Cascade Range.

The needles of the larch incandesce with the sunlight of an entire alpine summer in the days before they must fall dead to the ground.

Portrait Inking on the OQO Model 02

Ken Hinckley — Mon, 21 Apr 2008 10:19:00 GMT

The OQO Model 02 is almost the same size as my Moleskine Pocket Sketchbook. I suspect this is no accident. To illustrate the point, I scanned them side-by-side. The OQO is slightly narrower, which is necessary to make it fit in my shirt pocket given its 1" girth. By the way, don't let this scan fool you - the screen on the OQO is gorgeous. It's just really hard to scan properly. The other photos below give a better sense of what the screen really looks like.

I set up a custom cover page for my OQO in InkSeine to make it feel just like a new moley fresh out of the shrink wrap. Now I feel like writing important stuff in here.

I also scanned my pocket Moleskine to use for the inside pages. I love having this page style on the OQO - it just seems right.

I prefer inking on the OQO Model 02 in the portrait orientation. I can grip the device more comfortably in this orientation, and there is more room to plant my hand on the screen. This also keeps the touch-scrollers out from underneath my hand. I've experimented some with using the "secondary portrait" orientation, to flip those touch scrollers over to my left hand. That feels great, but since the keyboard rotate function only flips between the primary landscape and primary portrait orientations, it's inconvenient to go to the options panel and hunt for the command to flip to the secondary portrait orientation.

There's one other tip I have for working in the portrait orientation on the OQO's small screen. I was thinking about why it seemed easier to draw in my pocket Moleskine, even though it has nearly identical dimensions as the OQO. It's not so much the small screen size of the OQO, as it is the thickness.

So I slide out the keyboard, and I rest the meat of my palm on that. This feels more like resting my hand on the desk while I draw in my (thinner) pocket Moleskine. The OQO keyboard keys are fairly stiff so I never trigger them by accident while I'm doing this. Typically I do this while holding the OQO in my left hand; the photo below shows me doing this on the desk because I was out of hands to hold the camera, and no tripod was handy :-)

The keyboard is also convenient for hitting the Enter key, modifier keys, or the special OQO hardware hotkeys (such as the screen rotation, brighteness, and keyboard backlight) when the occasion demands.

That closes the book on this post. I'm sure I'll have more thoughts and ideas about using the OQO as I continue to work with it.

Related Posts:

My very first impression of the OQO Model 02

Make a faux-OQO to see if the size is right for you