Announcing the Microsoft Research series from Microsoft Press

To help celebrate Microsoft Research’s 20th anniversary, Microsoft Press and Microsoft Research today jointly announce the launch of a new book series devoted to the work of Microsoft researchers. Microsoft Research has more than 850 researchers, including some of the world’s finest computer scientists, sociologists, psychologists, mathematicians, physicists, and engineers, and the Microsoft Research series shares their thoughts and efforts. Here's the vision for the series:

At Microsoft Research, we’re driven to imagine and invent. Our desire is to create technology that helps people realize their full potential and to advance the state of the art in computer science. This series shares the insights of Microsoft researchers as they explore the new and the transformative.

Available now, the first book in the series is The future of looking back (Microsoft Press, 2011), by Richard Banks, principal interaction designer at Microsoft Research. Richard's work describes how the digital legacies and "heirlooms" we're creating in our increasingly digital lives will change the way we reminisce and reflect. Future books in the series include titles by Roger Barga (on extracting insights from big data) and Tony Hoare (on the theory of programming, its motivations, and its philosophical inspiration).

In this post we share two extended texts to help introduce the first book in the series. The first is an interview with Richard, conducted by Devon Musgrave, series editor, and the second is the book’s foreword. The interview took place via email in September 2011.

Q: Your work on “Technology Heirlooms” at Microsoft Research Cambridge recently won an International Design Excellence Award in the Design Research category. What are the concerns of that work?

Banks: The socio-digital system team, which I’m a part of in Cambridge, focuses a lot on research for the home. We’re very interested in what makes people tick and how individuals, friends, and families live their lives day to day. We hope that by looking at what we think of as basic human values, we can figure out how technology should fit, rather than forcing technology onto people that doesn’t support the way they really live.

One of the things we’ve been interested in for a long time is how people arrange their homes as a form of personal expression. What their home says about them, if you like, as well as what their relationship is to the things in their home. For example, we’ve visited people’s homes to look at what they chose to put on display for visitors to see (maybe a formal family photo on the mantelpiece or a trinket from a holiday that triggers memory or invites conversation). In addition to what they put on display, we’ve also been interested in the things they decide to keep but don’t put out. We all have boxes in our basements full of stuff we can’t bring ourselves to throw away. We’ve been interested in what it is about that stuff for quite a while and why people feel such a strong attachment to it. Naturally, the issue of legacy comes up—people keep a lot of these things with the goal of passing them on to their offspring.

The Technology Heirlooms project has been all about thinking about what legacy means for the digital things in our lives. For example, in the past we took analog photos and could count them in the hundreds, maybe. Now we take digital photos, thousands of them, and post them online to share instantly with others. With the analog photos we might have just kept them in a shoebox or a photo album, which is how they would be passed on to our family. What’s the equivalent of that for digital photos? Similarly, in the past we might have written a diary, which again would be passed on to others. Now, we share our thoughts and actions online with friends, which is the closest digital equivalent to a diary and like a diary could form a valuable record of our lives once we pass away. In what form should we preserve these digital diary entries?

With this project, then, we’re interested in looking at the fundamental human values of legacy—why it feels instinctual to want to preserve and treasure the things we’ve been left by our grandparents, for example—and thinking about how those values apply in our digital world.

Q: Are you finding in this research that people want to get a handle somehow on the inadvertent heirlooms they’re creating digitally? Given the amount of material, is the urge for curation/editing/valuation still there? Or are some of these pictures and words considered akin to pre-digital memories and conversation despite their digital presence?

Banks: I think some of that urge for curation and editing has always been there. Many of us have boxes full of old photos that “one day” we plan to organize and maybe even arrange in photo albums. Unfortunately, we never find the time we thought we would have to deal with them, and every time we see them we feel a little pang of guilt. One of the challenges with digital inheritance may be that we feel exactly that same sense of guilt, but it is exacerbated by the growth in the number of items that we are dealing. In this sense the digital items are very like the pre-digital—the emotions and needs are the same but the scale is different. In physical terms we’re talking about rooms full of pictures, rather than boxes. People still want to do something with this content, but it is tougher to get a handle on it because it’s stored somewhere virtual (on a hard drive) and doesn’t act as much as a reminder. It’s also tougher to handle because the quantity of items makes it difficult to know where to even begin in organizing and particularly editing, narrowing down content to the most meaningful items. We either need systems that help us do this editing and maybe even edit automatically on our behalf, or we need to come at this content a different way. Maybe it’s better to let go of the idea of being able to organize all these digital heirlooms, but instead treat them as a large pool that we have a more serendipitous relationship with, where the delight is being shown something new and random from it, rather than something structured.

Q: As you describe in the book, significance in reminiscing and remembering is a moving target because our notions of what’s meaningful change over time. Do you think systems that would enable us to create multiple takes on a digital heirloom or allow us to have the serendipitous relationship you mention might better reflect that?

Banks: I think both the idea of multiple narratives on a legacy, and of providing more spontaneity for experiencing digital things that have been left behind, are powerful ideas that should be explored through technology and design. There’s no question that my mother has quite a different set of memories and perceptions of my grandfather than I do, for example, that aren’t just tied to the fact that she knew him for much longer than I did. To her he was almost a different person—a father rather than a grandfather—and she knew him in the context of my grandmother’s life, while I knew her less well. I was acutely conscious of this when creating some of the prototypes I did that were based on my grandfather’s life. I would love to see her take on his life through a device like the Timecard we built, overlaid on my own. What she might choose to show would be quite different, as well as how she might choose to describe him.

Digital technologies also allow us to play with time and bend it in unexpected ways. Multiple timelines can be visualized in parallel, for example. I could have a timeline of my life and put my grandfather’s next to it, both starting at the same moment, rather than being separated by 50 years. In this way I could compare what I was doing when I was 18 with what he was doing. I was starting at university when he started in the RAF. I had my daughter around the same time that he was fighting in WWII.

I think the idea of having a serendipitous relationship with the contents of a digital legacy is an interesting one. This is driven by the quantity of digital things we are likely to start inheriting, and the fact that we both can’t manage it all and can’t expect it to have been organized for us, so we have to find new ways of engaging with it. It’s a reminder that randomness can be a compelling thing. I like listening to all my MP3s on random, for example, and uncovering bands, albums, and tracks that I haven’t listened to in an age and that take me back to some period of my life. Similarly, we could have that kind of unexpected relationship with things we’ve been left—images of a person from a time we hadn’t focused on, for example. Many digital items have metadata built into them too now, details about time and place, that can help make this random experience of content have a little more context around it.

Q: Technology and design that allow for such narratives and relationships might prevent the massive deletions of legacy likely to occur otherwise. Do you have a philosophical stance on when it’s OK to delete legacy?

Banks: I think this tends to be a personal issue. When I’m taking photos with my camera, for example, I don’t delete anything. I think of each image as potentially connecting me to a moment that I might forget about, even if that image is a poor one. Perhaps I’m simply paranoid and don’t trust my head. My wife is happy deleting many of her shots, and I have friends who don’t keep images unless they are perfect.

So I don’t think that curation of a potential legacy is a bad thing, per se, except for the risks that might be inherent in not knowing what might be interesting to future generations. In the end, some order applied to these legacies is better than none at all. I suspect it’s human nature, though, to never get around to managing a legacy and therefore these large volumes of unsorted digital content are likely to be the norm.

One thing that seems true about our digital lives is that there is some implicit layer of organization that comes about through the things that are done to an item. If I post an image to the photo-sharing website Flickr, for example, that implies that it may be of more interest, generally, than other photos I may have taken at a similar time or at the same event. Once that photo is shared and it starts being viewed and commented upon by others it stands out a little more, again, from a general pool of images. It’s dangerous to assume the motives of posting one image online but not posting another, but with a very large body of digital items in a legacy these bits of activity that happen to some items but not others may be a key way in which we disambiguate between the things that “matter” and those that don’t.

Q: Your book discusses these issues with two audiences in mind: a general audience interested in how technology is altering human experience, and designers of devices and tools that handle and manipulate digital data. Would you say a bit more about this?

Banks: The book is about the changing nature of legacy. In the past much of this legacy was bound up in the physical things we preserved. Legacy was the responsibility of an individual and their key control for this was the material form: the choices that they made for what they kept and what they discarded, the space they had in their homes for making these choices, and so on.

With digital things this material connection to legacy starts to erode. A Tweet or a Facebook post might form part of my legacy, too. Suddenly I’m dependent on the systems that these items are part of to let me preserve the things I’m interested in for my own personal reasons. My legacy doesn’t just live at home—it’s distributed.

With this book I’m basically asking the general reader to consider the things that form their legacy, and the changing practices and forms that may occur around it. Hopefully it’s interesting to think about what we’re producing now—the digital byproducts of our lives, for example—as being worth preserving or to consider how we might deal with our own burgeoning digital photo collections as an archive.

For the designers of devices and tools I’m asking two things. The first is to consider the value of what they build in the long term, as experiences that have lasting meaning and that in the end might not belong to them as creators but instead might have value to their consumers well past the point in which they plan to invest their own time.

Secondly for these designers, I’m asking them to reflect on what it means to put a fundamental human value such as legacy at the center of their design thinking. How might it change the way they go about constructing the experience of their product, for example? What new and unexpected things might they think to build?

Q: In his foreword, Bill Buxton writes, “What I take from Richard’s book is a value statement that I find as provocative as it is insightful: the nature of legacy may well trump short-term coolness, in terms of long-term value. Furthermore, by asking the right questions at the right time, you may not need to pit one against the other.” You’re both emphasizing a significant expansion of design thinking—what are some of the current obstacles to that expansion?

Banks: I think a primary obstacle is that, as Bill implies, a lot of new technologies are simply very cool and compelling. They let us have experiences that we have never had before. They let us connect with others, for example, and share our thoughts in a dynamic fashion that wasn’t possible only a few years ago. We forget how new our digital world is. I read blogs like Gizmodo or PSFK, which highlight a lot of new services and technologies, and the rate of change is staggering. On a weekly basis I read about some new online site, for example, that I feel compelled to be a part of.

I think a lot of these tools offer real value and have real meat behind them. But we haven’t lived with them for very long, and the danger is that we never will. Something new will come along and replace them before we get a chance to really explore their potential and—a key point of the book—understand the impact they may have over a lifetime or even generations. Sometimes I think of this in terms of musical instruments. I see new blog posts regularly about a new digital musical instrument that someone has invented somewhere. It might use some new technology in an interesting way: be mind-controlled or gesture-based, for example. There’s a big difference, though, between inventing and tinkering with a new musical instrument, and investing a lifetime in one so that you really, truly understand its idiosyncrasies and nuances. That long-term investment implies that you make sacrifices and choose not to engage with a lot of our contemporary distractions.

We are very excited by our own creations, and in many cases there’s good reason for that, but it’s much harder in our technological world to imagine the impact they might have in the long term and the benefit they may provide over a lifetime and into legacy. The question is whether we will enter a phase of “digital normalcy” where we get used to technological change enough to give it more consideration over the long term, or whether our world from now will always be one defined by rapid change.

Q: What would be some clear signs of this “digital normalcy”?

Banks:  Perhaps a defined sense of the technology becoming much more a background rather than foreground part of our life. The motor (whether electric or internal combustion) is often used as an example of a technology that was once very visible but now permeates many aspects of our life such that we no longer think about it being technological. It’s pragmatic and modest.

One thing that I’m very interested in with my book is the sense of passing on between generations. We’re really getting to the first generation now that is leaving behind a digital legacy in addition to their physical one. So there’s a sense that people are starting to have to come to terms with issues of inheritance when it comes to our digital lives, whether they want to or not. We can start to design now with that inheritance in mind if we choose to, rather than having to leave it as an afterthought, and doing that may reframe and rebalance a lot of our technology in terms of the long term rather than the short.

Devon: Here's to that reframing and rebalancing. Thanks very much for sharing your thoughts, Richard, and congratulations again on the book.

Bill Buxton, principal researcher at Microsoft, whose most recent book is Sketching User Experiences: Getting the Design Right and the Right Design (Morgan Kaufman, 2007), contributed the foreword for Richard's book:

In this world of rapid—and too often planned—obsolescence, where we seem to automatically equate smaller, bigger, faster, cheaper, higher resolution, and so on, with better, there is a certain value in stepping back from time to time and meditating on what words like “better” or “progress” actually mean to us. Not just as individuals—although that is important—but also as a family, culture, and society. As someone who loves old books but who also works in the world of digital technologies, I find it interesting that not only can I read a first edition of the journals of one of the explorers of Canada, but the experience of doing so is significantly better, and more intimate, than reading the same text on the most fancy new gadget. Not that there aren’t times when I welcome having the text on a digital gadget. Being able to search for a particular passage rather than have to rely on my faulty memory, or possibly an inadequate index, is something that I don’t want to give up any more than my first edition. But in terms of the experience, the feel of the paper, the quality of the illustrations, and the visceral sense of connection to what I am reading about, there is no competition to the original book. I love, cherish, and regularly give thanks for the connection to the past that the technology from that earlier time offers me.

And to get closer to the point, I also realize that the nature and intensity of that connection would be even stronger if the book had been written by my ancestor, and even more so, had this copy been theirs. From such an example emerges a range of questions that should, but too seldom does, confront anyone designing or purchasing many of the “new and improved” technologies of today. The future of looking back, by Richard Banks, offers one of the most literate, thoughtful, and balanced explorations of such questions.

For example, some of my treasured books are well over 100 years old, yet they still function as well as they ever did. There is no dead battery, fried circuit board, or incompatible media that prevents me from picking up on the exact page from which some predecessor may have left off. Now think of the documents, videos, photos, recordings that you grew up with—not just yours, but those of your parents and their parents before them. Does it not seem ironic that chances are, the older the media, the longer they are likely to survive in a usable form? Sheet music from 200 years ago is as good now as it was when it was written. Can you say the same about your laser disks, 8-track tapes, favourite Atari video games, CD-ROMs, etc.?

What Richard does is explore this space in such a way as to judge it in the court of human values, rather than technological specifications or market penetration. And yet, in so doing, he does not present us with a whining tome nostalgic for the good old days. His approach is both balanced and positive. This is no diatribe bemoaning how technology is destroying our culture. It is far more closely aligned to one of my favourite insights, articulated by the historian of technology, Melvyn Kranzberg: “Technology is neither good nor bad; nor is it neutral.”

Essentially, Richard’s argument is that the future we get is the one that we design. If we ask the right questions and inform our decisions from a careful consideration of our true values, we can shape our technologies accordingly. Most appropriately, he makes his arguments around that most human of values and needs: weaving a thread of connectivity—emotional, intellectual, and tangible—from our past, through our present, to our future.

Legacy is not just what we inherit from those who came before us but also what we leave to those who follow.

What I take from Richard’s book is a value statement that I find as provocative as it is insightful: the nature of legacy may well trump short-term coolness, in terms of long-term value. Furthermore, by asking the right questions at the right time, you may not need to pit one against the other.

Thanks to Richard’s book, we are far more likely to ask such questions in such a way. For that, we should thank him—we and those who would inherit the richer legacy that could result.

Microsoft Research books will be available in print and electronic formats at your favorite book retailers. And see The future of looking back, available at Amazon, Barnes & Noble, and O’Reilly Media.