This post is in response to: “5by5 | The Critical Path #15: The Existence Proof” on asymco.com  and was last updated on 28 December 2011.

This podcast was particularly interesting to me because using visuals to supplement and replace words is critically important in what many of us are working on.  I wanted to ask your indulgence as I go off the deep end here and have some fun talking about finding the balance between words and visualizations, entertaining you along the way.

And in this, my first multimedia blog-reply, I’m officially starting my own trek to find the tools and obtain the skills to create the visualizations I need for my Mac OS app called MiR which accelerates the productivity of iOS and Mac OS developers.  I too am trial-and-erroring in multiple ways and I’ll be sharing and cataloging what I learn.

As for using visuals to enhance spoken podcasts and conference talks, and written blog posts and teaching materials, I’m looking forward to the day when the tools are available which will allow us to quickly throw together a visual with the ease-of-use of, say, Photoshop.

Take for instance these visualizations from the HGTV shows Kitchen Cousins and Income Property, and note the motion graphics by Scott McGillivray’s head…

You can AirPlay these 720p videos clips to your Apple TV from your iPhone or iPad by the way.

Of course, these visualizations easily and accurately transfer the imagery from the mind of the designer to the mind of the client describing objects, relationships, color, texture, and movement within a space.

Visualizations avoid the inevitable errors and misunderstanding that occurs when the original imagery is first converted into words by the speaker or writer, and then errors are introduced again when the listener or reader reconstructs those words into imagery in their heads.

We all understand this problem very well.  The “art of the explanation” is just the first half of the challenge (dehydration) and interestingly no one talks about the art of listening (rehydration).

Okay, moving on to the super deluxe end of visualizations, check out these visuals from Brian Greene’s PBS series The Fabric of the Cosmos and The Elegant Universe which help make understanding astrophysics and quantum mechanics relatively effortless.  :-)  Focus on the attention to detail paid on each of the visualizations…

I love the imagery (at right) used to visualize other dimensions moving around and intersecting one another.  It reminds me of what goes on in our heads, with each membrane corresponding to some knowledge or skill we have, and each side of the membrane representing the interface between easy and hard, known and unknown, clear and confusing.  And at the end of the clip when they were popping up in different colors, well that could represent the diversity of our listeners and readers, all with their own particular understanding of things in different places.

Many other shows use amazing visuals like these including Brian Cox’s Wonders of the Universe and Morgan Freeman’s Through the Wormhole, both on the Science Channel.  Well, they’re everywhere, from Avatar to cartoons.

What tools are all these post-production studios using?  I guess we can call them up on the phone and find out, but I also know most of us can’t afford them.  In addition, we need to be able to use them occasionally, not as part of our day job, so the learning and remembering cost-of-entry must be low enough, as well.

For example, I edited together all these video clips myself using Apple’s Final Cut Pro X, an incredibly approachable yet powerful tool for a weekend warrior like me.  You can tell in these videos that the “ripple edits” in my transitions still need a little tweaking.  

And lastly, we have visuals from the 52-episode CalTech series The Mechanical Universe from 1987 making the point that people have been doing this for at least 25 years already! The Mac was only a couple of years old.  (I silenced the audio, and skip through it if you want ‘cuz it’s kinda slow moving.)

It was a colossal project with each episode packed with custom visualizations and reenactments.  I included a shot of the lecture hall at the end to give a sense of how long ago this was!

So what’s the holdup?  Why aren’t these tools available?

Perhaps part of the challenge is that the world of visualization and animation subdivides into many types with tools tailored to those particular needs.  It’s certainly asking too much for one tool to be able to address all the different requirements, from static visualizations of an architectural design, to the evolving worldwide mobile device markets, to moving through fantasy kitchens, to hairy monsters in games, to explanations of quantum mechanics.

And in my field of software development we have a big problem…

Of the 120-plus books on iOS and Mac OS app development currently in print, and the hundreds of development-related documents, videos, blogs and forum posts, 99.99% of it uses only words with the occasional 2D graphic.   Even Objective-C code is a flat textual representation of an element that lives in a 4D world (3 dimensions + time).

This is not good, and it’s unworthy of the 21st century.  People wanting to redo their kitchens have it better than app developers do.  Pick just about any other field of endeavor and it’s probably the same situation.

Here’s what gets me though… An experienced app developer has a full working 4D model in his/her head of how all the objects and frameworks fit together, how the messages between objects fly through the system, the unique ways in which objects can invoke one another in doing their work, and the accepted construction patterns one uses to create new apps.

But when it comes to teaching a new person how to develop an app, that sophisticated 4D working model is translated into words with those same two places where errors in translation can happen.  On top of that, I’ve identified over 20 different, overlapping word-based sources an app developer learns from.  The student must draw from out-of-order streams of words and construct their own working 4D model piecemeal, repeatedly breaking down and rebuilding incorrect constructions in their heads during the learning process, and this takes months if not years.

4D visualizations of the iOS and MacOS development environment, if they existed, could help reduce the learning process to days and weeks because the chance for error is eliminated and learning time is significantly reduced.

Well, let’s see what we can do about that.  It’s an opportunity!

In summary, words are a stone age tool.  We use them because they’re supremely convenient.  They’re more than good enough in a fiction novel or a radio show when the reader or listener will add their own custom imagery.  But words alone are quite inadequate when trying to communicate and explain things with accuracy, and quickly.  In this case, visuals are an essential companion.  Words will never be eliminated and are used as the tree upon which the visuals are hung, like this blog post.  Being a good wordsmith is as important as being a good graphic visualizer.

Of course, text, words, and language will always have it’s place in the rich fabric of communication and media types, but it’s important to stop and notice when things are out of balance.

Well, it only took four calendar days to produce my first multimedia blog-reply, but I hope it just took a few enjoyable minutes to consume.  Practice makes perfect.  And during a real-time talk or podcast, all these (possibly interactive) visuals must be sitting off-stage and available to use at a moment’s notice without any undesirable pauses in the action!

- Mark

---

Acknowledgements:

[1] The first clip now has the acknowledgements added to the clips.

[2] The second clip is from Brian Greene’s NOVA series Fabric of the Cosmos episode 3 “Universe or Multiverse” and The Elegant Universe episode 3 “Welcome to the 11th Dimension” (Copyright WGBH and PBS) captured on a TiVo HD in 1080i and transferred to my iMac using iTiVo at broadcast quality.

I had to capture the first clip using my iPhone because TiVo is required by the cable companies to respect their copy protection flags, and Kitchen Cousins was copy protected. I can transfer a PBS NOVA series to my iMac because it’s not blocked, but not an episode of I Love Lucy.  AT&T U-verse, however, does not prevent you from playing a video in one room on a box in another like TiVo is prevented from doing.  It’s no surprise to see the content conglomerates playing their turf wars.

[3] The third clip is from The Mechanical Universe and Beyond (Copyright 1987 California Institute of Technology, The Corporation For Community College Television and the Annenberg/CPB Project). The .avi video was converted with MPEG StreamClip. The computer graphics in this 52 episode series were provided by the JPL Computer Graphics Lab.

All clips were edited on an early 2007 24″ iMac using Final Cut Pro X even though this model of iMac isn’t officially supported because its graphics card doesn’t support OpenCL.  I learned how to use Final Cut Pro X in only a few hours by watching the Final Cut Pro X Essential Training tutorial on Lynda.com and I balanced it out with a great text-based book by Larry Jordan for further depth and better random access to information.  The videos are hosted on this WordPress.com blog and published using VideoPress at a maximum of 720p.

The amount of amazing technology I’m standing on the shoulders of is remarkable.