Blog Archive

Tuesday, May 21, 2013

Barfing up Visual Hairballs.


No, this is not a lolcats meme.  This is about poorly formed visual representations of networks.  

Networks are relationships between things.  The ‘things’ could be people, in terms of social or institutional networks, or they could be computing and communication networks that pass and transform information between sender and receiver. 

Why is this important?

  •  On August 14th, 2003, our electrical grid failed.  50 million people lost power for up to two days.  11 people died.  The outtage caused an estimated $6 billion in damages. 
  •  Facebook says it now has 1.1 billion users across the world.  The definition of a user: someone who has a relationship to someone else. 
  • Young adults, and even more terribly - children -  die of when one part of their body (a giant network) fails to work properly with other parts.    
The list goes on and on.  But we pretty much still do a bad job of being able to simply show how these kinds of relationships work. 

WHAT’S GONE WRONG HERE? 


This happens all of the time: a scientist pairs an important but terribly impenetrable discussion or academic paper with an obviously half-baked network diagram that satisfies the “but we need a visual for this.”  There’s plenty of examples but in the spirit of not using this blog to single out individuals, I created my own nasty one for the sake of example:  




Look, you can’t just ram a bunch of network data through a batch of code and blast it out raw onto a page.  Network diagrams need to be the starting point, not the end point of your visualization.  Once you have rendered the layout using whatever approach or algorithms you choose, you need to then turn to user interface and annotative content to satisfy the reader. 

A QUICK SUMMARY OF TECHNICAL ALTERNATIVES. 

Robert Kosara drew on his extensive IEEE VAST contributions and collaborations to put together a nice summary of alternatives to the visual display of networks.  By the way, IEEE VAST is a wonderful collaboration of some of the sharpest, most intriguing experiments in designing information display that satisfies human perception and cognition.  It’s truly wonky, a bit intimidating for the non-scientific person like me, but it’s rewarding reading.  


Here, I'll build on the shoulders of this great work and instead, I’ll lay out some suggestions on framing the user’s needs, and how that has played out with some great visualizations that I’ve seen.  

LET’S CONSIDER: WHAT DOES THE USER WANT?

 Having worked out a few network diagrams with clients, and having read through many more attempts by others, I’m generally seeing three use cases: 
  • Inquiry, where the user is asking specific questions about how things relate to each other;
  • Perspective, where the user wants a view that might yield unexpected insight into patterns or trends of relationships;
  • Analysis, where the user wants to identify how the network will behave under stress, and when/how it will break down.  


1. INQUIRY

Fathom and Reuters recently put out their wonderful Connected China piece, and it's a great example of structuring a visual display of a network for inquiry and exploration.  At its core is a force-directed relationship diagram that shows the institutional- and relationship-based sources of power and influence.  But the strength and depth of this interactive comes from the content and user interface that allow us to explore the topic.  For example: 
  • The interactive features that promote a clean design, in particular linkages that are revealed through hovering the mouse pointer over the profiled individual;
  • Networks and relationships are built over time, and to see this, we can easily transition from relationships to timelines that show an individual's ascension and stations in the Chinese power structure;
  • There are plenty of opportunities to jump straight from an individual 'node' in the network to in-depth content concerning the individual’s relationships and career.  


2. PERSPECTIVE

I recently had a higher education program as a client.  They wanted to review hiring patterns of program graduates by employers, and develop a strategy for improving graduate recruitment.  They had a simple request: they knew the details of graduate recruitment statistics, but they wanted to see the big picture of what they might be missing.  In particular, they wanted to know:

  • Which employers were hiring program graduates out of recognition of the strength of the program itself, versus out of circumstance (e.g. location of the university relative to the company)? 
  • Were universities inappropriately becoming recruiters, given the mission and purpose of the program was to offer graduates commercial careers, rather than academic careers?  Or, was there a hiring company that, due to their intense hiring practices, was unduly influencing the practices of that local instance of the program?




The bigger value for the client was not the network map by itself, but the summarization of findings on the right. Since this was primarily going to be a paper-based discussion and presentation, we developed an 'edgeless' network diagram that substituted topographic map-like placement and color-coding to convey relationship.  We then carried the visual coding over to the stand-alone summary of what we saw as some potential trouble spots.  These weren’t conclusions, either statistically or otherwise.  But they framed important questions that the program felt were important to probe further.  


3. ANALYSIS

 We really don’t want to lose power to the eastern half of the country, like we did in 2003.   The culprit was embarrassing: an untrimmed tree and a broken alarm system in a control room.  How could we have done a better job of seeing this coming?  We need a visual system that helps us search for and spot dominoes that are starting to line up. 

Where to turn?  Albert-Laszlo Barabasi and Ben Schneiderman are two very well-known contributors to the field of systems visualization and network science.  Each have researched how progressive and cumulative failure of individual nodes in a network can eventually cascade into an overall network failure.  

Zhicheng Liu and Jeffrey Heer have made some great contributions to presenting visual tools that illustrate and aid analysis of networks.  For example, Liu’s NetClinic developed a framework for expressing root causes of network failures.  By using machines as central nodes, he then displayed context and relative significance of each machine in terms of the applications they host and the configuration of these applications.  Having established each machine as its own manageable domain, he then turned his attention to the established protocols for communications between machines.  

Again: paying attention to the user experience.  Liu recognized that showing all connections at once was both unimportant and visually disruptive.  Instead, he provided a default view that only showed the connections suspected of playing a role in the network failure.  By creating a context that made sense in the real world, and limiting the amount of information as a default view, Liu made a great start at illustrating a diagnosis. 

Control rooms of major logistics and utility companies feature network diagrams and visuals that, over time, controllers have learned to interpret and manage.  And as technology lets us crunch larger and larger data sets, we can use standard equations and algorithms to help us identify the most important nodes that, if removed, will compromise or destroy the network. Lots of people have worked out a way to visually analyze giant networks, but nothing yet that doesn’t require a pretty steep learning curve and is dead simple. 

The challenge: will we ever have a simple, universally applicable visual tool for helping us quickly illustrate inherent weakness and emerging threats to the stability of a network, no matter how large and complex the network?  Or is this just a fantasy? 

Readers, if you have examples of progress, I’d love to hear from you.  






3 comments:

  1. You might take a look at BioFabric (www.BioFabric.org), which I released in October 2012. It takes a new approach to node-link network visualizations by representing the nodes as horizontal lines instead of as points. Edges are then represented by distinct, unambiguous vertical lines. A quick D3 demo is at: http://www.biofabric.org/gallery/pages/SuperQuickBioFabric.html and there are examples and discussion at: http://biofabric.blogspot.com/ Thanks!

    ReplyDelete
  2. William - yes, I saw this when it came out. It looked like a very clean way to represent large scale relationships. I especially liked the World Bank contract representations. Keep us updated on developments!

    ReplyDelete