Blog Archive

Tuesday, June 25, 2013

Machines That Mine Cultural Meaning.

Last week’s blog looked at the implications of our government’s use of data capture and analysis.  Technology will inevitably render our private correspondence with others, whether personal or commercial, more and more vulnerable to interception and compromise.  But there’s a bright side to the ability to step back and examine our body of expression – better insight into who we are, and how we are changing.  Still, we have to remember: meaning is highly subjective. 


The Cocteau Twins were a Scottish alternative music group active in the 1980’s and 1990’s.  They employed musical soundscapes that built a cult following and passionate inspirations.  Their lead singer, Elizabeth Fraser, is a fascinating yet intensely private star whose lyrics defied interpretation and translation.   It was hard to pick out words from her tremulous yet shattering voice, further overdubbed and processed in the studio.  What made it harder: part of her art was the invention of a personal but highly expressive lyrical language.  

I love music that is emotional and challenging – whether it be angry, sad, frightening, or in the case of the Cocteau Twins, often an expression of joy and ecstasy.   So, I was drawn to their music from the first time I heard it.  Now, as an information designer, I am equally drawn to the challenges of developing perspective for others.  How can I provide perspective on something critical, meaningful, and beautiful across a hugely complex body of information?

In this visualization, I took Liz Fraser’s body of known lyrics, and arranged it into a semantic analysis that helped create some suggestions about her personal transformation.  

I could have picked an easier project.  But it wasn’t the complexity or scope that made this project difficult.  In fact, it was this question I kept coming back to: how could I honor Liz Fraser's mystery without trampling on it with cold, impersonal analysis?  Was I furthering an intrusion laid down by others’ analysis, regardless of how much love those who had done the analysis had for the music? 

It made me feel better that I used a set of lyrics that had been transcribed by people who clearly cared about the work and had been occasionally in touch with the group members over the years.   But the final step I took left me with mixed feelings.  I analyzed Fraser's language to interpret her personal transformation from a young and expressive woman entering the spotlight, to a more self-possessed adult.   On some days, when I look at this, I feel like I’ve been presumptuous and arrogant.  On others, I feel I did a respectful, honoring job of creating some perspective on this question for others to enjoy.  


Last week, I mentioned the Narus Semantic Traffic Analyzer as an example of how law enforcement and national security agencies have probably overstepped their boundaries.  But on a brighter note, scholars have found opportunities to take algorithms - established by mathematicians like Andrey Markov and shaped into information theory by  mathematicians like Claude Shannon - and to apply computing technology to produce semantic analysis.  The result: machine-generated perspective on art and literature.

A leading example of this work is Stanford Literary Lab’s Franco Moretti.  Using what he and others have coined ‘distant reading’, ‘digital humanities’, and ‘literary macroanalysis’, this machine-generated perspective has inspired both admiration and scorn, depending on the context and perspective in which his ideas are taken. If you are excited (like me) by the possibility of being able to use computing to observe semantic or structural trends over time or across a major body of literature, then you will understand that his work isn’t a reductionist’s attempt to pass off literature as predictable, formulaic exploitations of human interests.    

Above is just one example. Moretti and his colleagues took nearly 3,000 19th century British novels and developed a cohort analysis to describe the trends they saw when analyzing the language of this body of work.  It offers a rich opportunity for illustrating theories of how societal and economic trends of 19th century England may have affected her literature.

I love wonderful literature.  When I’ve finished fantastic books, I’ve felt transformed, and more accepting of our struggles and joys.   I’ll be reading books for the rest of my life.  But I also love the idea of also seeing how art and literature can express how we transform as a society.  

Lev Manovich of the Software Studies Initiative has taken the same approach to works of fine art, in particular works by Van Gogh, Mondrian and Rothko.  By arraying their works over time in tight clusters, we can see how each of these artists evolved in their use of color and representation.  

This is made possible through Lev and his team’s development of the ImagePlot macro for ImageJ.   It’s a double win: his team has both leveraged the highly sensitive and perceptive analytical capabilities of ImageJ, and yet thoughtfully used a simple GUI to help the tool be accessible to a much broader set of users.     


On a small scale with the Cocteau Twins, a medium scale with Manovich’s work, and on a grand scale with the Moretti work, there’s a common thread.  We’re looking for meaning.  What did the author or the artist mean by what they wrote and painted?  What affected their work?  How were they influenced? 

Despite the massive amount of mathematical tools available for semantic analysis, Moretti described the difficulty he and his team had in finding the right word associations that would result in a reliable framework.  It was a bit of a fluke, that, out of frustration, when they entered the word ‘hard’, they found a statistically reliable anchor for much of their work.  It’s also a bit ironic, given how the word both is used as a descriptive term for physical (versus social) sciences, and how that word is rarely used to describe human-centered themes.  Especially when words like love, pain, or special are so rich and deep in subjective meaning.

Was their selection of a less 'human' word a cop-out? Noam Chomsky, one of the most celebrated 20th century linguists, takes issue with any presumption (much more our efforts) that mathematics, statistics and other pattern recognition tools will help us reliably divine meaning.  Regardless of what side of the argument you choose, It raises an important question: at what point can we confidently say we’ve moved from arrogance and presumption into a confident interpretation of others’ decision-making, and action-taking?  

There’s no better example of how we struggle with this than our own judicial system. Possession and murder are presentable staples of the legal system.  But there’s mitigating and aggravating circumstances, and these often boil down to intent.  Intent creates the need for interpretation and decision making by courts, judges and juries.

Going back to the problem of government surveillance: our evidence-collecting technology will always, inevitably, improve.  However, subversives also tend to innovate.  Technology tends to be the eventual equalizer between freedom fighter and totalitarian.  And technology tends to accelerate, rather than delay, the cycle between regime and revolution.  

We can worry and fret as much as we want about the manifestation of the inevitable in government agencies, but our gravest concern must be the preservation of our independent, uncorrupted human judgment in our institutions of justice.   We can use machines to assist with interpretation, but to them we will foolishly cede final decision making.   Sooner or later, that evidence must be presented for a decision before a human judge and jury.  For those of us in the United States, we still take this for granted.  Let's not let that part slide away.  For those of you who live in countries where the judicial system is overtly a tool of the government, your voice is truly important in this discussion.  

Consider this: if you live in a country where you are still able to vote for judges in your local courts, when was the last time you had an inkling of familiarity about the candidate to whom you gave permission to judge?  Was the candidate for whom you voted in the pocket of the government, or an independent check to executive power? Were they intellectually lazy, or did they truly respect the responsibility of having to delve deep into the humanity of intent? 

The judicial system is the one of the final defenses between us and a government that wields tools to advance their own power.  We must keep it stocked with people who understand the importance and depth of the subjectivity of our truth, and who have the humility to continue questioning themselves and others.  

No comments:

Post a Comment