Newsweek has a fascinating article about an archaeological site at Göbekli Tepe in Turkey that is well worth a look.

The site is the oldest religious temple ever discovered.  Preliminary carbon dating has determined that some of the artifacts date from 9,400 BC, which makes the place about 11,500 years old. (Which, just to be clear, is 7000 years before the Great Pyramid and 6500 years before Stonehenge.)  The article further explains:

The ruins are so early that they predate villages, pottery, domesticated animals and even agriculture – the first embers of civilization.  … [It] may be the very first thing that human beings ever built.

And yet, the site is amazing.  The pillars show beautiful stone carvings and there are examples of sophisticated engineering techniques.  The stone circles are nearly 30 yards across with pillars that stand more than 17 feet tall.  Many of the stones (some weighing up to 50 tons) were first quarried and then transported half a kilometer to the site, where they were erected.  What staggers me, though, is that the stone circles were roofed.

This quote from Ian Hodder, head of archaeology at Stanford University, summarizes my response pretty well:

[Göbekli Tepe] is unbelievably big and amazing, at a ridiculously early date.  The huge stones and fantastic, highly refined art [changes everything].  It overturns the whole apple cart.  All our theories were wrong.

This doesn’t happen often.  Scientists don’t admit mistakes and call for established theories to be overturned.  But when faced with such a revolutionary piece of evidence, you have little choice.

Göbekli  is literally an outlier in every way.  It shows engineering, organization, and artistic sophistication that seems to materialize out of nowhere.  The only other comparable examples won’t appear for five thousand years.

To really put this in perspective, consider the timeline below.  Arrayed across the bottom axis are the reigns of several ancient civilizations: the Chinese, Romans, Egyptians and Mesopotamians.  In addition to this information, I’ve also placed the approximate dates of the the ice age, stone age and examples of religious and cultural monuments (the oldest of which dates to about 3500 BC).

When compared with Göbekli, the great civilizations and monuments of the ancient world seem to to huddle in an upstart mob at the right of the chart.  Even the very oldest of the examples, a Mesopotamian palace, is separated from Göbekli by the same span of time that divides the ancient age from the modern day.

Such an amazing and sophisticated example at such an early date, literally, boggles my mind.  It's absolutely amazing.  And, paradoxically, the amazement and wonder helps to explain why Göbekli has remained essentially unknown.  A discovery of this magnitude demands enormous attention and dedication.  It takes almost as much as it gives, particularly from those that discovered it; and not every scientist is willing to give that kind of commitment.  Thus, I completely understand the response of the man who discovered the site.

[Unable to interpret what he saw], the [American] archeologist who stumbled on [on the site] in the 1960s simply walked away.

But, even so, the evidence at Göbekli has the potential to completely transform the history of civilization.  And I, for one, look forward to seeing what emerges.

Note: You can view a high resolution PDF of the timeline by clicking on the image, or here.

Similar Posts:

• Moving to blog.oak-tree.us
• Typeset Your Curriculum Vitae – Part 1: The xetexCV Document Class
• Big Things, Little Things and Unassociated Thoughts
• Image Gallery: The Notebooks of Leonardo Da Vinci
• Photos of Horses

Copywrite 2009: Rob Oakes. Apolitically Incorrect

Temples and Timelines

No study of the history of scientific communication can be complete without mention of Joseph Charles Minard, a 19th Century French civil engineer and cartographer.

At the end of his life, Minard created two very famous examples of statistical charts, called flow maps, that every scientist, engineer and student should be familair with.  The first showed Hannibal’s crossing of the Alps (218 BC, Second Punic War), and the second describes Napoleon’s disastrous invasion of Russia (1812-1813).

Both examples are beautiful works of art and masterful examples of evidence.  But they are also more than that, they tell cohesive and interesting stories.  In this post, I thought it might be interesting to take a closer look at the history of Hannibal and Napoleon, and highlight the ways which Minard’s charts help us to explain their eventual outcome.

(Note: High resolution, PDF versions of the two maps are available for download.  These versions have been translated from the original French.  To download, either click on the images, or here for the Hannibal invasion of Northern Italy, and here for the French Invasion of Russia.)

Hannibal Invasion of Italy (218 BC, Second Punic War)

Hannibal was perhaps the single greatest threat to Roman power to ever live.  At the same time that Roman legions marched unopposed through most of the Mediterranean world, Hannibal lead a force of twenty-five thousand soldiers into the heart of Roman territory. He then spent the next fifteen years occupying portions of the Italian peninsula and generally wreaking havoc.  But even though he ravaged the Roman countryside, Hannibal was never successful in subduing Rome.  Ironically, this might be due to one of his greatest exploits: the overland crossing of the Alps.

Hannibal departed his home base in Spain with nearly 90,000 troops (one of the largest armies that had ever been fielded at that point in history) and had a number of early successes.  He marched north over the Pyrenee mountains and into Gaul (modern day France).  He subdued the Celtic tribes and crossed the Rhone River before the Romans could take measures to stop his advanced.  He forged important alliances and found unexpected friends in his fight against the Romans.

After crossing the Rhone, however, Hannibal’s luck began to change.  He received word that a large Roman force was marching from the Mediterranean coast to intercept and destroy him.  As the force outnumbered him by a significant margin, Hannibal was faced with a difficult choice.  Should he stay at his current location and engage the Roman army or retreat and prepare a defense?

Not content with either choice, Hannibal ultimately chose a third option; which, ironically, would have enormous consequences for his overall campaign.  To evade the Romans, Hannibal took the unprecedented action of turning inland and marching over the Alps.  (It is thought that he probably crossed over the valley of the Drome and south of the Col de Montgenevre or near the Col de Mont Cenis.)  This had never been done before – indeed many thought it impossible – and has long been praised as a brilliant tactical decision.

Minard’s reconstruction of Hannibal’s journey shows the journey from Spain, through Transalpine Gaul and eventual arrival in Italy, allows it to be seen a slightly different light, however.  Through use of a flow line, Minard shows how the strength of Hannibal’s army waned through the march, with a disastrous loss of twenty thousand men while passing over the Alps.  This steady loss of strength, subtly reinforced by Minard’s map, helps to explain why Hannibal failed to subdue Rome.

Hannibal’s brilliant tactical decision was also a strategical disaster.  It came at a dramatic loss of men, war elephants, and material with the most disastrous loss being the destruction of his siege engines.  Without the siege engines, Hannibal was never able to successfully attack Rome.  And eventually the Romans realized this and changed their tactics.  Instead of engaging Hannibal directly, they instead fought a long-pitched war of attrition and eventually destroyed the one resource he couldn’t replace: his men.  Thus, even though crossing the Alps helped Hannibal to win a few stunning victories, the decision eventually caused him to lose the war.

Napoleon’s Invasion of Russia (1812-1813)

Like in the case of the Hannibal, Minard’s depiction of the 1812-1813 invasion of Russia also tells an extraordinary story.  It combines information about the size of the army, geography, sub-campaigns and temperature into a single narrative.  This time, though, it is anything but subtle.

Note: Because of  the elegant management of all this information – at any time showing the army’s direction, size, and loss relative to the start – some statisticians and data visualization experts (such as Edward Tufte, Etieene Jules Marey and Howard Wainer) have declared Minard’s depiction of Napoleon’s Invasion of Russia to be the greatest numerical graphic ever created.

From the very beginning, the army met with tremendous hardship.  The cold temperatures and lack of available resources took an enormous toll.  Napoleon had planned on scavenging for supplies to support his massive army, but the Russians had destroyed all of their crops and burned their villages so that Napoleon wouldn’t be able to make use of them.  As a result, the army was heavily reliant on slow supply trains from Europe that were unable to keep pace with its quick advance.  Hundreds of thousands died from starvation and exposure, greatly outnumbering those lost in combat.

Minard’s map shows this trend of devastating loss.  Napoleon’s Grand Army starts out as a wide, brown line to the left of the map.  As the army progresses across Russia, the number of men decreases dramatically.   A few ribbons of men are dispatched to other targets, but by far the majority succumb to the elements.

Eventually, a substantially reduced force (100,000 men, about 20% of those who began the campaign) arrived in Moscow.  What they found did little to improve their circumstances.  The city had been abandoned and burned to the ground.  No food was available.  The only available option was to return to France.

Demoralized and broken, the Grand Army began a slow retreat to the west.  Their progress is shown in the black.  As they crossed the vast Russian wilderness, the temperature turned bitter cold (shown in the bottom third of the map).  When the army arrived at the Niemen river, there were a scant 10,000 troops.  More than 400,000 had died.

Such a devastating loss of life crippled Napoleon’s young empire and left him open to invasion by his enemies.  He was defeated later in 1813 and exiled to the island of Elba.

Conclusion

Minard’s graphics are successful because they demonstrate powerful trends and place evidence within its proper context.  In the case of Hannibal, it shows how he began in a position of strength and then undermined himself through a brilliant tactical innovation.  In the case of Napoleon, it shows how distance and temperature ravaged an army and destroyed an emperor.

The clear evidence and thoughtful presentation define the story, they show where things are headed and help to foreshadow the inevitable conclusion.  For this reason, they are powerful examples of clear communication.

____________________________________________________

Unrelated Thoughts: Re-Visioning Minard with Open Source Tools

Since first seeing these charts as part of an undergraduate statistics course, I’ve been obsessed with them.  I even have a poster sized copy of the Russian invasion (available from Graphics Press) hanging in my office.  Given this obsession, it should come as no surprise that I’ve wanted to include the maps in my book.

There is, however, just one problem; there is not anything new to say about them.  Edward Tufte has held up Minard as a “Da Vinci of Data” for the better part of thirty years.  Nearly every one of his books includes a copy of the maps (in the case of Beautiful Evidence, his most recent work, there is an entire chapter devoted to the graphics; including eight different reproductions).  Other than, “They’re beautiful, study them carefully, read Tufte’s analysis,” there just isn’t that much to say.

That is, until I started looking for examples of scientific illustration that could be reproduced using the vector illustration program, Inkscape. Any such example should have simple lines and clean integration of text and images.  I'm not sure that a better description of Minard's work exists.

The two images above (and attendant PDFs) were my first pass at re-drawing the maps.  And I have to say, I’m rather happy with the result.  For the most part, I was able to get a very faithful reproduction (with the exception of the type face, which I deliberately changed because I hate script fonts), and only required a couple of hours worth of work.

I think that I may have found my book examples.  It allows me to include one of the best statistical charts of all time, talk about flow maps, and even include some interesting how-tos which show how the graphics can be built in an automated fashion.  Which, come to think of it, might just count as a new contribution after all.

Acknowledgements and Further Reading

More information on the book can be found at the publisher’s website.

The source images for the two maps included “Re-Visions of Minard” by Michael Friendly (1999) and the reproductions in “Beautiful Evidence” by Edward Tufte.  Translation for the Hannibal map was adapted from those available on Edward Tufte’s website (Dawn Finley) and sources available at Re-Visions of Minard.  Translation for the Russian campaign map were taken from the reproduction in Edward Tufte’s “Beautiful Evidence”, Wikimedia Commons and sources available from Re-Visions of Minard.

Additionally, the Re-Visions of Minard Website include a number of interesting adaptations of the Napoleon graphic that might be of interest, including both Matlab and GGPlot2 reconstructions.

Similar Posts:

• Anathem: Big Book, Big Ideas
• Customizing LyX: Character Styles and the LyX Local Layout
• Heart Arrhythmias and MRI
• WPF – SVG Graphics and XAML – Part 3
• Typeset Your Curriculum Vitae – Part 1: The xetexCV Document Class

Copywrite 2009: Rob Oakes. Apolitically Incorrect

Hannibal, Napoleon, and Joseph Charles Minard

There are three tools that a professional, scientific or technical writer needs to make use of: words, numbers and images.  In many cases, such as an effective illustration or chart, all three will be used.

The following books introduce principles and examples of how to use these tools to their fullest extent.  Some of the titles are historical and others are academic.  In every case, though, they highlight strategies that can be used to more effectively communicate ideas.  Additionally, each one is also an interesting and fantastic read.

	Math and the Mona Lisa by Bulent Atalay. For more than 500 years, the name of Leonardo Da Vinci has been synonymous with brilliance.  His careful observation of nature, collection and analysis of evidence, and use of mathematics to explain his observations represented a radical shift that foreshadowed the modern scientific method.In this book, Bulent Atalay explains why Leonardo was a remarkable artist, engineer and scientist.  He looks at the hidden patterns, geometric concepts and impeccable perspective in order to probe the mind that dreamt of helicopters, unsinkable ships and underwater exploration.
	Leonardo’s Notebooks, edited by H. Anna Suh.  To understand a man, you must read him in his own words.  This volume provides an opportunity to sample Da Vinci’s writings on anatomy, botany, architecture, sculpture and the physical sciences.  The key illustrations from his notebooks have also been reproduced.
	Galielo at Work: His Scientific Biography by Stillman Drake.  Like Leonardo, Galileo was a scientific titan.  As Stephen Hawking aptly summarized, “Galileo, perhaps more than any other single person, was responsible for the birth of modern science.”But why?  What was it about his innovative combination of experiment and mathematics that was so important.  How did he analyze data?  How did he present it to others?This book attempts to answer those questions.  It lays aside the philosophical implications of Galileo’s rift with the Catholic church and instead looks at how Galileo focused his mind on physical quantities and the mathematical relationships between them.
	The Visual Display of Quantitative Information by Edward Tufte.  Communicating complex ideas is difficult.  One of the most important tools in that struggle are charts, graphs and illustrations.  Unfortunately, however, these important figures often receive less attention than other aspects of a manuscript.In this book, Tufte provides inspiring examples of graphics that are beautiful to behold and illuminating to ponder.  He also includes shockingly bad examples and explains why they are so dangerous.
	Visual Explanations by Edward Tufte.  In his earlier work, Tufte showed how important it is for numbers to be communicated clearly and without distracting  ornamentation.  In this volume, he turns his attention to a slightly different series of questions: What is the best way to show cause and effect?  Or to demonstrate evolutionary change?But the most important question he asks is far more universal: How can an information display be be used to reveal the truth?  To answer this, he analyzes a cholera epidemic in 19th century London and explains how poor communication contributed to the explosion of the Challenger space shuttle.
	Beautiful Evidence by Edward Tufte.  Like in his previous books, Tufte again tackles the question of how to best reveal truth through the graphical display of information.  But where earlier books focused on principles, Beautiful Evidence is about how seeing turns into showing.  To explore that theme, this book is filled with hundreds of spectacular examples and thoughtful commentary on what makes them unique.
	Now You See It by Stephen Few.  The human mind is amazingly adept at seeing and understanding patterns.  An informed eye can distinguish between authentic and forgery and arrive at startlingly accurate calculations with minimal effort.  But even though we are capable of recognizing the hidden influences in the world around us, we can also be mislead and exploited far too easily.  We become awash in a sea of data of our own making.This book attempts to explain how the mind interprets and sees information.  As the author explains in the introduction, “[This book] provides tools to dive into the ocean of information, net the best of it, bring it back to shore and sort it out.”  In essence, it’s a book about seeing and distinguishing patterns on a conscious level.
	Visual Thinking by Rudolf Arnheim.  It’s long been known that “seeing is believing.”  This book explains why seeing is also synonymous with thinking.
	Maps and Civilization: Cartography in Culture and Society by Norman J.W. Thrower.  The history of exploration and discovery is also the history of cartography.  As mankind sailed out of sight of shore, he needed to learn techniques for representing his position and understanding the natural forces that he might encounter.  This book tells the history of mapmaking and how advances in cartography impacted civilization.
	The Elements of Graphing Data by William S. Cleveland.  In this book by William Cleveland, he presents the nuts and bolts (the how-to) of graphing data.  Then he goes on to explore the science in which his principles are based..
	Visualizing Data by William S. Cleveland.  Whereas The Elements of Graphing Data is primarily  focused on the principles of quality display and exploration of many types of common statistical charts, Visualizing Data takes the next logical step.  It introduces a number of new chart types and techniques for creating insightful and clear graphics.
	Fiasco: The American Military Adventure in Iraq, 2003 to 2005 by Thomas E. Ricks.  Serious endeavors require careful forethought and nuanced planning; and few enterprises are more serious than the business of war.  This controversial book looks at the missteps and mistakes of the American military as it justified, planned and executed the 2003 Iraq War.It contains haunting examples of how information can be distorted and obfuscated by both well-meaning individuals and those with insidious hidden agendas.  It also explains how the adoption of American corporate culture and leadership by PowerPoint lead to serious miscommunication and early failure.
	The Challenger Launch Decision: Risky Technology, Culture and Deviance at NASA by Diane Vaughan.  The Challenger explosion on January 28, 1986 changed the course of manned space flight forever.  But how did it happen?  What factors lead to it?  Might it have been prevented?In The Challenger Launch Decision, Diane Vaughan attempts to answer those questions.  In the process, she reveals that the Challenger explosion wasn’t the result of intentional wrongdoing but rather a slow-creeping definition of “normal” and comfort with the status quo.

Similar Posts:

• Hannibal, Napoleon, and Joseph Charles Minard
• Temples and Timelines
• Customizing LyX: Character Styles and the LyX Local Layout
• Back In Time (Part 1): Linux Backup Made Easy
• Anathem: Big Book, Big Ideas

Copywrite 2009: Rob Oakes. Apolitically Incorrect

Working With Words, Numbers and Images: A Reading List

When I graduated from college and had to choose between a career in industry or academics, I found it to be an easy decision: I stayed in academics.  I like to have my head in the clouds and enjoy the intellectual lifestyle.  (I actually consider the label of “absentminded” to be a compliment.)

It should come as no surprise, then, that one of the most enjoyable aspects of writing a book has been the opportunity to research my subject.  My reading list has included books on analytic design, illustration, anatomy, typesetting, scientific communication, web technologies, LaTeX, the history of science, statistics and informational graphics. And as I worked my way through it, I took some extremely interesting side trips.  One of the most intriguing, however, was an extended tangent through the notebooks of Leonardo Da Vinci.

Da Vinci died in the year 1519, nearly five hundred years ago.  Yet, the modern world remains fascinated by him.  His name adorns the side of best selling books and conspiracy fiction; and his drawings have become cultural icons.  As an example of his popularity:

In October of 2009, Martin Kemp, a professor of art and history at the University of Oxford, found a portrait of an Italian girl.  Up until Kemp took an interest, it was widely accepted that portrait had been painted sometime in the nineteenth century by an unknown artist.   After a great deal of investigation and the use of a multispectral camera, however, Kemp discovered something startling.  The painting had actually been done by Leonardo and nearly overnight, it went from a value of 19,000  British pounds to over 100 million.

I’m no different than the masses.  Leonardo fascinates me.  He had a very distinctive way of seeing the world and an engaging style.  Maybe that’s why it’s so easy to lose yourself in the details of his work.  Given my interest, a thorough study of Leonardo’s notebooks seemed only natural.

What I didn’t foresee, however, is that I would start to digitally collect his sketches; and in the past several months, I’ve put together a rather eclectic mix from across the internets.  Earlier today, I realized that the images might be of interest to others as well; thus, I’ve created a special online gallery for them.  It can be found under “Art and Photography” –> “The DaVinci Notebooks”  To get there more quickly, you could also just click here.

Enjoy!

Similar Posts:

• Working With Words, Numbers and Images: A Reading List
• About This Book Thing
• Greetings From a Horseman
• A Guy Like the Rest of Us
• How to Scrub a PC Hard Drive

Copywrite 2009: Rob Oakes. Apolitically Incorrect

Image Gallery: The Notebooks of Leonardo Da Vinci

When doing math or numerical analysis, the knowledge of the technique is far too often tied to the tool performing the calculation.  Consider an engineer whose understanding of the Fast Fourier transformation is inseparably tied to the fft function in Matlab.  Of course this hypothetical engineer understands what the results mean (more or less) but may not be able to duplicate his analysis if Matlab were taken away.

In most cases, it is likely that no deeper understanding will be required.  But what happens if the computer makes a mistake?  Or the program becomes unavailable?  Both situations are entirely possible.  Computer algorithms aren’t perfect and occasionally arrive at results make little sense; and hardware has been known to fail.

When the engineer understands how the computer arrived at the answer, however, he can recognize, understand, and ultimately correct those cases where the results are unexpected.  This is an important reality check that can prevent costly disasters later down the line.  Or, if the hardware is unavailable, he can use an alternative tool or software package to duplicate the analysis.

But while such a situation can arise with any type of numerical software, it’s most likely to happen to users of a statistical package.  I find this extremely ironic since a proper understanding of statistics is essential to live in the modern world.  (Much more so than an understanding of the Fast Fourier transform, at any rate.)  The rules of probability, the normal curve, correlation, and multivariate statistics can have a direct impact on how we live our lives.  They are used in making important decisions in finance, medicine, science and government.  A misunderstanding of stats and the methods of science (from which statistics is inseparable), underlies the most divisive issues of our day: abortion, stem cell research, and global warming.

Moreover, neither side has a monopoly on ignorance or misunderstanding.  People fail to distinguish between correlation and causality, or insist in using the word “average” as a slur.  Nearly as bad are those that – like the hypothetical engineer described above – only understand statistics within the narrow context of their stats package.  Casual statisticians are nearly as dangerous as the wholly uninformed.

The Statistical Package for the Social Sciences (SPSS), is one of the biggest perpetrators of this crisis.  Which is hugely ironic, because I happen to love SPSS.  SPSS is probably the first statistical package that has placed advanced statistical methods within the grasp of the novice user.  I’ve been a happy user for nearly a decade (ever since I was introduced to the program in high school).  But there is no doubt that I’ve come to understand statistics within the context of SPSS and its GUI.

Please don’t misunderstand me, I have a pretty good grasp of basic statistics.  I can sling probability with the best of them and take relish in describing when to use the Fischer Exact test instead of a Chi-Square; but advanced statistics are a completely different matter.  Advanced stats scare me.  I can certainly use these more complicated methods.  I’ve analyzed and written about multi-variate models and even ventured into Analysis of Variance (ANOVA).  But I have to rely on SPSS and the aid of my institution’s biostatistician to help me recognize when there is a problem.

Which is why, in a time of tight budgets, losing the institution’s SPSS license has been a crushing blow to my productivity.  (Whoever made that decision should be hauled out and shot!)  Because I don’t have my statistics software any more, there are certain aspects of my job that are much more difficult to do.  And unfortunately, there is only logical conclusion to draw: I’ve become a victim of the statistical ease of SPSS.

Open Source Alternatives

I went through a similar experience about a year ago.  At the time, I had become increasingly frustrated with the restrictions, licensing fees, and limitations of the Matlab technical computing language.  After one particularly infuriating meeting, I decided that I had had enough and was going to do something about it.  In the months that followed, I spoke with friends and colleagues, and experimented with every alternative I could get my hands on.  I looked at Octave (the “Open Source Matlab”) and Ruby, before eventually settling on a combination of Python and PyQt to meet my needs.  The result of these changes has been tremendously positive.  Python is both easier to use and far more powerful than Matlab could ever hope to be.  Not only am I happier and more productive, but so are those who work with me.

It is, therefore, logical that when I lost my statistical language of choice that I would look to open source to provide an alternative.  Fortunately, the Open Source community delivers not one alternative to SPSS, but two: Gnu PSPP and R.

Gnu PSPP

As the name implies, PSPP has one simple goal: to clone SPSS in every way that matters.  It can perform descriptive statistics, T-tests, linear regression and non-parametric tests.  It has an easy to easy to use and relatively intuitive GUI.  It can use SPSS syntax and read SPSS data files.  It supports an obscene number of variables and cases (about a billion).  It interoperates with Gnumeric and OpenOffice.  Finally, it’s fast.

Aside from its horribly ugly icon, PSPP would appear to deliver exactly what I want and need.  Except, you might have noticed that this article is titled “Statistics with R”, not “Statistics with PSPP”.  Obviously, I chose to go with the second alternative.  But why?

PSPP works as advertised.  I found it able to deal with nearly all of the old SPSS data files and syntax that I threw its way.  But, the program suffers from the problem of all clones everywhere: it’s greatest aspiration is to be a copy of something else.  That is to say, it seeks to be “Good Enough”,  and therein lies the problem, I don’t want a tool that is good enough.  I want to use excellent software, even if it’s different or requires me to learn new things.  Even if I have to pay for it.

I’m not trying to pick on or be unfair to PSPP.  It meets an important need in the free software landscape.  It just doesn’t fit in my with my desires or preferences very well.

The R Statistical Project

This is where R steps into the picture.  Whereas PSPP is “aimed at statisticians, social scientists and students requiring fast convenient analysis of sampled data (emphasis added)”, R is the software that most statisticians actually use.  When I contacted the statistician at my institution to ask, “What statistical software should I use?  I’m looking at R and PSPP.”

He responded, “Oh that’s easy.  Use R.  There will be a learning curve, but it’s much more powerful and capable than even SPSS or SAS.”

As I’ve started to explore the feature set and available modules, it readily becomes apparent as to why.  R is a huge language.  There are thousands of packages that cover every type of statistics I’ve ever heard of, and many more I haven’t.

Even better, people have gone to great lengths to incorporate R into other tools.  It has a set of excellent python bindings and interoperates very well with LyX and LaTeX.  As just a single example, using the Sweave document class, you can use R to easily embed code in reports and other documents that need to be updated on a very frequent basis.  This allows for these publications to be generated on demand with the most recent data.  The only other place I’ve seen the equal to this feature is within the proprietary universe of Microsoft Office and SQL Server.

Easing Into R

Indeed, if R can be said to have a major weakness, it would be that it is too full featured and capable. Particularly for someone who is a statistical novice.  The sheer number of packages and options available is absolutely overwhelming.  Moreover, the reference material is distributed in nature.  Like other open source tools, you can find answers to your questions; but you need to be intelligent about how you ask them.

As I’ve explored R, there have been quite a few painful moments.  This isn’t because R is more difficult than SPSS or SAS, but rather because it is tremendously different.  As an example, consider the differences in how the programs work with data.

Both SPSS and SAS use one main data structure, the data set.  A data set can be thought of as a big spreadsheet where the variable names are kept in columns and the individual observations are kept in rows (SPSS calls them cases).  In contrast, R uses may types of data structures.  It has an a two dimensional array that is similar to the data set, but it is possible to use one dimensional arrays of data (similar to vectors), or three dimensional arrays (which might contain extremely complex data).  The added options raise very complicated questions: What sorts of statistical calculations are done on a three dimensional arrays?  Why are they necessary?  How do I need to code my data so that I can take advantage of R’s advanced features?

Those aren’t issues that a user of SPSS or SAS even needs to consider.  But with R, they present themselves before you even begin to use the program; and there is no centralized source of information to help you figure out the answers.  The result of too many options and too little information is unproductive agony.

Series Introduction

Which is why I decided to write this series.  R does some spectacular things, it is excellent software.  But there are some things you need to know before using it.  Here are just a few examples:

• What user interfaces are available for R and which should you use?
• How can you use R to summarize data and do basic comparisons?
• How does R handle moderately advanced statistics like one-way ANOVA and non-parametric tests?  What about mult-variate statistics and regression analysis?
• How does R work with other programs?  How can you format your output into publication quality figures?
• What support does LyX and LaTeX offer for users of R?

The purposes of these articles are to address the concerns of the novice statistician or scientist.  I will try and avoid jargon and other indecipherable terms.  I will ensure that the examples are interesting and relevant.  But most importantly, I will try and to help build a deeper statistical foundation.  I know exactly what it feels like to be the “hypothetical” engineer who has become too reliant on his tools.  As long as the tool is nearby, you’re fine.  But when that tool is taken from you, be prepared for a world of hurt.

Until fairly recently, I’ve been in that world of hurt.  You might just say that these articles are my way of explaining how I got out.

Similar Posts:

• Back In Time (Part 1): Linux Backup Made Easy
• Install Time Drive On Ubuntu and Other Debian Linux Distributions
• Installation of PyQt on Mac OS X
• The iPhone App Store and Software Quality: A User’s Perspective
• Create a Unified Inbox in Gnome Evolution

Copywrite 2009: Rob Oakes. Apolitically Incorrect

Statistics With R – Part 1: An Old Dog Learns New Computing Tricks

After nearly 18 months of work, my research group finally published our big study!  It should appear in the journal of the American Heart Association, “Circulation” on April 7, 2009.  This also means that I can finally talk about what I’ve been doing for the last long while.

Atrial fibrillation (AF) is the most common – and perhaps most insidious – form of heart arrhythmia (a change in the normal electrical patterns of the heart) in existence.  It affects millions of people and while it isn’t life-threatening in its early stages, it eventually leads to stroke or heart attack in many of those people who are afflicted with it.

Unfortunately, the current ways of treating AF are completely inadequate.  The class of medication used for treatment (anti-arrhythmics) often cause more problems than they prevent, and interventional treatment is still highly experimental.  It is, therefore, all but a guarantee that the AF will become more serious over time.  Patients cannot be as physically active as they used to be and eventually must adjust to the symptoms of the disease.

A great part of the reason why AF is so difficult to treat is that we lack a good understanding of what causes it.  About fifteen years ago, some researchers in France recognized that the random spots of electrical activity within the pulmonary veins (which return blood to the heart from the lungs) were a likely source of the arrhythmia.  As a result, there is an entire arm of interventional treatment designed to destroy these spots and isolate the pulmonary veins so that bad electrical signals are unable to influence the heart as a whole.  While this is effective in some people, it doesn’t work for everyone.

More recently, other researchers found that AF actually changes the underlying tissue of the heart (a process known as remodeling), which results in other electrical and mechanical adaptations.  These changes make the arrhythmia more serious as well as difficult to treat.  Even though the changes are a sign that the person has a more advanced form the arrhythmia, the only way to ascertain the degree of change is to invasively measure electrical changes from the inside surface of the heart.

That is, until about a year ago.  In the Circulation paper, we describe a method to determine how much the heart has remodeled by using MRI.  We also showed that the degree of change is the single best indicator of how the patient will respond to treatment.  While this may seem obvious, it isn’t something that could be measured easily or safely before.  Now, we have a tool which allows us to see how the heart changes over time in response to AF.  This will lead to a better understanding of the disease, and provide insight in how to treat it.

The image below shows an example of just how good this detection can be.  The first column of images (A) shows a three dimensional model of the human left atrium.  In the top row, we are looking at the back (or posterior) view of the left atrium.  The pointy bits projecting to the left and right are the pulmonary veins.  In the bottom row, we are looking at the atrial septum – part of the wall that divides the heart into left and right sides.  The really interesting information is in the second column (B) which shows a 3D model where two different types of tissue have been identified.  Healthy tissue (shown in blue) and diseased tissue (shown in green).

This particular patient has some diseased tissue, though overall the heart looks fairly healthy.  What is striking, however, is how well this diseased tissue compares to the measurements made by invasive mapping (taken during a catheter intervention).  On the back side (top) we can see a donut of unhealthy tissue, and on the right side (in the septum), we can see a hook of unhealthy tissue.  The location and shapes of these tissue are mirrored on the invasive maps (C).  And all of this can be determined without exposing a patient to radiation or needing to cut them open!

These types of MRI techniques open a whole new branch of research that can be done on the heart that has important implications for more than AF research.  The remodeling and changes observed in AF can be seen in atrial tachycardia and even some types of ventricular arrhythmias.  Moreover, it appears to be related to other conditions, like coronary artery disease or ischemia (from smoking, for example).  More on that later, though.  If interested, you can find the complete text of the Circulation article here.

Similar Posts:

• AF and Smoking
• Atrial Fibrillation Study Published
• On the Surface Versus Working Deep
• Why Bother With a Personal Website?
• Forgotten Places – Sewell, Chile

Copywrite 2009: Rob Oakes. Apolitically Incorrect

Heart Arrhythmias and MRI

I am happy to note that the first of my papers with the Atrial Fibrillation Group at the University of Utah was published!  The full text can be found here.  While I know that I have an obvious bias, the paper is still very important.  It describes the University of Utah's methodology for visualizing tissue damage following the ablation.  Further, it lays the groundwork for the next studies: the detection of low voltage tissue prior to ablation, and the determination of what ablation parameters will result in the induction of scar.  All in all, a nice little paper.

As an add on interest, we found that the degree of enhancement (or damage) seemed to relate to how well people did following the procedure.  There are a lot of theories flying around as to why this might be the case.  My own personal theory is that targeted ablation induces a change in diseased tissue.  I was also happy to notice that we aren't the only people who see it as such.  Our study was of sufficient interest that it was accompanied by editorial comment, as a highlighted article.  Not bad, if I might be allowed a smug moment.

For those that care about such things, here is the citation:

McGann CJ, Kholmovski EG, Oakes, RS, Blauer JJ, Daccarett M, Segerson N, Airey KJ, Akoum N, Fish E, Badger TJ, DiBella EV, Parker D, MacLeod RS, Marrouche NF.  New magnetic resonance imaging-based method for defining the extent of left atrial wall injury after ablation of atrial fibrillation. J Am Coll Cardiol.  2008 Oct 7; 52(15): 1263-71.  PMID: 18926331.

If you'd like to take a look at the full article, you can get the accepted draft from the Science and Technology page.

Similar Posts:

• AF and Smoking
• Heart Arrhythmias and MRI
• Big Things, Little Things and Unassociated Thoughts
• Mac OS X doesn’t need to be defragmented? Yeah, right.
• Some Alternatives to MobileMe

Copywrite 2009: Rob Oakes. Apolitically Incorrect

Atrial Fibrillation Study Published

Several weeks ago, while working on the revisions for a larger study, I noticed an important trend. Patients who suffer a recurrence of atrial fibrillation following catheter ablation quite often have a history of smoking. "That's interesting," I thought at the time. I then successfully stopped thinking about it, so that I could think about other important things. The ability of men to focus only on the task at hand being what it is. While spending time at Dr. Marrouche's party last night, I ran into Brent Hill. Brent is the research coordinator for the Division of Cardiology. While talking, I mentioned the finding. He got quite excited about it. So excited, in fact, that he thought about co-opting the idea for his thesis. Some additional ideas that he shared:

1. By what mechanism does smoking cause a recurrence of AF following the procedure? Might it be due to smoking induced ischemia?
2. Is the effect mitigated by the number of cigarettes which the person smokes per day?
3. Would offering smoking cessation materials help to increase the effectiveness of the ablation procedure?

Interesting things to think about! I think the next step is to formulate a number of specific hypotheses and draw up a formal proposal. At that point, it would be interesting to approach Marrouche and get his take on it.

Update

Though it may have been interesting to further think about and develop a map of how AF and smoking relate, I do not think that will ever be a possibility; unfortunately. Due to a number of personal reasons, I have decided that it is time for me to leave the AF group. I have greatly enjoyed the time that I have been able to work with the people in the group, but it is possible that my future lies somewhere outside of medicine. I will be posting a "Swan Song" shortly which describes my thoughts of how the various projects interconnect, but that will be my last thoughts on atrial fibrillation and its treatment.

Similar Posts:

• Atrial Fibrillation Study Published
• Heart Arrhythmias and MRI
• Big Things, Little Things and Unassociated Thoughts
• The iPhone App Store and Software Quality: A User’s Perspective
• Time Drive 0.1.5: Incremental Changes

Copywrite 2009: Rob Oakes. Apolitically Incorrect

AF and Smoking