No study of the history of scientific communication can be complete without mention of Joseph Charles Minard, a 19th Century French civil engineer and cartographer.

At the end of his life, Minard created two very famous examples of statistical charts, called flow maps, that every scientist, engineer and student should be familair with.  The first showed Hannibal’s crossing of the Alps (218 BC, Second Punic War), and the second describes Napoleon’s disastrous invasion of Russia (1812-1813).

Both examples are beautiful works of art and masterful examples of evidence.  But they are also more than that, they tell cohesive and interesting stories.  In this post, I thought it might be interesting to take a closer look at the history of Hannibal and Napoleon, and highlight the ways which Minard’s charts help us to explain their eventual outcome.

(Note: High resolution, PDF versions of the two maps are available for download.  These versions have been translated from the original French.  To download, either click on the images, or here for the Hannibal invasion of Northern Italy, and here for the French Invasion of Russia.)

Hannibal Invasion of Italy (218 BC, Second Punic War)

Hannibal was perhaps the single greatest threat to Roman power to ever live.  At the same time that Roman legions marched unopposed through most of the Mediterranean world, Hannibal lead a force of twenty-five thousand soldiers into the heart of Roman territory. He then spent the next fifteen years occupying portions of the Italian peninsula and generally wreaking havoc.  But even though he ravaged the Roman countryside, Hannibal was never successful in subduing Rome.  Ironically, this might be due to one of his greatest exploits: the overland crossing of the Alps.

Hannibal departed his home base in Spain with nearly 90,000 troops (one of the largest armies that had ever been fielded at that point in history) and had a number of early successes.  He marched north over the Pyrenee mountains and into Gaul (modern day France).  He subdued the Celtic tribes and crossed the Rhone River before the Romans could take measures to stop his advanced.  He forged important alliances and found unexpected friends in his fight against the Romans.

After crossing the Rhone, however, Hannibal’s luck began to change.  He received word that a large Roman force was marching from the Mediterranean coast to intercept and destroy him.  As the force outnumbered him by a significant margin, Hannibal was faced with a difficult choice.  Should he stay at his current location and engage the Roman army or retreat and prepare a defense?

Not content with either choice, Hannibal ultimately chose a third option; which, ironically, would have enormous consequences for his overall campaign.  To evade the Romans, Hannibal took the unprecedented action of turning inland and marching over the Alps.  (It is thought that he probably crossed over the valley of the Drome and south of the Col de Montgenevre or near the Col de Mont Cenis.)  This had never been done before – indeed many thought it impossible – and has long been praised as a brilliant tactical decision.

Minard’s reconstruction of Hannibal’s journey shows the journey from Spain, through Transalpine Gaul and eventual arrival in Italy, allows it to be seen a slightly different light, however.  Through use of a flow line, Minard shows how the strength of Hannibal’s army waned through the march, with a disastrous loss of twenty thousand men while passing over the Alps.  This steady loss of strength, subtly reinforced by Minard’s map, helps to explain why Hannibal failed to subdue Rome.

Hannibal’s brilliant tactical decision was also a strategical disaster.  It came at a dramatic loss of men, war elephants, and material with the most disastrous loss being the destruction of his siege engines.  Without the siege engines, Hannibal was never able to successfully attack Rome.  And eventually the Romans realized this and changed their tactics.  Instead of engaging Hannibal directly, they instead fought a long-pitched war of attrition and eventually destroyed the one resource he couldn’t replace: his men.  Thus, even though crossing the Alps helped Hannibal to win a few stunning victories, the decision eventually caused him to lose the war.

Napoleon’s Invasion of Russia (1812-1813)

Like in the case of the Hannibal, Minard’s depiction of the 1812-1813 invasion of Russia also tells an extraordinary story.  It combines information about the size of the army, geography, sub-campaigns and temperature into a single narrative.  This time, though, it is anything but subtle.

Note: Because of  the elegant management of all this information – at any time showing the army’s direction, size, and loss relative to the start – some statisticians and data visualization experts (such as Edward Tufte, Etieene Jules Marey and Howard Wainer) have declared Minard’s depiction of Napoleon’s Invasion of Russia to be the greatest numerical graphic ever created.

From the very beginning, the army met with tremendous hardship.  The cold temperatures and lack of available resources took an enormous toll.  Napoleon had planned on scavenging for supplies to support his massive army, but the Russians had destroyed all of their crops and burned their villages so that Napoleon wouldn’t be able to make use of them.  As a result, the army was heavily reliant on slow supply trains from Europe that were unable to keep pace with its quick advance.  Hundreds of thousands died from starvation and exposure, greatly outnumbering those lost in combat.

Minard’s map shows this trend of devastating loss.  Napoleon’s Grand Army starts out as a wide, brown line to the left of the map.  As the army progresses across Russia, the number of men decreases dramatically.   A few ribbons of men are dispatched to other targets, but by far the majority succumb to the elements.

Eventually, a substantially reduced force (100,000 men, about 20% of those who began the campaign) arrived in Moscow.  What they found did little to improve their circumstances.  The city had been abandoned and burned to the ground.  No food was available.  The only available option was to return to France.

Demoralized and broken, the Grand Army began a slow retreat to the west.  Their progress is shown in the black.  As they crossed the vast Russian wilderness, the temperature turned bitter cold (shown in the bottom third of the map).  When the army arrived at the Niemen river, there were a scant 10,000 troops.  More than 400,000 had died.

Such a devastating loss of life crippled Napoleon’s young empire and left him open to invasion by his enemies.  He was defeated later in 1813 and exiled to the island of Elba.

Conclusion

Minard’s graphics are successful because they demonstrate powerful trends and place evidence within its proper context.  In the case of Hannibal, it shows how he began in a position of strength and then undermined himself through a brilliant tactical innovation.  In the case of Napoleon, it shows how distance and temperature ravaged an army and destroyed an emperor.

The clear evidence and thoughtful presentation define the story, they show where things are headed and help to foreshadow the inevitable conclusion.  For this reason, they are powerful examples of clear communication.

____________________________________________________

Unrelated Thoughts: Re-Visioning Minard with Open Source Tools

Since first seeing these charts as part of an undergraduate statistics course, I’ve been obsessed with them.  I even have a poster sized copy of the Russian invasion (available from Graphics Press) hanging in my office.  Given this obsession, it should come as no surprise that I’ve wanted to include the maps in my book.

There is, however, just one problem; there is not anything new to say about them.  Edward Tufte has held up Minard as a “Da Vinci of Data” for the better part of thirty years.  Nearly every one of his books includes a copy of the maps (in the case of Beautiful Evidence, his most recent work, there is an entire chapter devoted to the graphics; including eight different reproductions).  Other than, “They’re beautiful, study them carefully, read Tufte’s analysis,” there just isn’t that much to say.

That is, until I started looking for examples of scientific illustration that could be reproduced using the vector illustration program, Inkscape. Any such example should have simple lines and clean integration of text and images.  I'm not sure that a better description of Minard's work exists.

The two images above (and attendant PDFs) were my first pass at re-drawing the maps.  And I have to say, I’m rather happy with the result.  For the most part, I was able to get a very faithful reproduction (with the exception of the type face, which I deliberately changed because I hate script fonts), and only required a couple of hours worth of work.

I think that I may have found my book examples.  It allows me to include one of the best statistical charts of all time, talk about flow maps, and even include some interesting how-tos which show how the graphics can be built in an automated fashion.  Which, come to think of it, might just count as a new contribution after all.

Acknowledgements and Further Reading

More information on the book can be found at the publisher’s website.

The source images for the two maps included “Re-Visions of Minard” by Michael Friendly (1999) and the reproductions in “Beautiful Evidence” by Edward Tufte.  Translation for the Hannibal map was adapted from those available on Edward Tufte’s website (Dawn Finley) and sources available at Re-Visions of Minard.  Translation for the Russian campaign map were taken from the reproduction in Edward Tufte’s “Beautiful Evidence”, Wikimedia Commons and sources available from Re-Visions of Minard.

Additionally, the Re-Visions of Minard Website include a number of interesting adaptations of the Napoleon graphic that might be of interest, including both Matlab and GGPlot2 reconstructions.

Similar Posts:

• Anathem: Big Book, Big Ideas
• Customizing LyX: Character Styles and the LyX Local Layout
• Heart Arrhythmias and MRI
• WPF – SVG Graphics and XAML – Part 3
• Typeset Your Curriculum Vitae – Part 1: The xetexCV Document Class

Copywrite 2009: Rob Oakes. Apolitically Incorrect

Hannibal, Napoleon, and Joseph Charles Minard

In addition to nouns, verbs, and adjectives; there are other tools which supplement the writer's arsenal. some writers, this is a moleskin notebook and ballpoint pen. For others, writing is synonymous with the usage of computer word processors, such as Microsoft Word.  What is a given, however, is that the choice is intensely personal. If you ask ten different writers, "What is the perfect tool?" Expect ten different answers.

For those who create software, however, this is big problem. Programs can't be customized to the individual whims of every writer. (A more diverse or idiosyncratic group does not exist.)  It is necessary to select a cross section of features that meet the needs of most people.  Unfortunately, however, this has resulted in some serious compromises and an emphasis on the technical process of writing, often to the exclusion of the creative process.

The Problem

Like most writers, I have some strong feelings about the tools I use; and like most writers, I have never found one that fits my needs.  But this is not for a lack of trying.  In fact, I’ve been searching “perfect tool” for a very long time, at least since I was a senior in high school.

At that time, I enjoyed writing things out in longhand, with separate drafts copied over from the first iteration. I fancied myself a budding novelist and fervently believed that such an exercise would stimulate my creativity.

A decade later, however, I have become impatient. Now, I prefer to peck out my thoughts at the keyboard.  Sometime in college, I realized that writing by hand is slow and while it can focus the mind on word choice, it's a plodding and frustrating exercise. In contrast, typing at the computer is quick; and since I can type nearly as fast as I can think, it allows the text to keep pace with my thoughts.

Moreover, working on the computer is organic. There isn't a discrete first, second, and final draft; instead, I can add, modify or change my writing as I form the ideas. If I want to jump back and change a word or sentence, I do so. Pen and paper isn't nearly so adaptable and many ideas are lost before they can be captured and expanded.

The Limitations of Existing Tools

But even though I prefer to write at the computer, I haven’t been able to find a writing program that lets me creatively adapt the structure and flow of my document.  I have tried nearly everything available, but nothing fully meets my needs.

Consider Microsoft Word, for example.  Even with all of its power, there are a number of things that it doesn't handle very well. Ever tried to work with a complex Word document, numbering into the hundreds or thousands of pages? If not, I don't recommend that you do.  Large Word documents cause the program to slow down or crash and they have a tendency to become corrupted.  There is nothing more disconcerting than finding out one of your book chapters is no longer usable due to bit rot.

Introducing LyX: The Document Processor

Luckily, there may a potential solution.  The open source document processor, LyX, meets many of Word’s shortcomings.  LyX encourages people to approach writing based on the structure and content of the document, rather than the appearance. Thus, it isn't a "What You See Is What You Get" so much as a "What You See Is What You Mean" (WYSIWYM).

When used right, LyX focuses the user on writing the content. The program handles the mechanics of layout and output. To do so, LyX leverages the LaTeX typesetting system, which is used heavily in the engineering and science disciplines to publish dissertations, grant proposals, journal articles, and books.

When I first found LyX three or four years ago, it fundamentally changed the way that I worked. Before, I manually formatted everything in a given document. While this approach is okay for simple letters or lists, it quickly breaks down in anything longer than a few pages.

In contrast, LyX makes extensive use of tags that tell the program what a given piece of text is. The title is enclosed by a "title" tag, book sections by the "section" tag, chapter headings by the "chapter" tag, block quotes by the "quote" tag, and paragraph text by the "body" tag. The program then applies formatting rules globally and in a consistent manner. Text within LyX has a meaning and will be processed in a specific way.

And thus we have the first important component of a perfect writing environment: the program is aware of what content is. Such awareness allows for extremely complicated written pieces to be assembled in a relatively easy fashion.

A Sloppy Process

Despite its strengths, though, LyX is an incomplete solution. It focuses on the mechanical aspects of writing and ignores what is arguably the more important component: the creative process. Using LyX, I can assemble a true monstrosity of a manuscript replete with footnotes, endnotes, references, tables, lists of figures, indices, appendices, and all of the other components of something unreadable. Yet such elements are mere after-thoughts rather than the heart and soul of the work. The center is all about ideas.

Which brings me to the serious limitations inherent in most word processing programs: joining ideas together is a sloppy process and they simply don’t facilitate the sloppiness.

Every writer is creative in a different way. I personally start by making a miscellaneous brain dump. Everything relevant to the topic is listed out and then I start to type additional thoughts which seem relevant. After a while, I run out of steam which prompts research, reference collection, reference reading and chicken scratch notes.  Then I groan, swear a little, go for a walk, and kick ideas around in my head.

When I come back to the computer, I move things around even further and excise any bits that now seem irrelevant (carefully saving them for later). Usually by this point, I have probably started over a few times and hopefully clarified my goals.  If the process works, I get a slightly better fix on direction I intend to take and, eventually, a first draft magically seems to appear.  That is when the real fun starts.

Unfortunately, LyX was designed to work linearly. You start at the beginning and continue to the end. All of the words and ideas are supposed to come marching out like perfect little soldiers on parade day. Using such a tool can be extremely frustrating to someone who doesn't think in a linear fashion: I want the freedom to be intuitive!

And in the name of freedom, I will probably use of four or five programs spanning two operating systems: OneNote for data collection, KDissert or FreeMind for MindMapping, and Word or LyX for the actual writing. Then I go back and update my files in the other programs.

There should be a better way! I want to use the same tool from beginning to end, not continually move between the word processor, the outliner, the mind mapper, and the note taker.

A Potential Solution: Scrivener

There just happens to be a program on Mac OS X that was apparently made for scattered people: Scrivener. Scrivener employs a number of simple (yet profoundly powerful) metaphors that let you manage the anarchy of creativity. These include:

1. An intuitive and simple map of the project (similar to Word's Outline View).
2. A corkboard that lets you create any number of "index cards" with a title and short summary of the card's contents.
3. An outline view where the entire structure of your document can be seen and manipulated through drag and drop.

Scrivener leverages its corkboard metaphor and outliner by linking the "index card" to a real piece of text. Thus, the index card actually represents the physical chunk of text in the underlying draft. The index cards can be moved, nested and set into folders and any changes made to the index cards are then instantly applied to the overall structure of the document. Pretty cool, huh? A given index card can represent a section, paragraph, chapter, or sentence. Even better is that cards can be split or joined together easily.

The practical result is that the entire draft is instantly fluid. You can experiment with new ideas, split away sections of text for later use, and re-arrange things as inspiration strikes you. It substantially enhances the organic process already provided by the computer. I know a surprising number of people who have moved to the Mac platform just so that they can use Scrivener.

While other programs (like Semantik, formally known as KDissert) use similar metaphors, the use of a "summary" field is what makes Scrivener stand apart. The summary begins life as an outline point and later becomes the synopsis of the section. It can keep notes, goals, or thoughts in addition to a regurgitation of what is in the text.  And no other program uses anything like it. Therefore, while you can replicate many of Scrivener's best features through Word or LyX's outline view, you are stuck manipulating the entire text (which becomes impractical and unwieldy very quickly).

The Outliner. Scrivener includes a high quality outliner that allows individual elements to be repositioned via drag and drop. Each outline item is connected to the text in the actual draft; therefore, changes in the outline view are immediately reflected in the structure of the document. Scrivener's outliner is particularly effective via the inclusion of a "synopsis" field that contains a summary of the text. This summary field isn't part of the full draft, and as a result, can begin life as an outline point that reflects the goal of the section. In contrast, the outline view in Microsoft Word uses the text of the actual document. In long or complicated documents, it is a slightly less than effective way to add new ideas.

The Corkboard. As an alternative to the outliner, Scrivener also provides a "corkboard" where parts of the document can be moved around on index cards. Each card has a title and summary that is is linked to a block of text in the document. Unlike the outliner, however, the corkboard only deals with one navigational element of the document. When at the top level, the cards might represent the parts or chapters of a book. When working on the corkboard of a book chapter, they might represent sections or individual scenes. There is no limit to the number of cards or nesting that can be done.

Scrivener's Singular Strength

Though Scrivener handles the creative process beautifully, it isn't a serious word or document processor. On the program's website, the primary developer states that Scrivener is a "personal writing tool." As a result, it seems uniquely geared to creative writing; that is to say, novels and screenplays. Not many novels or screenplays have a need for a bibliography, index, appendix, footnotes, or endnotes. As a result, Scrivener cannot handle the minutiae of complicated documents.

Just forget support for hyperlinks, cross-links, cross-references, figures, legends or graphics. They're outside the developer's stated need and are unlikely to appear. That's too bad, because it makes Scrivener worthless to those who need support for these features. Microsoft Word has them and LyX has them, OpenOffice pretends that it has them. However, Word, LyX, OpenOffice and other word processors completely lack the creative tools that make Scrivener a joy to use.

A Glorified Text Editor. The weakest component of Scrivener is the actual word processor. Advanced (and even not so advanced) features like footnotes, endnotes, cross references of links are completely unsupported. It is simply unsuited to writing a complicated document like an article or cross-referenced book.

A Combined Approach

The simplicity of Scrivener's word processor causes a semi-crisis. If given my choice, I would compose nearly everything in it for the convenience of its outliner and index cards. However, it isn't up to supporting the type of heavy lifting I am often asked to do. While I could use it as my creative crutch, that worsens the overall problem. Instead of four programs across two platforms (Window/Linux), I would be using five programs across three platforms; not what I'm aiming for. I want to simplify my work process, not make it more complicated.

I've tried to find what I want in the software of others and determined that it doesn't exist. The fallback is to develop it myself by turning to opensource solutions. Since LyX already meets my needs as a document processor, it is a great place to start. As a front-end to LaTeX, it automatically supports the advanced features I need. It is cross-platform, which means that the writing environment would be the same regardless of whether I am working on Windows, Mac OS X, or Linux. Additionally, it has deep integration with revision control systems, which greatly simplify collaboration. Last, it is available under the GNU public license, which means that I can access the source code. LyX combined with the same creative tools found in Scrivener would be nearly perfect.

Almost There: LyX's Existing Features

After a careful review of LyX, it becomes obvious that much of the functionality required for a robust outliner and corkboard is already present. At left is the outline panel in LyX. The outline is automatically generated from the heading information in a given document and any child documents. The particular example shown here is series of three essays kept in four different files: a master document and three child documents.

LyX already recognizes blocks of text and understands their contextual meaning. In fact, you can already specify the order of any element and what position it holds. Further, individual text elements can be moved from one child document to another. In other words, nearly all of the background elements for a Scrivener-like outliner or corkboard are already present.

Form and Function

What is missing is the ability to assign a summary or "synopsis" to a given chunk of text. This would ideally exist as associated meta-data and affiliated with a particular tag. Additionally, it might prove convenient to associate other meta-data in addition to the summary.

To add the "creative elements,"  most development could focus on creating a user interface that  blends the existing outline view with the functional equivalents of the corkboard and outliner from Scrivener. Convenient drag and drop of document sections from within the document map, outliner, or corkboard would allow text to be moved and positioned. The outliner would show the complete document with the option to expand or collapse a given section. The corkboard, in contrast, would show only the elements contained in one particular outline node.

The figures below show an example of what such a user-interfaces might look like. The first figure shows the outliner and the second shows the corkboard. (Though the index cards and outline points of the example correspond to the chapters and sections of the document map, this should not necessarily be required in an ideal implementation. Instead, the user would be able to determine which index cards correspond to sections and which correspond to a specific piece of evidence, argument, or narrative. That is to say, the user should be allowed the freedom to create the outline structure, rather than have it imposed.)

Proposal

I propose to create an add-on for LyX which would combine the best features of Scrivener with the already powerful capabilities of LyX. As described above, this would consist of two inter-related tools:

1) The Outliner. The outliner would be an expandable panel where the structure and rough content of a document could be laid out prior to beginning the first draft. Specific features of the Outliner would include:

a. Each outline point would have room for a title, synopsis and type of content (drawn from the list provided by the LaTeX class).

b. Individual nests in the outline can be expanded or collapsed.

c. Elements can be dragged and dropped from one point to another. The underlying document will be updated to reflect any changes.

d. Though most points in the outline might correspond to headings (e.g. book parts, chapters, sections, or subsections), this would not be required. Instead, any block of text could be assigned to a point in the outline. It would behave in precisely the same manner as other outline elements.

2) The Corkboard. Housed in the same panel as the Outliner would also be a digital Corkboard which would display a small "Index card" with the title of an outline element in addition to the synopsis. However, in contrast to the outline, only the individual elements for a selected point in the document map would be shown. In the example below, the sections of a specific chapter, "Eragon Shadeslayer: Sociopath" are shown with their corresponding summaries. Changes in the order of the cards would be reflected in the underlying draft.

Conclusion

The proposed additions to LyX would further expand its other excellent features, making it the only document/word processor with a powerful set of creative tools. By associating blocks of text with additional meta-data (like a synopsis), working with the document structure would become a visual and interactive process, thereby allowing for much greater flexibility. Indeed, such changes would bring LyX one step closer to writing perfection.

Note: A PDF copy of this article can be found here.

Similar Posts:

• Introducing LyX-Outline 0.1
• Typeset Your Curriculum Vitae – Part 2: Extending and Customizing an Existing Document Class
• On Writing – Software that cuts out distraction and clutter
• WPF – SVG Graphics and XAML – Part 2
• Customizing LyX: Create an NIH Grant Proposal Template

Copywrite 2009: Rob Oakes. Apolitically Incorrect

Creating the Perfect Writing Tool: A Proposal