Apolitically Incorrect » Writing and Literature

Hannibal, Napoleon, and Joseph Charles Minard

Rob Oakes — Mon, 22 Feb 2010 21:49:11 +0000

No study of the history of scientific communication can be complete without mention of Joseph Charles Minard, a 19th Century French civil engineer and cartographer.

At the end of his life, Minard created two very famous examples of statistical charts, called flow maps, that every scientist, engineer and student should be familair with. The first showed Hannibal’s crossing of the Alps (218 BC, Second Punic War), and the second describes Napoleon’s disastrous invasion of Russia (1812-1813).

Both examples are beautiful works of art and masterful examples of evidence. But they are also more than that, they tell cohesive and interesting stories. In this post, I thought it might be interesting to take a closer look at the history of Hannibal and Napoleon, and highlight the ways which Minard’s charts help us to explain their eventual outcome.

(Note: High resolution, PDF versions of the two maps are available for download. These versions have been translated from the original French. To download, either click on the images, or here for the Hannibal invasion of Northern Italy, and here for the French Invasion of Russia.)

Hannibal Invasion of Italy (218 BC, Second Punic War)

Hannibal was perhaps the single greatest threat to Roman power to ever live. At the same time that Roman legions marched unopposed through most of the Mediterranean world, Hannibal lead a force of twenty-five thousand soldiers into the heart of Roman territory. He then spent the next fifteen years occupying portions of the Italian peninsula and generally wreaking havoc. But even though he ravaged the Roman countryside, Hannibal was never successful in subduing Rome. Ironically, this might be due to one of his greatest exploits: the overland crossing of the Alps.

Hannibal departed his home base in Spain with nearly 90,000 troops (one of the largest armies that had ever been fielded at that point in history) and had a number of early successes. He marched north over the Pyrenee mountains and into Gaul (modern day France). He subdued the Celtic tribes and crossed the Rhone River before the Romans could take measures to stop his advanced. He forged important alliances and found unexpected friends in his fight against the Romans.

After crossing the Rhone, however, Hannibal’s luck began to change. He received word that a large Roman force was marching from the Mediterranean coast to intercept and destroy him. As the force outnumbered him by a significant margin, Hannibal was faced with a difficult choice. Should he stay at his current location and engage the Roman army or retreat and prepare a defense?

Not content with either choice, Hannibal ultimately chose a third option; which, ironically, would have enormous consequences for his overall campaign. To evade the Romans, Hannibal took the unprecedented action of turning inland and marching over the Alps. (It is thought that he probably crossed over the valley of the Drome and south of the Col de Montgenevre or near the Col de Mont Cenis.) This had never been done before – indeed many thought it impossible – and has long been praised as a brilliant tactical decision.

Minard’s reconstruction of Hannibal’s journey shows the journey from Spain, through Transalpine Gaul and eventual arrival in Italy, allows it to be seen a slightly different light, however. Through use of a flow line, Minard shows how the strength of Hannibal’s army waned through the march, with a disastrous loss of twenty thousand men while passing over the Alps. This steady loss of strength, subtly reinforced by Minard’s map, helps to explain why Hannibal failed to subdue Rome.

Hannibal’s brilliant tactical decision was also a strategical disaster. It came at a dramatic loss of men, war elephants, and material with the most disastrous loss being the destruction of his siege engines. Without the siege engines, Hannibal was never able to successfully attack Rome. And eventually the Romans realized this and changed their tactics. Instead of engaging Hannibal directly, they instead fought a long-pitched war of attrition and eventually destroyed the one resource he couldn’t replace: his men. Thus, even though crossing the Alps helped Hannibal to win a few stunning victories, the decision eventually caused him to lose the war.

Napoleon’s Invasion of Russia (1812-1813)

Like in the case of the Hannibal, Minard’s depiction of the 1812-1813 invasion of Russia also tells an extraordinary story. It combines information about the size of the army, geography, sub-campaigns and temperature into a single narrative. This time, though, it is anything but subtle.

Note: Because of the elegant management of all this information – at any time showing the army’s direction, size, and loss relative to the start – some statisticians and data visualization experts (such as Edward Tufte, Etieene Jules Marey and Howard Wainer) have declared Minard’s depiction of Napoleon’s Invasion of Russia to be the greatest numerical graphic ever created.

From the very beginning, the army met with tremendous hardship. The cold temperatures and lack of available resources took an enormous toll. Napoleon had planned on scavenging for supplies to support his massive army, but the Russians had destroyed all of their crops and burned their villages so that Napoleon wouldn’t be able to make use of them. As a result, the army was heavily reliant on slow supply trains from Europe that were unable to keep pace with its quick advance. Hundreds of thousands died from starvation and exposure, greatly outnumbering those lost in combat.

Minard’s map shows this trend of devastating loss. Napoleon’s Grand Army starts out as a wide, brown line to the left of the map. As the army progresses across Russia, the number of men decreases dramatically. A few ribbons of men are dispatched to other targets, but by far the majority succumb to the elements.

Eventually, a substantially reduced force (100,000 men, about 20% of those who began the campaign) arrived in Moscow. What they found did little to improve their circumstances. The city had been abandoned and burned to the ground. No food was available. The only available option was to return to France.

Demoralized and broken, the Grand Army began a slow retreat to the west. Their progress is shown in the black. As they crossed the vast Russian wilderness, the temperature turned bitter cold (shown in the bottom third of the map). When the army arrived at the Niemen river, there were a scant 10,000 troops. More than 400,000 had died.

Such a devastating loss of life crippled Napoleon’s young empire and left him open to invasion by his enemies. He was defeated later in 1813 and exiled to the island of Elba.

Conclusion

Minard’s graphics are successful because they demonstrate powerful trends and place evidence within its proper context. In the case of Hannibal, it shows how he began in a position of strength and then undermined himself through a brilliant tactical innovation. In the case of Napoleon, it shows how distance and temperature ravaged an army and destroyed an emperor.

The clear evidence and thoughtful presentation define the story, they show where things are headed and help to foreshadow the inevitable conclusion. For this reason, they are powerful examples of clear communication.

____________________________________________________

Unrelated Thoughts: Re-Visioning Minard with Open Source Tools

Since first seeing these charts as part of an undergraduate statistics course, I’ve been obsessed with them. I even have a poster sized copy of the Russian invasion (available from Graphics Press) hanging in my office. Given this obsession, it should come as no surprise that I’ve wanted to include the maps in my book.

There is, however, just one problem; there is not anything new to say about them. Edward Tufte has held up Minard as a “Da Vinci of Data” for the better part of thirty years. Nearly every one of his books includes a copy of the maps (in the case of Beautiful Evidence, his most recent work, there is an entire chapter devoted to the graphics; including eight different reproductions). Other than, “They’re beautiful, study them carefully, read Tufte’s analysis,” there just isn’t that much to say.

That is, until I started looking for examples of scientific illustration that could be reproduced using the vector illustration program, Inkscape. Any such example should have simple lines and clean integration of text and images. I’m not sure that a better description of Minard’s work exists.

The two images above (and attendant PDFs) were my first pass at re-drawing the maps. And I have to say, I’m rather happy with the result. For the most part, I was able to get a very faithful reproduction (with the exception of the type face, which I deliberately changed because I hate script fonts), and only required a couple of hours worth of work.

I think that I may have found my book examples. It allows me to include one of the best statistical charts of all time, talk about flow maps, and even include some interesting how-tos which show how the graphics can be built in an automated fashion. Which, come to think of it, might just count as a new contribution after all.

Acknowledgements and Further Reading

More information on the book can be found at the publisher’s website.

The source images for the two maps included “Re-Visions of Minard” by Michael Friendly (1999) and the reproductions in “Beautiful Evidence” by Edward Tufte. Translation for the Hannibal map was adapted from those available on Edward Tufte’s website (Dawn Finley) and sources available at Re-Visions of Minard. Translation for the Russian campaign map were taken from the reproduction in Edward Tufte’s “Beautiful Evidence”, Wikimedia Commons and sources available from Re-Visions of Minard.

Additionally, the Re-Visions of Minard Website include a number of interesting adaptations of the Napoleon graphic that might be of interest, including both Matlab and GGPlot2 reconstructions.

Working With Words, Numbers and Images: A Reading List

Rob Oakes — Sun, 14 Feb 2010 18:10:28 +0000

There are three tools that a professional, scientific or technical writer needs to make use of: words, numbers and images. In many cases, such as an effective illustration or chart, all three will be used.

The following books introduce principles and examples of how to use these tools to their fullest extent. Some of the titles are historical and others are academic. In every case, though, they highlight strategies that can be used to more effectively communicate ideas. Additionally, each one is also an interesting and fantastic read.

	Math and the Mona Lisa by Bulent Atalay. For more than 500 years, the name of Leonardo Da Vinci has been synonymous with brilliance. His careful observation of nature, collection and analysis of evidence, and use of mathematics to explain his observations represented a radical shift that foreshadowed the modern scientific method.In this book, Bulent Atalay explains why Leonardo was a remarkable artist, engineer and scientist. He looks at the hidden patterns, geometric concepts and impeccable perspective in order to probe the mind that dreamt of helicopters, unsinkable ships and underwater exploration.
	Leonardo’s Notebooks, edited by H. Anna Suh. To understand a man, you must read him in his own words. This volume provides an opportunity to sample Da Vinci’s writings on anatomy, botany, architecture, sculpture and the physical sciences. The key illustrations from his notebooks have also been reproduced.
	Galielo at Work: His Scientific Biography by Stillman Drake. Like Leonardo, Galileo was a scientific titan. As Stephen Hawking aptly summarized, “Galileo, perhaps more than any other single person, was responsible for the birth of modern science.”But why? What was it about his innovative combination of experiment and mathematics that was so important. How did he analyze data? How did he present it to others?This book attempts to answer those questions. It lays aside the philosophical implications of Galileo’s rift with the Catholic church and instead looks at how Galileo focused his mind on physical quantities and the mathematical relationships between them.
	The Visual Display of Quantitative Information by Edward Tufte. Communicating complex ideas is difficult. One of the most important tools in that struggle are charts, graphs and illustrations. Unfortunately, however, these important figures often receive less attention than other aspects of a manuscript.In this book, Tufte provides inspiring examples of graphics that are beautiful to behold and illuminating to ponder. He also includes shockingly bad examples and explains why they are so dangerous.
	Visual Explanations by Edward Tufte. In his earlier work, Tufte showed how important it is for numbers to be communicated clearly and without distracting ornamentation. In this volume, he turns his attention to a slightly different series of questions: What is the best way to show cause and effect? Or to demonstrate evolutionary change?But the most important question he asks is far more universal: How can an information display be be used to reveal the truth? To answer this, he analyzes a cholera epidemic in 19th century London and explains how poor communication contributed to the explosion of the Challenger space shuttle.
	Beautiful Evidence by Edward Tufte. Like in his previous books, Tufte again tackles the question of how to best reveal truth through the graphical display of information. But where earlier books focused on principles, Beautiful Evidence is about how seeing turns into showing. To explore that theme, this book is filled with hundreds of spectacular examples and thoughtful commentary on what makes them unique.
	Now You See It by Stephen Few. The human mind is amazingly adept at seeing and understanding patterns. An informed eye can distinguish between authentic and forgery and arrive at startlingly accurate calculations with minimal effort. But even though we are capable of recognizing the hidden influences in the world around us, we can also be mislead and exploited far too easily. We become awash in a sea of data of our own making.This book attempts to explain how the mind interprets and sees information. As the author explains in the introduction, “[This book] provides tools to dive into the ocean of information, net the best of it, bring it back to shore and sort it out.” In essence, it’s a book about seeing and distinguishing patterns on a conscious level.
	Visual Thinking by Rudolf Arnheim. It’s long been known that “seeing is believing.” This book explains why seeing is also synonymous with thinking.
	Maps and Civilization: Cartography in Culture and Society by Norman J.W. Thrower. The history of exploration and discovery is also the history of cartography. As mankind sailed out of sight of shore, he needed to learn techniques for representing his position and understanding the natural forces that he might encounter. This book tells the history of mapmaking and how advances in cartography impacted civilization.
	The Elements of Graphing Data by William S. Cleveland. In this book by William Cleveland, he presents the nuts and bolts (the how-to) of graphing data. Then he goes on to explore the science in which his principles are based..
	Visualizing Data by William S. Cleveland. Whereas The Elements of Graphing Data is primarily focused on the principles of quality display and exploration of many types of common statistical charts, Visualizing Data takes the next logical step. It introduces a number of new chart types and techniques for creating insightful and clear graphics.
	Fiasco: The American Military Adventure in Iraq, 2003 to 2005 by Thomas E. Ricks. Serious endeavors require careful forethought and nuanced planning; and few enterprises are more serious than the business of war. This controversial book looks at the missteps and mistakes of the American military as it justified, planned and executed the 2003 Iraq War.It contains haunting examples of how information can be distorted and obfuscated by both well-meaning individuals and those with insidious hidden agendas. It also explains how the adoption of American corporate culture and leadership by PowerPoint lead to serious miscommunication and early failure.
	The Challenger Launch Decision: Risky Technology, Culture and Deviance at NASA by Diane Vaughan. The Challenger explosion on January 28, 1986 changed the course of manned space flight forever. But how did it happen? What factors lead to it? Might it have been prevented?In The Challenger Launch Decision, Diane Vaughan attempts to answer those questions. In the process, she reveals that the Challenger explosion wasn’t the result of intentional wrongdoing but rather a slow-creeping definition of “normal” and comfort with the status quo.

Image Gallery: The Notebooks of Leonardo Da Vinci

Rob Oakes — Fri, 05 Feb 2010 04:31:37 +0000

When I graduated from college and had to choose between a career in industry or academics, I found it to be an easy decision: I stayed in academics. I like to have my head in the clouds and enjoy the intellectual lifestyle. (I actually consider the label of “absentminded” to be a compliment.)

It should come as no surprise, then, that one of the most enjoyable aspects of writing a book has been the opportunity to research my subject. My reading list has included books on analytic design, illustration, anatomy, typesetting, scientific communication, web technologies, LaTeX, the history of science, statistics and informational graphics. And as I worked my way through it, I took some extremely interesting side trips. One of the most intriguing, however, was an extended tangent through the notebooks of Leonardo Da Vinci.

Da Vinci died in the year 1519, nearly five hundred years ago. Yet, the modern world remains fascinated by him. His name adorns the side of best selling books and conspiracy fiction; and his drawings have become cultural icons. As an example of his popularity:

In October of 2009, Martin Kemp, a professor of art and history at the University of Oxford, found a portrait of an Italian girl. Up until Kemp took an interest, it was widely accepted that portrait had been painted sometime in the nineteenth century by an unknown artist. After a great deal of investigation and the use of a multispectral camera, however, Kemp discovered something startling. The painting had actually been done by Leonardo and nearly overnight, it went from a value of 19,000 British pounds to over 100 million.

I’m no different than the masses. Leonardo fascinates me. He had a very distinctive way of seeing the world and an engaging style. Maybe that’s why it’s so easy to lose yourself in the details of his work. Given my interest, a thorough study of Leonardo’s notebooks seemed only natural.

What I didn’t foresee, however, is that I would start to digitally collect his sketches; and in the past several months, I’ve put together a rather eclectic mix from across the internets. Earlier today, I realized that the images might be of interest to others as well; thus, I’ve created a special online gallery for them. It can be found under “Art and Photography” –> “From History” –> “The DaVinci Notebooks” To get there more quickly, you could also just click here.

Enjoy!

About This Book Thing

Rob Oakes — Thu, 28 Jan 2010 21:22:09 +0000

In the past few weeks, I’ve had several observant readers ask about one of my “secret” projects. They’ve wondered what I’m up to and why it’s detracting from other endeavors. After answering another query this morning, I decided that it’s probably time to speak openly about it. So, here’s my public confession: I’m writing a book.

It’s about scientific and professional writing and open source. Moreover, it will be interesting, intriguing and revolutionary. (Yes, I have an inflated sense of ego.)

Before really diving into the details, I’d like to give a bit of personal background. This might help you understand why I’m passionate about the subject.

Background

Ten years ago, had someone told me that I would end up a scientist and engineer, I would have laughed at them. At the time, I had just started at University and I was fully set on a career in either illustration, design or architecture. I was much too “visual” and “right-brained” to surround myself by geeks, freaks and nerds. It didn’t help that I spent a huge amount of time grooming myself to be an “artist”.

During high school, I had been cursed with moderate talent and highly indulgent instructors. They praised my artwork. They called it interesting and innovative. They encouraged me to refine my technique and to major in visual arts. So, I did.

But as time went on, I realized that I wasn’t very happy. I realized that I had other interests. I enjoyed art, I did well in it; but art classes weren’t my favorites. That honor, as it turned out, was reserved for mathematics and science.

There was also another problem, I found that I lacked the discipline required to systematically create an individual style and build a portfolio. I wanted create art for myself, not for other people; and that is a fatal flaw in an illustrator (the type of work that most interested me). Illustration, by definition, is work that has been requested for a particular use. I was more interested in my own whims than those of potential clients. Thus, not long after recognizing my problem, I decided to go a different direction and changed my major to engineering.

Analytic Design

But even though I condemned myself to a life of investigation and exploration, I remained very interested in art, photography, layout and design. This interest (and some substantial urging by the doctor I worked for at the time) resulted in a fascination with where science and art seemed to intersect.

And this is when I noticed, like many others, a strange trend. If you look at the work of those considered the greatest minds in history – Leonardo Da Vinci, Isaac Newton, Galileo Gallilei, Thomas Jefferson, Marie Curie, Albert Einstein, etc. – you find something intriguing. In nearly every case, those recognized as brilliant weren’t judged solely on the quality of their thinking, but also by how they presented and explained their insights. Indeed, the thing identified as “brilliance” was not merely exceptional thinking, but clear communication. After, all, clever thoughts and ingenious solutions mean nothing if they can’t be captured, understood and disseminated.

What is often overlooked, however, is just how much work is required to explain ideas which aren’t inherently simple, and that raises an important problem. Communication in general, and writing in particular, is very hard. To distill random and chaotic thought into a coherent and logical narrative requires energy, time and effort. This is especially true of scientific, technical or professional subjects.

Such documents contain an enormous amount of baggage: figures, theorems, mathematical equations, algorithms, cross references and indexes. And while such things simplify the life of a reader, they can positively drown a writer.

Tools and Technology

Luckily, however, there are tools and technologies that can simplify the organizational side of communication and allow for a writer to focus on the important parts: words, pictures and evidence. But finding these tools and making them work together can be a very daunting task. And that’s why I’m writing a book.

In its pages, I want to introduce principles and ideas that are characteristic of good communication. For example, why was Albert Einstein able to convince both scientists and the general public that E=MC^2? Why do we still read the scientific treatises of Isaac Newton or Charles Darwin – centuries after their ideas have been edited, amended, extended and restated? Why does scientific communication often fail and what can be done to prevent breakdowns or misunderstanding?

Then, I want to show how those principles can be applied. I’ll look at open source tools and how they work. What programs are available for capturing information and references, organizing thoughts, and eventually creating a cohesive manuscript? How can open source tools be used to create beautiful charts, maps, graphs and tables? How can the process of collaboration be simplified?

Finally, I want to show why open source is awesome. I want to demonstrate why it is functionally superior to proprietary alternatives; and I want to do this in a way that shows it to be accessible. (Far too often, I find that people are afraid of open source because they view at as “complicated”, which is very unfortunate.)

Conclusion

When I first switched from art to science, I found myself confused, lost and overwhelmed. Not only did I need to adjust to a completely new discipline and culture, but I also found that no one used the same computer programs! And while it took some time to understand why, once I did, I realized that I would never go back to the “old” way of doing things.

Thus, I’m writing the book I wish someone had given me as a new science student; a book that explained the functional bits of scientific and professional writing with lots of examples, code, and pretty pictures. This is what I’ve tried to put together, something that is part theoretical treatise but mostly instructional manual.

So, for those of you who have wondered where I’ve been and where I’ve wandered; that’s the answer. I’m writing book. The good news, however, is that it’s nearly finished. And it’s in a state where I can start to share thoughts and examples from its chapters. In the next few months, expect to see some of that information here.

But because I’m passionate about the subject, I would love to hear others have to say. What makes for a good piece of informative writing? What are examples of clear communication? What lessons might we in the sciences learn from others? And what tricks do you have for being more efficient?

Typeset Your Curriculum Vitae – Part 3: Automatically Generate a List of Publications

Rob Oakes — Wed, 02 Dec 2009 15:19:44 +0000

Publications are the currency of ideas. Through them the experts, thinkers and dreamers of this world can share their thoughts and insights. A good publication is not only influential, but it’s even capable of shifting the course of a whole society, as Martin Luther King demonstrated with his “Letter from a Birmingham Jail”.

Since publications are so important to the dissemination of knowledge, there is a rather high expectation that an academic author should publish prolifically. The mantra “Publish or Perish” is not just a clever quip, but a very serious way of life.

It is ironic, then, that the most prolific of academic writers can suffer from a surprising problem: it can be very difficult to keep track of all of their work. Yet, an up to date CV is very important. After all, publishing your work in influential journals is an important first step toward establishing tenure!

Members of a research team or those who collaborate outside of their institution experience this same problem, only more so. Such a person may work on many projects at once, but only have direct responsibility for one or two of them. This places the researcher in the unenviable position of trying to track the work of others. This situation becomes even more complicated if the collaborator refuses to play by the rules of common decency.

It would be nice, for example, if the primary author of a publication would notify the co-authors of its progress, or when it has been submitted. But … that doesn’t always happen. Academic researchers are busy people and soliciting feedback from all of your collaborators can be difficult … and there is a tendency for difficult things to go undone. Thus, if you don’t follow what your team mates are working on, it is quite possible that an abstract might have gotten submitted while your back was turned.

To stay on top of the “delightful chaos”, you need to have some kind of system. Personally, I keep my list of projects and publications in three places. The first (and perhaps most important) is the hand-written list in my experimental notebook. Any time I hear about a new project, it gets added to this list. I keep track of what I’ve contributed, what papers or abstracts have been created from the data, and what their status is. When I know that an abstract or paper has been accepted, I then create an entry for the item in my bibliography manager. Once in the bibliography manager, I can cite the reference in other documents such as proposals or related papers.

About once a year, I go through the tedious process of updating my CV. This typically involves manually sorting through both my project list and my reference database and account for new items or reconcile differences. Every time I do this, it’s painful; and because I’ve historically formatted the reference list by hand, it’s not uncommon for a typo to sneak its way in or for an author to accidentally get left off of a citation. These mistakes are never intentional, but they do happen.

When I find such an error in the reference database, I fix it. But since I often import these references from websites, the errors tend to be few and far between. Moreover, my reference database is something that I use every day; as a result, it gets a lot of scrutiny. My CV, on the other hand, gets updated much less frequently and errors tend to persist longer.

For a very long time, I’ve wanted to automate the process. Instead of keeping three separate lists – active projects, reference database, and CV – I’d prefer to keep only one (or two). But I’ve never found a really satisfactory way of doing so. Or at least I hadn’t found a system until quite recently.

In my last review of different ways to typeset a CV, I came across an interesting article by Dario Taraborelli. In it, he described how to create a CV based on the standard “article” document class. It was well designed, elegant, simple and attractive. From his work, I created the xetexCV document class. Additional research turned up an add-on module that makes it convenient to automatically generate a list of publications. So, for the first time in a great while, I have finally found a way to automatically generate a publications list in a simple and automated manner. In this article, I will demonstrate how that is done.

The Basic Requirements

For any automated solution to be successful, it needs to meet three primary criteria:

When generating a list of references, bibliographic details should be pulled from the database. This ensures that the information is always up to date.
The citation style must be customizable. I want the ability to specify whether my CV includes page numbers and hyperlinks. Not all citation styles do.
The publication list should be broken into different categories. Book chapters should not be listed with peer-reviewed journal articles, nor should abstracts be listed amongst the books.

cvsplitbib and xetexCV

All three of these requirements can be met by using the xetexCV document class and the cvsplitbib style (both are provided in this zipped directory). cvsplitbib is a version of regular splitbib, a package that makes it possible to create categories in your bibliography. While all of the main features work as described in the splitbib manual, the macros have been changed to use the \cvsection and \cvsubsection macros of the xetexCV document class. This ensures that formatting and indentation of the publications list is consistent with the rest of the document.

This combination also has an additional benefit. Because cvsplitbib works entirely through LaTeX, any bibtex citaiton style can be used. In the example below, I’ve used the the “plain” style, though it would have been just as easy to use ama, MLA, chicago or any other style in my distribution.

Note: There is an important difference between cvsplitbib and splitbib. In the standard version of splitbib, it is possible to create subsubcategories (a second level of nested references). Cvsplitbib does not support this as xetexCV does not define a \cvsubsubsection macro.

Creating a Categorized List

Creating different bibliography sections is achieved through the use of the “category” environment. The arguments of the environment define the title of the section while the \SBentries macro is used to specify the citation key of any reference that should be included in that particular list.

Note: Citation keys should be added as a list separated by commas, any spaces will result in an error.

The following example creates a categorized bibliography with three different sections (Novels, Children’s Literature and Comics, Graphic Novels). The bibliography data is pulled from the file “NeilGaiman-Publications.bib”, part of the xetexCV examples (see below):

\begin{category}[A]{Novels}
\SBentries{Gaiman2001-American-Gods,Gaiman2005-Anansi,
Gaiman1990-Good-Omens,Gaiman1996-Neverwhere,
Gaiman1999a-Stardust,Gaiman1985-Ghastly}
\end{category}

\begin{category}[B]{Children’s Literature}
\SBentries{Gaiman1998a-Goldfish,Gaiman2008-Graveyard}
\end{category}

\begin{category}[C]{Comics and Graphic Novels}
\SBentries{Gaiman1994b-Angela,Gaiman1993-Angels,
Gaiman1989-Black-Orchid,Gaiman1993a-
Books-Magic,Gaiman1994a-Death-Living,Gaiman1997-Death-Time-Life,
Gaiman2000a-Green-Lantern,Gaiman2003-Marvel-1602,
Gaiman1999-Midnight-Days,Gaiman1994-Anthology-Virus,
Gaiman2000-End-World,Gaiman1992-Signal-to-Noise,
Gaiman1998b-Smoke-Mirrors,Gaiman1998-Inkeeper,
Gaiman1991-Sandman,Gaiman1987-Violent-Cases}
\end{category}

Any reference that is not included in a specified category will be added to a fourth category, titled: “Miscellaneous”.

Generating the Bibliography

Once the various references have been categorized, you can generate the bibliography as you would in any other document:

\nocite{*}\bibliographystyle{plain}
\bibliography{NeilGaiman-Publications}

Here the \nocite{*} command is used to list all of the publications in the .bib database. \bibliographystyle{plain} specifies the BibTeX style to use, and \bibliography{Database-Name} tells BibTeX which database file to pull the information from. Part of the formatted output from the example is shown below.

Examples and Class Files

Document Class and Examples (.zip). This file contains the document class, cvsplitbib package, examples and other supporting files.
Neil Gaiman (Automatic Bibliography). LaTeX Source, PDF Output, Reference Database
Albert Einstein. LaTeX Source, PDF Output
Isaac Newton. LaTeX Source, PDF Output

______________________________________________________

Acknowledgements and Further Reading

The xetexCV document style is based on work from Dario Taraborelli’s website. The cvsplitbib package is a modified version of splitbib, by Nicolas Markey.

This article is part 3 of a four part series. Part 1 introduces the xetexCV document class and describes its use. Part 2 dissects the code and explains how it works. Part 4 describes how to use the document class and a corresponding layout file with LyX

Typeset Your Curriculum Vitae – Part 2: Extending and Customizing an Existing Document Class

Rob Oakes — Mon, 30 Nov 2009 18:54:16 +0000

Many first-time users of LaTeX often mistakenly look at the language as a a type of glorified word processing software – albeit a particularly complicated one. While such an analogy may be apt in helping new users become acclimatized to the language, it suffers from a rather nasty problem: LaTeX isn’t a word processor.

If anything, LaTeX shares more in common with a programming languages than any type of application. In fact, the document processing system is really nothing more than a bunch of re-usable pieces of programming called macros. Everything is a macro. That includes the commands that every user is familiar with: \title{}, \section{}, \subsection{}; in addition to the internal formatting commands that allows LaTeX to function. (Most of the macros were originally created or packaged by Leslie Lamport as a way of making TeX – the typesetting system created by Donald Knuth – easier to work with.)

This has some rather practical consequence; because everything in LaTeX is a macro, it is far more extensible than a word processor could ever hope to be. If you require a feature that doesn’t yet exist, it typically isn’t all that difficult to add it. And when your extension is packaged inside a style or class, you can use those customizations in anything that you want to write.

But though creating macros isn’t particularly complicated, it is a different beast than just using the stock macros for writing. This is not surprising, the craft of design is inherently different than the craft of writing. There are different conventions to follow and different topics to obsess about. In the first article of this series, I introduced the xetexCV document class, which is one example of where I decided to don the designer hat.

But before you get too far down the road of customizing and extending, there are a some important things that you need to know. These include the general conventions used when working with document classes, their internal anatomy, an understanding of how macros are created, and how to handle formatting and layout challenges. In this article, I will look at these issues more in detail, particularly as they pertain to xetexCV. In the process of reviewing these topics, I will also explain some of my design choices.

Understanding Conventions

When trying to customize or extend a document class, perhaps the very first thing to understand is that there are certain conventions. The document “LateX 2E for class and package designers” describes them this way:

LaTeX has three types of commands.

There are the author commands, such as \section, \emph and \times: most of these have short names, all in lower case.

There are also the class and package writer commands. Most of these have long mixed-case names such as the following:

\InputIfFileExists \RequirePackage \PassOptionsToClass

Finally, there are the internal commands used in the LaTeX implementation, such as \@tempcnta, \@ifnextchar and \@eha: most of these commands contain @ in their names, which means they should not be used in documents, only in class and package files.

These conventions help to better separate procedural formatting from the descriptive tags used in writing a document. But it also makes it easier to identify what different sections of the code do. Sections that are responsible for importing and configuring packages will typically use mixed case names, while variables will contain the @ symbol, and user facing macros will be in lower case.

The Internal Anatomy of a Document Class

The next important thing to understand is that all document classes have a few well defined components. That is to say, they have an internal anatomy which contributes to their function and physiology. The internal bits include the class description and required statements, a block that imports and configures packages which new features will depend on, and a section where new macro commands are created or existing commands are redefined.

Class Description and Required Statements

\ProvidesClass

The very first such section gives the package name and description. Also included is a definition of the most recent version of LaTeX that the document class will work with. The relevant code from xetexCV is shown below:

\NeedsTeXFormat{LaTeX2e}
\ProvidesClass{xetexCV}[2009/11/30 – Modern looking résumé which uses
the xetex typesetting system]
…

Here, the \NeedsTeXFormat{LaTeX2e} command is the earliest version of LaTeX that the document class will work with. \ProvidesClass{xetexCV} identifies the class name (xeteCV) and everything in square brackets is the document description.

\LoadClassWithOptions

Generally speaking, there are two major types of LaTeX document classes: those that are free standing, and those that are variations of other document classes. The standard classes (article.cls, report.cls, and book.cls) are examples of the former. They contain all of the code needed to compile and work without additional extensions. xetexCV, is an example of the second type.

To load an existing class as the foundation, you can use the \LoadClassWithOptions macro:

…
\LoadClassWithOptions{article}
…

xetexCV uses “article” for its foundation. For that reason it is likely that other packages and styles will work without conflict.

Building on the Foundation

\RequirePackage

After the base class has been loaded, the new package is fully functional, if identical to the foundation class. The next step in the customization process, then, is to begin adding new features and modifying old ones. In the case of xetexCV, there were several changes which I wanted to make. First, I wanted to use an 8 1/2 inch by 11 inch paper (US letter) by default. I also wanted to set a 1 inch margin. Next, I wanted to use the XeTeX typesetting system and a pair of advanced OpenType fonts.

These changes are made through the use of other LaTeX packages. In a document class, add-on modules are loaded through the \RequirePackage command. Below is the related code from xetexCV:

…
\RequirePackage{fontspec}
\RequirePackage{xunicode}
\RequirePackage{xlxtra}

\RequirePackage{graphicx}
\RequirePackage[colorlinks]{hyperref}

\RequirePackage{ifthen}
…

The first three packages (fontspec, xunicode, and xlxtra) are used by XeTeX to set fonts and provide unicode support. fontspec, in particular, is very important. It provides convenience macros that make it easy to change the the various font families: \setmainfont{fontname}, \setromanfont, \setsansfont, and \setmonofont. It also provides a way to manipulate font mappings, color, scaling, and inter-word spacing. graphicx is used to add images to the document and manipulate their size.

hyperref makes it possible to add clickable hyperlinks and bookmarks. The [colorlinks] option specifies that these special links should be rendered in a different color than other text. The hyperref package is then configured using the \hypersetup{} macro. By default, I’ve configured xetexCV to use blue for urls and webpages and black for filenames:

\hypersetup{linkcolor=blue,citecolor=blue,filecolor=black,urlcolor=blue}

The ifthen package provides several macros that make it easier to apply conditional formatting. These will be discussed in more detail below.

Creating New Features and Modifying Old Ones

The last major section of a document class is where new macros are defined and old ones are modified. As might be expected, this section typically accounts for the majority of the code.

\def and \newcommand

There are two major commands used to define macros and variables: \def and \newcommand. \def is a TeX primitive while \newcommand is a LaTeX macro that checks for name clashes or other problems. In many cases, they may be used interchangeably or in concert. Consider the code block below which defines the \cvname macro:

…
\def\@cvname{\relax}
\newcommand{\cvname}[1]{\gdef\@cvname{#1}}
…

\@cvname (according to the LaTeX conventions) is a variable/macro which should only be used from inside the LaTeX document class. Consequently, it needs a user-facing macro that can be used to assign it a value. This is what \cvname does. In this example, the first line of code creates the variable and assigns it the LaTeX equivalent of a null value (\relax). The second line is a little more interesting. It defines a macro called \cvname with one argument [1]. Then, it takes the \cname argument and assigns it to \@cvname.

A second, related example can be seen in the definitions of \@cvimage and \cvimage:

…
\def\@cvimage{\relax}
\newcommand{\cvimage}[1]{\gdef\@cvimage{#1}}
…

Again, the first line of code creates a variable (\@cvimage) with a null value, and the second line creates the user facing macro (\cvimage). In later commands, the internal variables are used for layout and formatting (see below).

\renewcommand

If I need to modify a particular piece of code, those changes can be made using the \renewcommand macro. This is similar to over-riding a method in an object-oriented programming language like C++ or Python. Everything that hasn’t been changed continues to work as before. In xetexCV, I wanted to modify the \section and \subsection macros. To do so requires \@startsection and six arguments:

\@startsection {NAME}{LEVEL}{INDENT}{BEFORESKIP}{AFTERSKIP}{STYLE}

Generic command to start a section.

NAME : e.g., “subsection”
LEVEL : a number, denoting the depth of the section in the table of contents
INDENT : Indentation of the heading from the left margin
BEFORESKIP : Space to leave above the heading
AFTERSKIP : Space to leave after the heading
STYLE : Commands to set the font and style of the heading

I, therefore, copied over the definitions from article.cls and modified the spacing and fonts:

\renewcommand\section{\@startsection
{section}{1}{\z@}%
{-3.5ex \@plus –1ex
\@minus -.2ex \vspace{1mm}}%
{0.5mm}%
{\sffamily\large\bfseries}}

\renewcommand\subsection{\@startsection
{subsection}{1}{\z@}%
{-3.5ex \@plus -1ex \@minus -.2ex}%
{3.0mm}%
{\sffamily\mdseries}}

But while I chose to completely rewrite my section definitions, much the same thing could have been accomplished through the sectsty package. Despite the ease of sectsty, I felt that it would be better to define my styles from scratch. sectsty often throws errors when used with other document classes, like scrbook.

Rules for Creating New Macros

The third important thing to understand when writing a document class is when you should add new macros rather than modifying the behavior of old ones. In trying to make this decision, it is good to think about how changing a macro’s behavior will influence other document features. In cases where a change might cause other features to break, it is better to add a new macro.

Managing Complex Formatting – \cvsection and \cvsubsection

While I was redefining the \section and \subsection macros, I ran into one such difficulty. I wanted to set one indentation level for the header and another for the body text. Further, I wanted to include a decorative line following the header. Yet, no matter how I chose to define the STYLE or AFTERSKIP arguments, I couldn’t quite achieve the formatting I wished (shown below).

I finally decided that the formatting was too complex and required a new macro that could be used instead of \section and \subsection. These new additions became \cvsection and \cvsubsection:

\newcommand{\cvsection}[1]{\leftskip 0cm
\section*{#1}\decorativeline\marginpar{\vspace{0.3ex}}
\leftskip 116pt}

As in the previous examples, \newcommand was used to define a code block with a single argument – the text to be formatted). Then, I specified the procedure that I wanted to apply to the text

The left text margin is moved so that it is flush with the page margin (\leftskip 0cm)
A non-numbered section header is added (\section*{#1})
The decorative line is included (\decorativeline)
The text indentation is returned to the previous value (\leftskip 116pt).

The definition of \cvsubsection is similar, with two exceptions. First, the \cvsubsection style does not include a decorative life. The second major difference is more substantial. I found the vertical spacing preceding the \subsection definition to be too large, I therefore moved the entire text block up by 0.2 cm (\vspace{-0.2cm}). This was ultimately easier than trying to find a better value for the BEFORESKIP argument:

\newcommand{\cvsubsection}[1]{\leftskip 0cm \vspace{-0.2cm}
\subsection*{#1}\vspace{1.0mm} \leftskip 116pt}

Note: When given a negative value, \vspace moves up the block of text by the specified amount.

Layout and Formatting

One of the very best features of TeX is the automatic layout algorithm that it uses to position content. In nearly all cases, allowing for LaTeX to manage spacing and layout will result in a better looking document than if you try and do so yourself. There will be circumstances, however, where it is necessary to manually specify the position of photos or text in a more precise manner.

In xetexCV, there are two such cases. In the first, special handling is required so that the CV contact information is correctly labeled and typeset. Additional formatting commands are required so that the images and text of the title section have a “balanced” feel to them. All of this processing occurs in the \makecvtitle macro.

Conditional Formatting – \ifthenelse

While it may appear simple, the formatting of the xetexCV contact section (which includes the institution, address, phone, fax, email and website information) is rather involved. First, in order to simplify the use of the document class, I chose to add this information through the use of special macros (\email, \website\, \cvname, etc.). Because the information is added through tags, this means that logical rules must be set up to handle to handle the layout. What should happen, for example, if a fax number or website isn’t supplied?

The obvious answer is that the information should be left out of the contact details. That also means that the label (“Fax:”, “Phone:”, “Email:”) should be omitted. To accomplish this goal requires conditional formatting, better known to programmers as the “if, then, else” loop. “If” a condition exists, “then” do one thing, “else” do another.

In LaTeX, conditional formatting is performed by the “ifthen” package and uses the \ifthenelse macro.

\ifthenelse takes three arguments:

\ifthenelse{CONDITION}{IF TRUE}{ELSE}

CONDITION : The test condition, often expressed as \equals{VARIABLE}{VALUE}
IF TRUE : The code that should be executed if the condition is satisfied
ELSE : What to do if the condition is false

At several instances in xetexCV, \ifthenelse is used to test if there is a null (or empty) value for a particular tag. In the cases where it finds a null value, then no formatting commands are executed. If, however, there is a value, then text is added. A few of the simpler examples are shown below:

\ifthenelse{\equal{\@phonenumber}{\relax}}
{}{Phone: \texttt{\@phonenumber}\\}
\ifthenelse{\equal{\@faxnumber}{\relax}}
{}{Fax: \texttt{\@faxnumber}\\[0.2cm]}
\ifthenelse{\equal{\@email}{\relax}}
{}{Email: \href{mailto:\@email}{\@email}\\}
\ifthenelse{\equal{\@website}{\relax}}
{}{Website: \href{\@website}{\@website}\\}

In each of these cases, the \equal macro is used to check if the pertinent variable (specified by the \@ symbol) still has a value of \relax (\equal{\@variablename}{relax}).

Note: \relax can be thought of as an equivalent to null in other language.

If the value is still set to \relax, then nothing is done (denoted by the empty diamond brackets). If there is a non-null value, however, then the appropriate label and formatted text is added to the document.

While these examples are relatively simple, the same principle can be used to for very complicated formatting. \makecvtitle, for example, uses a similar conditional statement to see if the user has specified an image. If not, then it will use one formatting style for the entire title section. If so, then it uses the layout shown in the examples.

Balancing Images and Text – \makecvtitle

In addition to the conditional formatting issues, \makecvtitle must also wrestle with the best way of creating a single, unified title block. The CV name, contact information and photo should all be included. Moreover, subsequent sections should be added after the end of the photo, rather than wrapping around it. One of the easiest ways to create such a unified block of content is to use the minipage environment.

The minipage is extremely flexible. It can have text, formatting and image within a self-enclosed space. Furthermore, they can be embedded within one another. By creating a series of embedded minipages (as shown below), you can group related content. You can additionally specify how wide you want each minipage environment to be, or you can let the LaTeX engine make those determinations.

Note: An additional advantage of minipage is that any footnotes added to text within the minipage are part of that environment instead of the main document. This is useful for adding footnotes to figures or text. Accomplishing the same thing in Microsoft Word without tedious manual formatting is impossible.

I’ve used this strategy to ensure that text containing the contact details is properly aligned against the cv photo. The minipage environment from xetexCV is shown below:

\begin{minipage}{6in}
\begin{minipage}{114pt}
\resizebox*{100pt}{!}
{\includegraphics{\@cvimage}}
\end{minipage}
\begin{minipage}{4in}
\ifthenelse{\equal{\@institution}{\relax}}
{}{\bfseries\@institution\\}
\mdseries\@contactaddress\\[0.2cm]
\ifthenelse{\equal{\@phonenumber}{\relax}}
{}{Phone: \texttt{\@phonenumber}\\}
\ifthenelse{\equal{\@faxnumber}{\relax}}
{}{Fax: \texttt{\@faxnumber}\\[0.2cm]}
\ifthenelse{\equal{\@email}{\relax}}
{}{Email: \href{mailto:\@email}{\@email}\\}
\ifthenelse{\equal{\@website}{\relax}}
{}{Website: \href{\@website}{\@website}\\}
\end{minipage}
\end{minipage}

As you may notice, there are three embedded minipages here (the main minipage and two sub pages). The main minipage is set to span the entire width of the text. However, the first embedded minipage is only 114 pt in width (about 1.6 inches). Within this first box, I place the cvphoto, resized to a width of 100 pt:

\resizebox*{100pt}{!}
{\includegraphics{\@cvimage}}

Note: The {!} argument in \resizebox allows the image to stretch proportionally in the vertical direction.

The second minipage is allowed to take up the rest of the paper width and includes the conditional formatting for the contact details.

Conclusion

Through the use of custom formatting, packages and macros, I’ve created a document that is substantially different from the foundation class (article.cls). Despite those differences, nearly every major customization was made using simple LaTeX macros which many users will be familiar with. Further, because it is built on a solid foundation, the ability to use other add-on packages is retained.

Extending an existing document class does not need to be difficult. It is possible to use many of the commands used for writing a document. Additionally, the modifications could be packaged as a style (.sty) and used to enhance any content that I might write in the future.

This flexibility is one of the reasons why LaTeX is such a good candidate for the typesetting of scientific and technical material. It’s also why LaTeX is a good candidate for creating design heavy documents, such as a curiculum vitae.

______________________________________________________

Acknowledgements and Further Reading

Additional information on the structure of document classes can be found in the article “Rolling your own Document Class: Using LaTeX to keep away from the Dark Side” by Peter Flynn of Silmaril Consultants.

A second article called “How to develop your own document class – our experience” describes additional techniques for documents that might have non-standard formatting. It outlines several common mistakes and ways to get avoid them.

This article is part 2 of a four part series. Part 1 introduces the xetexCV document class and describes its use. Part 3 shows how to generate a publications list through the use of a .bib database and custom BibTeX style. Part 4 describes how to use the document class and a corresponding layout file with LyX.

Typeset Your Curriculum Vitae – Part 1: The xetexCV Document Class

Rob Oakes — Wed, 25 Nov 2009 04:02:27 +0000

Very few documents are more personal than a curriculum vitae (CV). A CV lists a person’s educational history, who they’ve worked for and what they’ve accomplished. Moreover, a CV is frequently used to judge a person’s inherent worth and value (or at least exploitability). A quality curiculum vitae matters, a lot.

For that reason, a CV not only needs to include all the pertinent information of a person’s life, but it also needs to look good. An attractive CV with good spacing and contrast leaves a positive impression and makes it easier to find information. When laid out correctly, a reviewer might just find themselves scouring past accomplishments for interesting tidbits: “I didn’t realize that this applicant organized a lecture series with Patch Adams and other notables, that’s interesting!”

There are many dedicated CV classes which can be used to typeset a CV with LaTeX. Unfortunately, they all share one thing in common: they’re complicated. To ensure that the spacing and fonts are correct, far too often a document based on these classes will devolve into a jumbled mess of tags and obscure page layout commands.

I’ve tried to typeset my CV with LaTeX a number of times, and in nearly every case, I’ve been unhappy with the output. Sure, the end product is attractive enough; but it doesn’t quite match the CV that I have in my head. To really arrive at my coveted ideal of perfection, I would need to manually specify the font and position of nearly every element on the page; which is why I’ve normally used a dedicated layout program (such as Adobe InDesign) for the job.

This is unfortunate, however, since a CV based on LaTeX would be much easier to keep current. Additionally, a well designed document class and corresponding BibTeX style would allow me to use my existing bibliographic database to automate the process.

Thus, any time that I need to make major updates, I always check the internet for information and new LaTeX classes. While conducting one such review a few days ago, I came across a very interesting article and set of examples by Dario Taraborelli of the University of Surrey. In his write-up, Dr. Taraborelli expresses frustrations which are similar to my own; but instead of complaining and throwing his hands up in exasperation, he went on to solve the problems in an elegant way.

Rather than use or modify one of the existing CV classes (as I would have done), Taraborelli opted to extend the article class by adding additional features. Perhaps the most important addition was the use of XeTeX and the \fontspec package, which allows for OpenType or expert fonts to be used and other advanced features. The end result of this work is an attractive and flexible document template.

Dr. Taraborelli’s example also shows how easy it can be to extend an existing document class to meet a more specific need. As I read through the example code, I realized that it would be straightforward to build upon his work and create something that would suit my particular aesthetic.

So … I did, and this series is the result of my efforts.

In the remainder of this article, I will introduce the new CV document class I created (called xetexCV) and provide a few examples of how to work with it. In part 2, I will dissect the class code and explain how it works. I will also provide hints on how it might be modified. In part 3, I will show how to generate a publications list through the use of a .bib database and custom BibTeX style. In Part 4, I will describe how the document class and corresponding layout file can be used with LyX.

xetexCV – An Attractive and Easy to Use CV Document Class

Downloads and Installation

All of the files needed to use the class (including xetexCV.cls, examples and supporting files) can be found here. To install, copy to an appropriate folder in your tex path and refresh the tex database:

sudo texhash

Package Requirements

XeTeX Typesetting Engine. For users of Mac OS X and Windows, XeTeX is included in the MacTeX and MikTeX distributions (respectively). Users of Linux, however, will need to install the texlive-xetex package
Fontspec package (included with MacTeX, MikTeX or texlive-xetex)
OpenType Expert Fonts. By default, the document class uses Fontin and Fontin sans.

Curiculum Vitae Examples

Albert Einstein. LaTeX Source, PDF Output
Isaac Newton. LaTeX Source, PDF Output

xetexCV Class Usage

To use the xetexCV class, there are a few important macros. These include:

The \makecvtitle macro and title block tags
The \cvsection and \cvsubsection macros
The \years tag for hanging notes

\makecvtitle

The title block is created through the use of the \makecvtitle macro. The name, institution and contact information are all specified by descriptive tags. The example below shows what these are and how they are used:

\documentclass{xetexCV}

\cvname{Albert Einstein}
\cvimage{Albert-Einstein.jpg}
\institution{Institute for Advanced Study}

\contactaddress{Einstein Drive\\
Princeton, New Jersey 08450
United States of America}

\phone number{609-734-8000}
\faxnumber{609-924-8399}
\email{a.einstein@ias.edu}
\website{http://www.ias.edu/spfeatures/einstein}

\makecvtitle

\cvsection and \cvsubsection

Sections and subsections can be added through the \cvsection{section name} and \cvsubsection{subsection name} macros. The following codeblock creates the “Publications” section header and “Peer Reviewed Journals” subhead seen at right:

\cvsection{Publications}
\cvsubsection{Journal Articles}

…

Hanging Notes

Hanging notes can be specified through the use of the \years macro. This is helpful for typesetting education, employment history, and awards. The following example produces the output seen below:

\cvsection{Appointments Held}

University of Bern \years{1908-1911}\\
University of Zurich \years{1911-1912}\\
Charles University of Prague \years{1912-1914}\\
Prussian Academy of Sciences, Berlin \years{1914-1932}\\
University of Leiden \years{1920-1930}\\
Institute for Advanced Study, Princeton \years{1932-1955}

Choosing a Font

To change the font, use the \fontspec package and appropriate \settext macro. The example below shows how to change the main font and sans font to Warnock Pro and Frutiger LT Std (commercial fonts available as part of the Adobe Font Folio 11):

\setmainfont[Ligatures={Common}, Numbers={OldStyle}]{Warnock Pro}
\setsansfont{Frutiger LT Std}

All of the examples shown here were typeset with Warner Pro and Frutiger. By default, however, the document class uses Fontin and Fontin sans, which are available as free downloads. Another excellent font alternative is Gentium Basic. Because XeTeX can use OpenType fonts, however, you can use any font installed on your computer.

Other Features

Font features such as ligatures and glyph variants can be accessed through the fontspec package
The examples and class are unicode compatible. This means means that you can write in Chinese, Cyrillic or Greek without additional packages

______________________________________________________

Acknowledgements and Further Reading

As noted above, this document class is based on a similar example available from Dario Taraborelli’s website. Other examples of how to create a resume or CV are available from Matthew Boedicker and Kamil Wójcicki.

Customizing LyX: Character Styles and the LyX Local Layout

Rob Oakes — Sat, 14 Nov 2009 21:00:00 +0000

Imagine for a minute that you’re writing a book or technical manual. Let’s say it’s a book on technology, maybe the open source tools used for scientific writing (to randomly pick an example). As you write this book, you realize that you need some way to cue the reader into different parts of the text.

For instance, you might want all definitions to appear in bolded text so that a reader pick out key terms quickly. Or you might want code examples to appear in a different font than the regular text, again, so they’re easy to find. What’s the best way to do this?

Sure, you could just bold the definitions, or manually change the font for the code examples. But that’s painful! Changing typeface and size every time that you have a section of code will eventually result in a lot of lost time. Moreover, you might make a mistake, which destroys your consistency and makes your writing look unprofessional. There must be a better way!

Thankfully, there is. It’s through the consistent use of styles.

Styles Defined

A style is a collection of formatting commands with a meaningful name. Examples in Microsoft Word include “Heading 1”, “Body”, and “Figure”. When you use them consistently, styles allow for “logical markup” of a document. That is to say, you tell the computer what the section, paragraph or text corresponds to and then it handles the formatting. For this reason, styles form the basis of the “What You See Is What You Mean (WYSIWYM)” philosophy used by LyX and other “document processors”.

There are three major types of styles: section styles, paragraph styles and character styles. You are likely familiar with section and paragraph styles, as they populate the main drop down box of LyX’s main toolbar. The most common section styles are: chapter, section, subsection. Examples of paragraph styles include: standard text, quotations, and code. But the third main type of style, “a character style” may be a stranger. It isn’t as common as it’s two older brethren and easier to lose track of; which is a huge shame, because it is just as important.

Character styles, as the name implies, work on individual characters within the text of a paragraph. To extend the examples given in the introduction, the most efficient way to bold a definition would be to use a pre-defined character style called “definition”. Ditto for an in-paragraph sample of code.

“Now wait a minute,” you’re probably saying, “I’m not sure that I follow the difference between character styles and other text formatting commands like ‘bold’, ‘italic’, or ‘underline’!”

That is perfectly alright, the distinction can be a bit hazy. Nonetheless, it is still key in understanding the difference between “logical markup” and formatting. Let me see if I can clear it up.

As described above, a style is typically used to describe what a part of the document is, rather than how it should look. Subsequently, you are unlikely to see a style called “bold”, “italic” or “center”. Those terms describe how the text should appear; which is to say, they are formatting commands. Bolded text uses a heavier line stroke and centered text appears in the middle of the page. A style, on the other hand, is used to describe a “definition”. You then associate that style with how you want “definition” to look. It could be numbered, bolded and centered; or it might be larger than the surrounding text and italicized. The appearance is determined by a “style sheet”.

Note: Since the use and customization of section and paragraph styles is well explained in the LyX help documentation, this article will focus on character styles.

Using Character Styles in LyX

LyX provides some common character styles through the use of its “Logical Markup” module. These include styles for nouns, emphasizing text, and code – which may be useful for highlighting filenames or programming syntax within the body of the paragraph.

The character styles are not loaded by default, however. Instead, they come as part of an optional module. You can use them by going to the “Document->Settings->Modules” pane and adding “Logical Markup” to your document.

Once the “Logical Markup” module has been loaded, you use of apply a particular character style through the “Edit->Text Styles” menu, or by first highlighting a block of text and selecting “Text Style” from the right click context menu.

Note: The emphasize style is distinct in that it can also be toggled on and off through the “Ctrl+E” keyboard shortcut.

After you’ve formatted a block of text with character style, it will appear differently from the surrounding text. It will not only use a different font, but will also be surrounded by a set of hash marks. The example below shows a document with two instances of the “Code” character style. Optionally, character styles have a label which you be toggled on and off. The screen shot below shows what this looks like. The first instance, “TEXMFLOCAL”, has the label toggled off while the instance, “/var/lib/texmf/tex/latex”, shows the same character style with the label visible.

Customizing Character Styles

But what if you need access to other character styles? Perhaps you’re writing a novel and would like to have a dedicated style for specifying when a character is talking to himself (called internal dialogue)? Or you’d finally get back to working on your textbook with bolded definitions. Such styles aren’t defined by the “Logical Markup” module, or any module at all for that matter. Nor are they likely to be used so frequently that it is worth trying to modify the built-in modules or create a new one to support them.

Luckily, there is a solution to this problem. In the first article in this series, I looked at how to create a template for an NIH grant proposal from an existing LaTeX document class. In that article, the secret to using the “nih.cls” document class lay in its corresponding LyX layout file (“nih.layout”). Also in that article, I explained how LyX layout files primarily contain information concerned with on-screen presentation.

Note: For more detail, see: Customizing LyX – Part 1, Understanding the Big Picture.

As a first approximation, what I said in that article is mostly true. But, naturally, the reality is more complex. LyX layout files can also contain information about character styles. This includes both information about how they should be presented on-screen and how the underlying LaTeX typesetting language should process them. Moreover, that information can either live in a global layout file (like nih.layout) or in the .lyx file you are working on (meaning that it can be conveniently added to a template). In the remainder of this post, I will show how to create custom character styles by modifying the local layout.

Example: Creating a “User Interface” Character Style

As you might have guessed, the example introduced above isn’t so random. For the past several weeks, I have been working on a book about scientific and technical writing. The book will cover a number of programs and technologies including LyX, LaTeX, BibTeX, Mendeley, Scribus and Kile.

Since it is geared to a somewhat technical audience, it includes both instructions on how to use program’s UI and code samples. For my code examples, I can use the internal LyX character style. But I’ve found that I would also like to highlight references to the UI. There isn’t a pre-defined UI character style.

Fortunately, I not the first person writing a book with UI references. If you look at the LyX user manuals, you will see that the LyX documentation team created a character style specifically for highlighting LyX menu commands. It uses a sans serif font and I think it is rather attractive. But, how do you go about adding it to another document?

It’s actually pretty straightforward:

Since there isn’t a UI to modify the local layout in LyX, open your document in a text editor
Create a local_layout section
Define how you want your character style to appear (both on screen and in the LaTeX output)

Step 1: Open your document in a text editor

As explained above, custom character styles live in the local_layout section (enclosed with the “\begin_local_layout” and “\end_local_layout”) near the beginning of a LyX document. Unfortunately, there is not currently a user interface pane within LyX to edit this section. As a result, open the document in your favorite text editor. The figure shows what this looks like for a file called “Book Chapter – Technical.lyx”.

Note: There is nothing to be afraid of, LyX’s file format is relatively easy to parse; even for human beings.

“Book Chapter – Technical.lyx” has three major sections:

Document: enclosed by “\begin_document” and “\end_document” (off-screen)
LaTeX Preamble: enclosed by “\begin_preamble” and “\end_preamble”
Local Layout: enclosed by “\begin_local_layout” and “end_local_layout” (off-screen)

I have added some text to the LateX preamble in this example. The command tells LyX to use the “Latin Modern” fonts instead of the LaTeX default fonts, which I find to be rather ugly:

% set fonts for nicer pdf view
\IfFileExists{\lmodern.sty}{\usepackage{lmodern}}{}

Step 2: Create a Local Layout Section

If it doesn’t already exist, insert a local_layout section. For organizational purposes, I like for it to appear following the preamble. Below is the full text of the layout section for “Book Chapter – Technical.lyx”:

\begin_local_layout
    Format 7
        InsetLayout    CharStyle:UserInterface
        LyxType               charstyle
        LabelString           userinterface
        LatexType             command
        LatexName             userinterface
        Font
            Family              Sans
        EndFont
        Preamble
            \newcommand*{\userinterface}[1]{{\sffamily #1}}
        EndPreamble
    End
\end_local_layout

In the example, there is some template code (also known as boilerplate), and values that you can modify in order to better customize the appearance of your character style. The boilerplate includes “Format 7”, LyXType charstyle”, and “LatexType command”. These values will be constant for all character styles, but everything else can be customized.

Step 3: Define how you want the character style to look

When customizing the appearance of your character style, the first thing to do is to decide what you want to call your style. This is done by modifying the “InsetLayout” argument. Since my book project covers many different programs, I’ve decided that I will use this character style for any reference to the ui. That includes menus, button names, or dialog box text. As a result, I’ve called it “CharStyle:UserInterface”. This is what appears when I go to the “Edit->Text Style” menu and in the dialog box that appears when I right click.

Next, you need to decide what you would like the “label” to be. The label appears when the “show label” option is toggled on. In this example, it’s “userinterface”.

I also need to associate my character style with a LaTeX name. Again, I’ve opted to call it “userinterface”.

The next block of options, “Font”, specify how the text will look on screen. You could be extremely ornate specifying text size, typeface, and font weight. In this case, however, I want to go with something simple. I’m just going to use the “Sans” font family for on-screen to display. Boring, but functional.

Note: A complete list of available options can be found in the LyX help documentation: “Customizing LyX: Features for the Advanced User”. See section 5.3.6.

The Preamble block tells LaTeX how it should process the character style. Again, I don’t want to do anything fancy, so I’m just going to use the “Sans Serif” font family. This is accomplished by the following LaTeX command:

\newcommand*{\userinterface}[1]{{\\sffamily #1}}

The “\newcommand” syntax tells LaTeX about my custom style, while everything after is specifying the text and the font family. “\ssfamily” refers to the sans serif font family. Other options I could have used include the roman font family (“\rmfamily”), the typewriter family (“\ttfamily”), medium text (“\mdseries”), italic (“\itshape”), or small caps (“\scshape”).

Note: For a more complete list with sample output, see here (Table 4.5).

When finished, save your document and exit the text editor. When you load the document back into LyX, your style should appear in the “Edit->Text Style” menu. The screenshot below shows how the “User Interface” character style looks in use.

Miscellaneous Things and Conclusion

Like the example in the previous article, I like to save documents with custom character styles as templates. This means that I can create additional documents very easily by going to the “File” menu and choosing the “New from template” option. The new document will have all of the template’s settings, including the LaTeX preamble and the custom character styles. It also means that I can keep separate templates for technical documents, scientific articles, book chapters and fiction. Each has character styles defined that make sense for that type of publication.

Character styles make writing long documents much easier. It’s so much better to label a piece of text as “code” or “userinterface” than it is to remember the exact formatting specifications. Moreover, it makes the document more consistent and provides an easy path to change formatting at a later date. As a first time author, I have no complaints about anything that makes my writing easier, more organized, and flexible.

Patronage in the Digital Age

Rob Oakes — Tue, 10 Nov 2009 18:20:22 +0000

As wonderful as the internet may be, it causes a lot of problems. For starters, it is putting newspapers out of business. It’s also radically changing how artists, writers and musicians make their living. And in case you weren’t paying attention, it’s starting to look like a crisis.

Different groups have responded to the impending collapse of publishing in different ways. Some writers sell sponsorships for their books and then offer an acknowledgement when it is printed. Many musicians have adopted the self-publishing and distribution tools long available to authors, leading to experiments like Amazon’s CreateSpace. And there are those who have gone the route of directly asking for contributions and donations to support their work; the digital equivalent of a performer passing the hat, you might say.

The problem is that some of these experiments are running head-long into good old American sensibility and propriety. There are even people saying that some of the new content generation schemes are inappropriate; including that old bastion of American common sense, Ms. Manners. Manners has even gone so far as to say that for a novelist to ask for a contribution is the same as begging, or panhandling.

She says it like it’s a bad thing. The simple truth is that artists, musicians and storytellers have long been beggars. The content industry of the 20 industry is a tremendously new invention, and as I noted above, it’s running into another time tested American value: frugality and a love of private property.

In fact, there seems to be this attitude that, “After I’ve purchased the novel or CD, I own the work and ideas. I’ve invested in its creation.” This little nugget rears it’s head most commonly when discussing music. Even the great Steve Jobs has been known to say, “People don’t want to rent music, they want to own it.”

Except … that’s bullshit. An interesting idea, or a well written book, or a beautiful piece of music isn’t like paying for a hamburger. You aren’t reimbursing someone for providing you a good or service. And I’m frankly shocked that anyone would think that Beethoven’s 9th Symphony or Antonio Vivaldi’s “Four Seasons” is only worth the price that paid on iTunes. The true worth is far greater than the price of admission. Would you seriously think yourself exploited for buying a second recording, or for paying to hear it at a concert?

Of course, that’s when people can be bothered to pay for content at all. An exacerbating factor is that many people expect ideas to be free or very inexpensive. How many times have you heard a variant of this argument, “I would buy more music (or books) if it wasn’t so expensive! Nine dollars for an album is just out of my budget!” Ironically, these same people don’t blanch at dropping hundreds or thousands of dollars for an iPod or iPhone.

While bad, this attitude can further devolve into something much more poisonous: “The artist owes me for reading, viewing or listening to their work. My piracy is helpful! After all, I am promoting them and making them famous!” But being famous doesn’t pay the bills. There have been many authors, artists or musicians who lived in squalor while enjoying enormous fame and prestige.

A music or literature pirate might even justify their position by saying, “I’m sticking it to the music industry (or publishing industry), they’re a bunch of greedy pigs!” And the pirate might have a point, if he weren’t doing far more damage to the creator of the content than to its distributor. Big businesses like record labels and big publishing houses don’t respond to that attitude by lowering prices or dealing fairly with their customers. Rather, they become more draconian in how that content is disseminated. Ever wonder why Digital Rights Management (DRM) and related technologies were born? It might just have something to do with the American sense of entitlement.

Clearly, something needs to change. Artists and musicians can continue to experiment with different pricing and distributions schemes, but I remain rather unconvinced that it will have a lasting effect. What we really need is a return to the patronage system of old, with a few major modifications. Certainly, artists should continue to sell recordings, books and other tangible goods. But the public should also undergo a shift in our attitudes and ideas about what the arts are and how we support them. That might mean that we transform our understanding of what a “donation” is.

When buying a book or donating to a writer, it’s foolish to think that you are somehow providing a fair compensation for the ideas and entertainment that you receive. Instead, it is much healthier to view your contribution as a support so that the artist can continue to create future content. This notion actually fits in pretty well with the concept of Fair Trade.

We also need to understand that the price we pay for a book or CD isn’t about the value of the materials. Textbooks aren’t expensive because they are printed on beautiful paper with artwork and in color; they’re expensive because researching and writing their content is hard. For example, the “Contributors and Reviewers” page for Gray’s Anatomy (the anatomical guide, not the television show) lists sixty different authors and content reviewers, though only the editor and chief is credited on the cover.

Except, how do you actually bring about the needed shift in attitudes and culture?

…

That’s an excellent question, and I’m not sure that I can offer any insight. The Europeans have tried to shape public perception through generous subsidies. But direct governmental support of news agencies and publishers is controversial for good reason. As a cure, it might even be worse than the illness. If you’ve got any ideas, let’s hear ‘em!

Statistics With R – Part 1: An Old Dog Learns New Computing Tricks

Rob Oakes — Mon, 09 Nov 2009 02:21:20 +0000

When doing math or numerical analysis, the knowledge of the technique is far too often tied to the tool performing the calculation. Consider an engineer whose understanding of the Fast Fourier transformation is inseparably tied to the fft function in Matlab. Of course this hypothetical engineer understands what the results mean (more or less) but may not be able to duplicate his analysis if Matlab were taken away.

In most cases, it is likely that no deeper understanding will be required. But what happens if the computer makes a mistake? Or the program becomes unavailable? Both situations are entirely possible. Computer algorithms aren’t perfect and occasionally arrive at results make little sense; and hardware has been known to fail.

When the engineer understands how the computer arrived at the answer, however, he can recognize, understand, and ultimately correct those cases where the results are unexpected. This is an important reality check that can prevent costly disasters later down the line. Or, if the hardware is unavailable, he can use an alternative tool or software package to duplicate the analysis.

But while such a situation can arise with any type of numerical software, it’s most likely to happen to users of a statistical package. I find this extremely ironic since a proper understanding of statistics is essential to live in the modern world. (Much more so than an understanding of the Fast Fourier transform, at any rate.) The rules of probability, the normal curve, correlation, and multivariate statistics can have a direct impact on how we live our lives. They are used in making important decisions in finance, medicine, science and government. A misunderstanding of stats and the methods of science (from which statistics is inseparable), underlies the most divisive issues of our day: abortion, stem cell research, and global warming.

Moreover, neither side has a monopoly on ignorance or misunderstanding. People fail to distinguish between correlation and causality, or insist in using the word “average” as a slur. Nearly as bad are those that – like the hypothetical engineer described above – only understand statistics within the narrow context of their stats package. Casual statisticians are nearly as dangerous as the wholly uninformed.

The Statistical Package for the Social Sciences (SPSS), is one of the biggest perpetrators of this crisis. Which is hugely ironic, because I happen to love SPSS. SPSS is probably the first statistical package that has placed advanced statistical methods within the grasp of the novice user. I’ve been a happy user for nearly a decade (ever since I was introduced to the program in high school). But there is no doubt that I’ve come to understand statistics within the context of SPSS and its GUI.

Please don’t misunderstand me, I have a pretty good grasp of basic statistics. I can sling probability with the best of them and take relish in describing when to use the Fischer Exact test instead of a Chi-Square; but advanced statistics are a completely different matter. Advanced stats scare me. I can certainly use these more complicated methods. I’ve analyzed and written about multi-variate models and even ventured into Analysis of Variance (ANOVA). But I have to rely on SPSS and the aid of my institution’s biostatistician to help me recognize when there is a problem.

Which is why, in a time of tight budgets, losing the institution’s SPSS license has been a crushing blow to my productivity. (Whoever made that decision should be hauled out and shot!) Because I don’t have my statistics software any more, there are certain aspects of my job that are much more difficult to do. And unfortunately, there is only logical conclusion to draw: I’ve become a victim of the statistical ease of SPSS.

Open Source Alternatives

I went through a similar experience about a year ago. At the time, I had become increasingly frustrated with the restrictions, licensing fees, and limitations of the Matlab technical computing language. After one particularly infuriating meeting, I decided that I had had enough and was going to do something about it. In the months that followed, I spoke with friends and colleagues, and experimented with every alternative I could get my hands on. I looked at Octave (the “Open Source Matlab”) and Ruby, before eventually settling on a combination of Python and PyQt to meet my needs. The result of these changes has been tremendously positive. Python is both easier to use and far more powerful than Matlab could ever hope to be. Not only am I happier and more productive, but so are those who work with me.

It is, therefore, logical that when I lost my statistical language of choice that I would look to open source to provide an alternative. Fortunately, the Open Source community delivers not one alternative to SPSS, but two: Gnu PSPP and R.

Gnu PSPP

As the name implies, PSPP has one simple goal: to clone SPSS in every way that matters. It can perform descriptive statistics, T-tests, linear regression and non-parametric tests. It has an easy to easy to use and relatively intuitive GUI. It can use SPSS syntax and read SPSS data files. It supports an obscene number of variables and cases (about a billion). It interoperates with Gnumeric and OpenOffice. Finally, it’s fast.

Aside from its horribly ugly icon, PSPP would appear to deliver exactly what I want and need. Except, you might have noticed that this article is titled “Statistics with R”, not “Statistics with PSPP”. Obviously, I chose to go with the second alternative. But why?

PSPP works as advertised. I found it able to deal with nearly all of the old SPSS data files and syntax that I threw its way. But, the program suffers from the problem of all clones everywhere: it’s greatest aspiration is to be a copy of something else. That is to say, it seeks to be “Good Enough”, and therein lies the problem, I don’t want a tool that is good enough. I want to use excellent software, even if it’s different or requires me to learn new things. Even if I have to pay for it.

I’m not trying to pick on or be unfair to PSPP. It meets an important need in the free software landscape. It just doesn’t fit in my with my desires or preferences very well.

The R Statistical Project

This is where R steps into the picture. Whereas PSPP is “aimed at statisticians, social scientists and students requiring fast convenient analysis of sampled data (emphasis added)”, R is the software that most statisticians actually use. When I contacted the statistician at my institution to ask, “What statistical software should I use? I’m looking at R and PSPP.”

He responded, “Oh that’s easy. Use R. There will be a learning curve, but it’s much more powerful and capable than even SPSS or SAS.”

As I’ve started to explore the feature set and available modules, it readily becomes apparent as to why. R is a huge language. There are thousands of packages that cover every type of statistics I’ve ever heard of, and many more I haven’t.

Even better, people have gone to great lengths to incorporate R into other tools. It has a set of excellent python bindings and interoperates very well with LyX and LaTeX. As just a single example, using the Sweave document class, you can use R to easily embed code in reports and other documents that need to be updated on a very frequent basis. This allows for these publications to be generated on demand with the most recent data. The only other place I’ve seen the equal to this feature is within the proprietary universe of Microsoft Office and SQL Server.

Easing Into R

Indeed, if R can be said to have a major weakness, it would be that it is too full featured and capable. Particularly for someone who is a statistical novice. The sheer number of packages and options available is absolutely overwhelming. Moreover, the reference material is distributed in nature. Like other open source tools, you can find answers to your questions; but you need to be intelligent about how you ask them.

As I’ve explored R, there have been quite a few painful moments. This isn’t because R is more difficult than SPSS or SAS, but rather because it is tremendously different. As an example, consider the differences in how the programs work with data.

Both SPSS and SAS use one main data structure, the data set. A data set can be thought of as a big spreadsheet where the variable names are kept in columns and the individual observations are kept in rows (SPSS calls them cases). In contrast, R uses may types of data structures. It has an a two dimensional array that is similar to the data set, but it is possible to use one dimensional arrays of data (similar to vectors), or three dimensional arrays (which might contain extremely complex data). The added options raise very complicated questions: What sorts of statistical calculations are done on a three dimensional arrays? Why are they necessary? How do I need to code my data so that I can take advantage of R’s advanced features?

Those aren’t issues that a user of SPSS or SAS even needs to consider. But with R, they present themselves before you even begin to use the program; and there is no centralized source of information to help you figure out the answers. The result of too many options and too little information is unproductive agony.

Series Introduction

Which is why I decided to write this series. R does some spectacular things, it is excellent software. But there are some things you need to know before using it. Here are just a few examples:

What user interfaces are available for R and which should you use?
How can you use R to summarize data and do basic comparisons?
How does R handle moderately advanced statistics like one-way ANOVA and non-parametric tests? What about mult-variate statistics and regression analysis?
How does R work with other programs? How can you format your output into publication quality figures?
What support does LyX and LaTeX offer for users of R?

The purposes of these articles are to address the concerns of the novice statistician or scientist. I will try and avoid jargon and other indecipherable terms. I will ensure that the examples are interesting and relevant. But most importantly, I will try and to help build a deeper statistical foundation. I know exactly what it feels like to be the “hypothetical” engineer who has become too reliant on his tools. As long as the tool is nearby, you’re fine. But when that tool is taken from you, be prepared for a world of hurt.

Until fairly recently, I’ve been in that world of hurt. You might just say that these articles are my way of explaining how I got out.

Customizing LyX: Create an NIH Grant Proposal Template

Rob Oakes — Mon, 02 Nov 2009 22:21:54 +0000

LyX is a wonderful writing program. It’s easy to use and produces beautifully typeset output. More importantly, though, it lets an author focus on the content and structure of his writing; rather than the formatting. It isn’t so easy to customize, though, which limits its usefulness in a big way. What if you need to create a new layout or take advantage of one of the thousands of specialized LaTeX styles? How, exactly, do you go about doing that?

That’s why this article was written. Recently, I was asked to help with a National Institutes of Health (NIH) R21 grant proposal. After some talk amongst the different investigators, it was decided that we would use LaTeX and LyX to draft it. Unfortunately, we hit a rather substantial hurdle early in the process: LyX doesn’t have an NIH grant template.

After additional debate, we decided to proceed with LyX anyway. But in the process, I found myself saddled with an additional job. In addition to responsibilities as research flunky and copy editor, I was tasked with creating a LyX and LaTeX template for our NIH grant. This article will summarize the steps I took and describe how to create a custom template using an available style on CTAN.

Note: All of the files in this tutorial can downloaded here (.zip).

Understanding the Big Picture

Before laying out the procedure, I will start with the theory. That includes understanding how LyX and LaTeX interact, as well as what files they use for their respective jobs.

An Introduction to LyX and LaTeX

As you probably already know, LyX is a document preparation system which supports LaTeX, DocBook, and other typesetting technologies. Writing with LyX is easier than using regular LaTeX or DocBook because it abstracts away the markup language and most of the associated pain. Want to insert a citation? Just hit Insert –> Citation. There’s no fussing with markup tags or macro commands. The same goes for inserting figure legends, tags, labels, footnotes or any other relatively complex formatting. LyX takes care of all the minutiae for you.

When you want to customize LyX, however, all of the ugliness and complexity gets shoved right back into your face. Instead of just writing and letting the program sort out the details, all of a sudden you are transformed from a user into a collaborating developer. You need to know how the underlying technologies work. What’s the difference between a LaTeX class and a style sheet? What does a LyX layout document do? How does that differ from a LyX template document? How do you install them and get the system to recognize that they’re present?

To get a handle on these questions, you need to understand that LyX and LaTeX accomplish two different (though complementary) goals. LaTeX is largely in the job of creating printed documents (typesetting), whereas LyX is about helping you to write and organize the document (on-screen display).

I like to think of LyX as an editorial assistant while LaTeX is the production staff. The editorial assistant organizes the various elements into a more logical layout, puts references and other material at your fingertips, and finally helps collate everything into a package that the production people can make sense of. Once the package has been assembled, the production staff then goes about laying it all out on paper, making sure that references are all numbered and formatted correctly, and otherwise handling the business of making the document beautiful.

LaTeX Classes and Styles

To go about its work, LaTeX uses a couple of important files: document classes (.cls) and styles (.sty). As mentioned, the entire purpose behind LaTeX is that you should be able to focus on the content of your writing without being distracted by its visual presentation. Classes and styles are what make this possible. As you are writing, you tell LyX (and by extension LaTeX) whether a block of text is a chapter, figure legend, table, or footnote. When ready, you can then transform the markup (which makes sense to a computer) into something that makes sense to a human being.

LaTeX classes are the first step in the transformation. They contain important information about what the structure of the document should be and where things should go. You might even say that a class file tells a document “what” it is. Subsequently, the most common types of document classes are articles, reports, and books.

A LaTeX style, in contrast, tells the file “how” it should look. If a class says, “You’re a book,” then a style will say, “you should have one inch margins” and “the chapter heading should be in Arial 16 point font with 0.2 inches of space above and below them”. Any given document will only have a single document class, but may use many different styles.

LyX Layout and Template Files

In contrast, a LyX layout file corresponds to a LaTeX class and tells LyX how to draw things on screen; and this is where things can get a little confusing. Even though a LyX layout is supposed to look like the printed page, this isn’t always true. For example, the final document may have two or three columns; but on-screen the LyX document will only have one. What is true, however, is that a LyX layout file will contain information about what document elements are available. If you’re writing a book, for example, the LyX layout will have an option called “Chapter”; even though the fonts and spacing used won’t always be exactly the same.

The final fie type that you need to know about is called a template. Whereas a layout file largely contains information about the on-screen display, the template file has information about typesetting and presentation. In many ways it inherits both the properties of the LaTeX .cls and .sty files and the LyX layout properties. This means that you can add both LaTeX and LyX information to it.

The figure below shows these relationships graphically.

Creating a Custom LyX Template – NIH Grant Proposal

With the definitions out of the way, here is the procedure for creating a custom document template. There are four basic steps:

Download the necessary document classes and styles
Install the files into the proper folder of your LaTeX distribution
Create an appropriate layout file corresponding to the document class or style
Create a template file with additional details and information

Step 1: Download the necessary styles and templates

The very first step is to locate the document class that you need to use. There are many good resources to do this. The first is CTAN, which is probably the most comprehensive resource available. (You can also find very good package lists on Wikipedia and through Google.) Since I already know the name of the document class I need (NIH), I will download it from the associated CTAN page.

Step 2: Install the files into the proper folder for your LaTeX distribution

Once you have saved the files onto your computer, you need to place them in one of several locations. If all users of the computer need it, install it to the system wide LaTeX tree. If, however, you are the only person who will be using the package, you can place it in your own “user” folder.

The location of the system tree varies depending on which version of LaTeX you happen to be running. It is typically defined by the “TEXMFLOCAL” variable. For individuals using Ubuntu Linux, you will find it at “/var/lib/texmf/tex/latex”. For users of other LaTeX distributions or operating systems, you will need to consult the documentation files. The “user” tree is located in your own user folder. For users of Linux, this will be “/home/username/texmf/tex/latex”. Note: Be sure to replace “username” with your appropriate login information.

To install the package, unpack the archive and copy all of the contents to the appropriate location. After, you will need to update the LaTeX database by running texhash.

sudo texhash

Step 3: Create an appropriate style layout for LyX

Now that the package has been installed, we need to create a layout file so that LyX can make use of it. This is a two step process:

Edit the document class and style so that it only contains general elements
Generate a new LyX layout file

Step 3.1: Edit the document class and style to only contain general elements

Working with LaTeX classes in LyX is slightly different than how you would work with them in a tool like Kile or TeXMaker. As already mentioned, LyX abstracts a lot of the technical detail of LaTeX away. However, when using a tool like Kile, you directly interact with the LaTeX code. When working this way, it can be convenient to place commonly used elements into the document class or style. The NIH files that we downloaded and installed in the steps above do this. (The page headers and footers contain the name of the primary investigator and the reference number for the grant.)

As you might guess, however, this isn’t a good thing if you are trying to create a general template. The document class and style files should only contain the elements that are going to applied to all of the documents. This shouldn’t include dates specific to a given revision, the name of a principal investigator, or the title of the grant. We want to install the LaTeX files and then forget about them, which is simply impossible if they contain such specific information.

As a result, before we go further, we need to redact this information from the nih.cls file. But even though we are removing it, don’t throw it away! Stick it in a spare text file for the time being. It is still needed so that the final document will match NIH style conventions. We will add it back to the template created in Step 4.

In the original files from CTAN, open up nih.layout in your favorite text editor and cut out the following text:

% preamble stuff
\newcommand{\nih@PIname}{Oakes, Robert Stilman}
\newcommand{\piname}[1]{\renewcommand{\nih@PIname}{#1}}
\makeatletter

% constants
\newcommand{\nih@sillysize}{\scriptsize}

…

\newtheorem{proposition}{Proposition}
\newtheorem{lemma}{Lemma}

In addition to redacting the information above, you will also want to add the following line close to the end of nih.cls. It will ensures that alphabetic section labels will be used instead of numeric labels:

\def\thesection{\Alph{section}}

Note: In the files for this tutorial, I have already redacted the unnecessary text from the nih.cls file and added it to the NIH Grant Proposal template. I have also added the section label code.

Step 3.2: Generate a new LyX layout file

The easiest way to create a new layout file is to modify one of the existing examples. If you look closely at the NIH document class (nih.cls), you will notice that it is based on the LaTeX “article” class. As we try and create an appropriate nih.layout, let’s start by modifying article.layout.

When you first open the file, you will see the following:

# Do not delete the line below; configure depends on this
# \Declare LaTeXClass{article}
…
Format 11
Input stdclass.inc

SecNumDepth             3
TocDepth                3

NoStyle Chapter
NoStyle Chapter*

Style Part
    Align                 Left
    AlignPossible         Left
    TopSep                2
    BottomSep             1.5
    Font
      Size                Larger
    EndFont
End
…

As you study the layout, you should start to see a few general sections. For example, the second line tells LyX that it is using the LaTeX class “article”. “Input stdclass.inc” tells LyX to import the standard list of document elements. Whenever you see a line that begins “NoStyle”, that is telling LyX to remove those particular section types from the available list of options since they aren’t supported by the document class.

Pay close attention to the section block that begin with “Style”. This tells LyX how to display that particular document element on screen. It includes information about how the text should be aligned (“Align”, “AlignPossible”) and what font should be used (“Font”, “Size”, “EndFont”).

To make article.layout work with nih.cls, we need to make two modifications. First, we need to change the \DeclareLaTeXClass and then we need to add a new “Style” block.

First, let’s change the \DeclareLaTeXClass. Add the following:

\DeclareLaTeXClass[nih]{proposal (NIH)}

In this example, nih is the name of the LaTeX document class (nih.cls) and the information in curly brackets is a description which will make it easier to find in the LyX document settings pane.

Next, we need to add a “Style” block that describes the “Section” element of the nih document class. By default NIH sections are labeled alphabetically: “A Specific Aims”, “B Background”, “C Methods”, etc. In contrast, the article labels its sections numerically: “1 Specific Aims”, “2 Background”, “3 Methods”, etc. If we don’t modify the standard labeling, it will be displayed incorrectly on screen

Luckily, it’s an easy fix. Add the following to article.layout:

Style Section
    LabelType             Counter
    LabelCounter          section
    LabelString           “\Alph{section}”
End

After you’ve made these changes, resave the file as “nih.layout”. Then copy the file to either the main LyX directory layouts folder (typically LyXDir/layouts) or to your LyX user directory (typically found at .lyx/layouts/). For LyX to see and use the new files, you will need to run Tools –> Reconfigure.

Note: A full version of the nih.layout file is available as part of the archive package for this tutorial.

Step 4: Create a LyX template file

With a fully functioning LyX layout (nih.layout), you can now create a template file that has all of the needed elements. As mentioned above, a LyX template file is nothing more than a regular LyX file. So, start by creating a new LyX document (File –> New).

After the new document is open, the very first thing that you should do is change the document class. Go to Document –> Settings –> Document Class. Then from the drop down menu, select “proposal (NIH)”.

Next, you need to add the information removed from nih.cls to the LaTeX Preamble. Go to Document –> Settings –> LaTeX Preamble, and paste the text into the provided space. While doing this, you may want to modify the name of the PI that will be shown in the document header. You make this change by adding your name to the line that reads:

\newcommand{\nih@PIname}{Last Name, First Name Middle}

When done, press “Close”. You now have a fully functional, NIH compliant grant proposal template.

But while that’s the end of the essential boilerplate, you may want to include a few additional goodies. Wouldn’t it be nice, for example, if you had access to clickable hyperlinks? Or if you were be able to use the bookmarks sidebar to navigate the document? For these and other desirables, add the following to the preamble:

% Hyperlink Options
\usepackage{ifpdf} % part of the hyperref bundle
\ifpdf % if pdflatex is used

% Use True Type Fonts Instead of Older LaTeX Fonts
\IfFileExists{lmodern.sty}{\usepackage{lmodern}}{}

\fi % end if pdflatex is used

% for correct jump positions when clicking on a link to a float
\usepackage[figure]{hypcap}

This code enables the hyperref package. Then, go to Document –> Settings –> PDF Properties and enable hyperref support in the options. While changing the PDF properties, I personally like to customize the colors of the links. I think that document links and cited references should be black; while urls and files should be blue. To make this change, add the following to the “additional options” line:

linkcolor=black, citecolor=black, urlcolor=blue, filecolor=blue

Using the Template

If you’ve followed all of the steps in this tutorial, you will now have a fully functioning NIH grant template. Now might be a good time to save it in the same location where you keep other templates. (If you don’t have a templates folder, you might want to create one.) I’ve saved mine as “NIH Grant Proposal.lyx”. From here on out, whenever I want to create a new NIH document, I simply choose New from template from the File menu and load the appropriate version.

Now that my template is set up, I’m back to an automated wonderland. LyX and LaTeX keep track of the references, labels, footnotes, formatting and the other minutiae that make complicated documents so painful. I’m free to focus on the ideas, voice, and message; which is exactly how it should be.

Dark Times for the Book Publishing Business

Rob Oakes — Tue, 16 Jun 2009 23:08:09 +0000

The publishing industry is in trouble. Sales of both books and newspapers are down, and even enormous powerhouses like the New York Times have found it necessary to address the once unthinkable. While it would make a better story to have a villain who can be nobly engaged and defeated, the fall and demise of the publishing industry can overwhelmingly be attributed to one factor: technology. Or more specifically, to the changes in behavior and habits that technology and the internet have brought with them.

It is, thus, ironic that many publishers are looking toward technology to solve their financial ills. And they don’t lack for places to turn: online search schemes, electronically disseminated e-books, subscription based web-pages are but a few examples. However, Amazon is preparing what may might actually be a savior to the beleaguered industry, electronically delivered content on the iPhone and Kindle.

Yet, at the same time that iPhone and Kindle provide access to new markets for publishers to peddle their existing wares, they also bode poorly for the future of the publishing powerhouses. The rise of the Kindle and the technology/distribution system it represents also holds the potential to irreversibly damage the relationship between publishers their most valuable asset: authors.

Here’s the problem, it’s very hard to make a living as a writer. I know, I tried. You have to be incredibly good at communicating and even better at self promotion. To actually be successful requires that you rise above the competition, and there’s a lot of it. Literally tens of thousands of people complete the “Great American Novel” each year, and any author who wants a break is competing with all of them. The goal? Land a contract with a publishing house. It’s their role as gatekeeper which has made traditional publishers the guardians of fame, fortune and success.

Electronic books in general, and Amazon in specific, have the potential to radically change this arrangement. Sure, Amazon is providing new markets for established publishers, but they also open those same opportunities to new authors. This creates an interesting conflict: in many ways, self-publishing through Amazon CreateSpace or alternative services like Scribd.com is a better deal than pursuing a traditional book contract. Self-publishers control the intellectual property rights to their work and make more money per-sale. Further, using Amazon’s services provides instant access to the eco-system and to the Kindle. And other technologies: email, blogs, and social networks have eroded the value of marketing and other support services that publishers have traditionally provided.

This might be a good time to ask, “Is this a good outcome?” After all, isn’t it better to have a larger selection and more new authors to choose from? Diversity and competition favor consumers and readers.

But let’s be honest, diversity of content isn’t really a problem. Whatever your interest, religion, or politics; you can find something to read which fills it. Are you a raging, America hating radical? Well, don’t worry, there are web-sites that have you covered. Ditto for closet-philosophers, back-seat mathematicians, and basement engineers.

In fact, the internet has made it absurdly easy for anyone to become a celebrity and propagate ideas, even unintentionally. Want to be a broadcaster? All you need is a camcorder and an abundance of free time. YouTube provides the rest. But unfortunately, just because you can find something doesn’t mean that it is worth anything. Writing is no different.

If publishing only served as a vehicle for content creation, then its death would hardly be a tragedy. In many ways, it might even be desirable as Amazon and others are positioned to provide the same services more efficiently and cheaply. But such a view overlooks another role that publishing houses have served: that of collaborative facilitator. Contrary to some opinions, a good book is not solely the product of a single mind, but may include the input and advice of many. So while some support services, like advertising and illustration may be available to self-publishers, the critical insights of experienced editors often are not; and they are essential. Some of the most promising fiction squandered its potential because the author was given far too much creative freedom.

There is a secondary point that is also important to consider, while publishers have served as fastidious gatekeepers; once you have passed through their gate, it was possible to make a decent living. That dynamic changes radically in the internet’s decentralized marketplace. Instead of convincing one person of your worth, it is instead necessary to convince hundreds of thousands, if not millions. So while the potential to earn more money is present, it is more difficult to actually do so. Worse, every other self-published author is rigorously competing for the attention of the reading public, which further amplifies the noise. And just because more options are present does not mean that people will spend more on books or spend more time reading. Very few have had breakthrough successes through self-publishing, and those that have are not necessarily exemplars of literary quality.

So, what’s the net result? Even fewer live the dream of the American novelist. While they might make some money, it is unlikely any will be able to quit the day job to pursue writing full time. In such a distributed market, no one ever really arrives. Many people make a little bit of money, but no one makes a lot of money. Those that are best served by Amazon’s publishing model already have an established and trusted brand. A fantastic example of the rich getting richer and the poor getting nothing.

This is a bad thing for writers and most especially for new authors. The changes in the marketplace might even mean that writing evolves into a supplement of other careers, rather than a career in its own right. Wouldn’t it be terrible if the William Shakespeare of the next generation were trapped in dead-end job with the fear that s/he would never be able to make a livelihood via her/his writing? The destruction of the publishing business might mean just that. More but not necessarily better, which means that dark times might await both publishers and authors.

Introducing LyX-Outline 0.1

Rob Oakes — Wed, 13 May 2009 22:16:12 +0000

Writing anything – whether it be a book, or a short story or an angry letter to your boss – is substantially more than starting from the first idea moment of inspiration and continuing to the final draft. Rather, writing involves a fair number of idea fragments, fleeting moments of inspiration, and a tremendous number of dead ends.

It is incredibly unfortunate that most writing software, however, is geared to organizing and structuring the document after most of the hard work has already happened. It simply assumed that most of the planning and layout has already happened and the author is ready to string words together. Unfortunately, this assumption overlooks one important truth: ideas are best defined as they are expressed. Thus, it’s usually about the time that the a writer sits down to compose the draft, that the document’s true structure becomes apparent. In my own case, this often leads to a flurry of reorganization. And it’s during the restructuring that the real battle begins.

When in full creative passion, I am typically working with three or four different programs all at the same time: OneNote is open so that I can access my ideas, Word is there to start collecting the somewhat finished text, and I’ll also probably be using a MindMapper so that I can see a visual representation of the document structure. The tools are separate and don’t communicate with one another. Thus, a change made in one place needs to be made everywhere. And all too often, that I end up fighting the word-processor and the notetaker, and the mind mapper. It is tremendously frustrating to battle the tools of your adopted trade. Luckily, I am not alone in my frustration.

In the past everal years, a number of programs have become available that leave the linear model of writing behind. On the Mac platform, one such tool looms above the others: Scrivener. Central to Scrivener’s function are two important metaphors: that of the outline and that of the corkboard. And it works really, really well. There are just a few minor problems. First: Scrivener is only available for Mac and Scrivener’s lead developer has made it clear that there won’t be versions for other platforms. Second: Scrivener was really designed with creative writing in mind. Thus, while it can be used for long and complicated documents, this is a slightly less than straightforward process. Last, to create said fancy documents, Scrivener requires the raw use of a markup language (and all of the associated headaches that come with it).

The document processor, LyX, however, excels in many areas where Scrivener falls short. It is built upon the mature and robust underpinnings of LaTeX, the typesetting language of choice in the science and engineering. And more importantly, it is easy to use (where LaTeX most decidedly is not). But it fails in the same way as Microsoft Word and other word processors, it is a linear writing tool and doesn’t offer a great deal of work-flow flexibility. That is where LyX-Outline comes in.

LyX-Outline is a marriage between Scrivener’s organizational tools and LyX’s typesetting tools.

Looking for Flexibility

While you can download and play with LyX-Outline, please reember that is still a forthcoming add-on to the LyX. And while you find it useful, this release is an alpha level technical preview. Right now, you can view the structure of your document at a glance in the corkboard, move things around, poke about in the outliner, and even jot your winded thoughts with the very basic plain text editor.

But even though some of the functionality is lacking, you can start to explore. And as you do so, one of the first things you might notice is that LyX-Outline has been designed with flexibility in mind. Nearly all of the tools can be docked, ripped away, or hidden if they are not needed. The work environment should adapt to your work habits, not the other way round. Whenever the writer needs to adapt to the environment, a moment of productivity and a spark of life has been lost.

A Quick Overview

The main interface consists of three main elements. This includes the document map, the editor pane, and the dock tools. As described above, the document map and the dock tools can be relocated, docked or left free floating (depending on your preference). And if you don’t find them useful, they can quickly be hidden.

The document map lets you see a hierarchical list of every item in your project. Selecting one of the items displays its content’s in the editor pane. The editor pane is the main text processing portion of LyX-Outlineand can be toggled between a text editor (plain text only at the moment) and a corkboard view.

In addition to the main editor and corkboard, there is an additional corkboard and outliner tool available as a dockable widgets. Just as in Scrivener, you can view the pieces of your documents in many way. Every item is a document, and an index card, and an outline point. They are intimately tied together and any change made to the order or content of one will be reflected everywhere. The document map is used to navigate the content of the main window.

The dock tools, however, can be controlled separately. You can view the entire document, or filter to a specific level in the hierarchy via a drop-down menu. In the future, filtering by search term, keyword, document structure, or tag will also be possible.

Item Summary and Organization

The health of any written work can also be greatly improved by connecting a simple synopsis to a larger chunk of text. This makes it much easier to check the flow of your work at a glance . A summary can easily be added to any individual item in the collection by double clicking on it’s index card in the corkboard or the synopsis column in the outliner. The summary text is independent from the full text and can include pertinent information that helps you structure the document. For now, the synopsis must be entered by hand; however, future versions will allow for automatic generation of the synopsis from the text.

Of Outlines and Index Cards

Scrivener (and LyX-Outline as it’s dedicated clone) utilizes multiple metaphors. The Corkboard may be useful in some instances, but will be less helpful in others. Ditto for the outliner. Either way, it is important to know that both the corkboard and the outliner display sub-items. If you are using the main corkboard, these sub items will be connected to the active item in the document map. If you are using the dock tool, they can be filtered via the drop-down menu. From the corkboard, you can then edit items by double clicking on either their title, or their summary. You can reorder them through drag and drop. From the outliner, you can add new items or sub items, or delete them from the document. You can also change their order (by using the up or down arrow buttons) or change their indentation level (with the left and right arrow buttons).

Minding the Potholes

And that is a very quick tour of the Outline modules for LyX. As noted above, this release of LyX-Outline is a prototype and it is still very far from its intended target. Thus, if it doesn’t meet your needs right out of the box, please be patient. The mantra of OpenSource is "release early, release often." In this case, I have erred on the side of early and as you might suspect, there are some dangerously rough edges. These include:

Unfinished interfaces. Most of your manipulation will probably need to happen in the outliner pane. This includes adding or removing new document nodes, as well as indenting or un-indenting the document selection. The final version will include ubiquitous drag and drop.
A crappy editor. The current editor is plain text is a placeholder only. Ultimately, LyX-Outline will be integrated with the LyX editor and have access to all of it’s very powerful features. It is important to keep that future in mind.
No export option. While you can save and open past documents, there is currently no way to actually get them out of the program.

In it’s current incarnation, this prototype is valuable for pecking out thoughts and playing with the organizational tools. I am releasing it so that people can see how the tools might look in their final form and provide feedback. Please do so. Download the source, use it as you can, and let me know what you think. What things work? What things don’t? What features would you as users like to see?

Update: You can download the source code and find installation instructions on the LyX-Outline project page.

Anathem: Big Book, Big Ideas

Rob Oakes — Mon, 09 Mar 2009 16:54:19 +0000

What can you say about a Neal Stephenson novel? Really. The man is a bit like J.R.R. Tolkien, he feels the need to go out and re-invent the wheel simply because he can. As a result, no amount of critical analysis, commentary, or old fashioned smack is really able to do his work justice. If you want to experience a Stephenson novel, you just have to go read it.

So it is with Anathem, a book about a place which isn’t Earth and a time that isn’t now. Though it certainly feels like both. Anathem is s a big book which contains big ideas: the observations of classical philosophers, rules of logic, and ultimately a polycosmic theory of connected reality. You know, light reading. It’s also a brilliant though extremely frustrating piece which simply defies any attempt at summary. The first third is spent drowning in detail, the middle third in quiet contemplation, and the last third in monumental disillusion.

Of Physics and Philosophy

After a few pages, it becomes clear that Stephenson is a philosopher. I’m not sure if it is the fifteen page description of a big clock, or the ten page description of how it’s wound. Nonetheless, there is a tremendous devotion to the mind’s perception of reality. In fact, the entire cosmos of Anathem is meant to glorify the rational and introspective. Imagine, if you will, what would happen if classical Greek culture had never been absorbed by the Romans and later disseminated into Roman Catholic or Byzantine thought? Instead, it had retreated behind monastic walls and allowed to flourish? What would that look like?

The Mathic Orders represent a tremendously good guess: communities dedicated to the pursuit of philosophy, as the ancient Greeks understood the term. A haven for those that seek to understand existence, knowledge, truth, beauty, justice, validity, mind, language, and logic. The avout are, literally, “the lovers of wisdom.” This is different from the pursuit of science, but rather the pursuit of knowledge in its purest form: theory.

In many ways, Stephenson’s dedication to rational philosophy drives a great deal of the ideological conflict. So while you can rest assured that religion versus science will make an appearance, there are shadows of far nastier clash: empirical evidence versus theoretical prediction. Remember, the sixteenth century brouhaha between Galileo Galilei and the Roman Potentate was not a fight between religion and science per-se, but a scuff between meta-physicists (philosophers) and their empirical colleagues. After all, the geocentric view of the cosmos was not a Christian belief, but a Greek one; it can probably be tracked back to Plato or even Socrates, though Aristotle is responsible for it’s place in Western canon. Galileo’s struggle wasn’t with the theists, but rather with his scientific colleagues. Similar fights have been waged on a far larger scales with greater consequences. What is Communism, for example, if not a magnificently beautiful and utterly failed theory?

Characters, Plot and Ideas

Here’s the plot summary: aliens show up for dinner, how should the world respond? That’s it, spread out over 900 some pages. And while Anathem pulls off a reimagining of “Close Encounters of the Third Kind,” it suffers from more than stale plotting. The characterization is generally terrible, in fact, most of the characters aren’t really all that likable or interesting. Actually that’s not fair to the variety of boring and non-likeable people who populate the worlds of the polycosm. Let me be more clear: nearly every character falls somewhere on the likable scale between mostly unpleasant and raging wanker.

What’s worse, I found that most were completely interchangeable; like gears in a watch or machine milled parts of a car. Sure, there may be an assortment of adolescent pricks, older pricks, computer pricks, and token religious nuts; but the characters play stereotyped roles and never give any hint of being rounded individuals. As long as there is a character in the right place to say the right things, that’s good enough.

For example, consider the “deolators” of Anathem. They get about as much respect as a Scientologist on a bad day. When he is feeling kind or generous, Stephenson dismisses the deolators and their motives as unfathomable. More often, though, they are linked to a substantially less flattering and destructive connotations. In Anathem’s nine-hundred pages, expect the word to be used interchangeably with misguided, superstitious, and stupid. In the six or seven examples of apocalypse, Stephenson was kind enough to credit the deolators for all of them. Apparently religion is to blame for all of society’s ills. Silly me, I thought that was the role of politics.

Don’t expect a consistent set of rules for anyone, however. Based upon Stephenson’s clear distrust of religion and its notions of “truth,” it is ironic that most of the other characters simply assume that they are right. Indeed, I found the general sense of self-satisfied superiority to be more than slightly obnoxious. After all, why should the core characters bother to question their own place in a recently rearranged cosmos? It’s not like the understanding of polycosmic existence isn’t about to be thoroughly challenged. Though I’m probably being a bit harsh, Stephenoson did manage to nail one stereotype: most of Stephenson’s “avout” are carbon copies of the dry and humorless academics I’ve worked with.

The surprising bit, however, is that the pathetic characterization doesn’t derail the book. After I realized that individuals don’t matter, I stopped keeping track of them. Instead, I started to follow the ideas; and that’s when Anathem became really enjoyable. The concepts, philosophical musings, and theories are the real stars.

Be prepared for information deluge, though. There is enough detail to either fascinate or thoroughly repulse. First off, the book is written is Socratic dialogue: this means there will be smug questioning, logical maneuvering and thorough pontification. In fact, when reading formal dialogue, keep one important fact in mind: there is a reason why the ancient people of Athens “allowed” Socrates to consume a fatal dose of hemlock, probably just before threatening to throw him off a cliff. At best, he was a bloody nuissance; more often, he actively undermined social order and stability. Plato may remember him fondly, but there is little in his thirty-five surviving works to say that he remembered him accurately.

Even so, the formal dialogues are easily the best parts of the book. Clearly, they are also the parts which Stephenson was most interested in writing. The care with which they have been laid out and composed is simply stunning. Much better than any of the actual “character development” or plotting. The ideas are presented logically, carefully and understandably. A few examples, like Stephenson’s description of systemic evolution (the Bat, Fly, Worm dialogue) are worth the price of the novel by themselves.

Unfortunately, even with the beautiful ideas, Anathem is stilted. There are so many times that a gorgeous ideas is given a grotesque application. To use Stephenson’s language, the theorics and praxis simply don’t line up. After hundreds of pages and thousands of words of logical dialogue, the conclusion is illogical. Theory and empiricism are fighting it out again and the result is polycosmic miasma. Not a good way to conclude a book that has been marvelously rational.

Conclusion

Okay, so I’ve been harsh. The simple truth is that I enjoyed many parts of Anathem thoroughly. I started off by listening to the audio book version, but later bought a hardcopy for closer inspection. That says something. I don’t often pay for a book twice.

But even so, this book only reinforces a previous conclusion: Neal Stephenson is a clever fellow. Unfortunately, he feels the need to beat us about the head with it. He brilliantly plays with the whole of human history and explores some fascinating theoretical possibilities, but then comes off as an utter ass.

Most very intelligent people, unfortunately, are little more than provocative gasbags. Far too often, “clever” is allowed to pose as brilliant and cliché to stand in for innovation. As a result, Anathem isn’t so much a story which raises a number of interesting ideas; rather, it’s a philosophical treatise masquerading as a piece of speculative fiction. If read as the first, it’s hopelessly and frustratingly flawed. When read as the second, however, Anathem is one of the most interesting books to appear in quite some time.

Creating the Perfect Writing Tool: A Proposal

Rob Oakes — Thu, 05 Mar 2009 02:32:33 +0000

In addition to nouns, verbs, and adjectives; there are other tools which supplement the writer’s arsenal. some writers, this is a moleskin notebook and ballpoint pen. For others, writing is synonymous with the usage of computer word processors, such as Microsoft Word. What is a given, however, is that the choice is intensely personal. If you ask ten different writers, "What is the perfect tool?" Expect ten different answers.

For those who create software, however, this is big problem. Programs can’t be customized to the individual whims of every writer. (A more diverse or idiosyncratic group does not exist.) It is necessary to select a cross section of features that meet the needs of most people. Unfortunately, however, this has resulted in some serious compromises and an emphasis on the technical process of writing, often to the exclusion of the creative process.

The Problem

Like most writers, I have some strong feelings about the tools I use; and like most writers, I have never found one that fits my needs. But this is not for a lack of trying. In fact, I’ve been searching “perfect tool” for a very long time, at least since I was a senior in high school.

At that time, I enjoyed writing things out in longhand, with separate drafts copied over from the first iteration. I fancied myself a budding novelist and fervently believed that such an exercise would stimulate my creativity.

A decade later, however, I have become impatient. Now, I prefer to peck out my thoughts at the keyboard. Sometime in college, I realized that writing by hand is slow and while it can focus the mind on word choice, it’s a plodding and frustrating exercise. In contrast, typing at the computer is quick; and since I can type nearly as fast as I can think, it allows the text to keep pace with my thoughts.

Moreover, working on the computer is organic. There isn’t a discrete first, second, and final draft; instead, I can add, modify or change my writing as I form the ideas. If I want to jump back and change a word or sentence, I do so. Pen and paper isn’t nearly so adaptable and many ideas are lost before they can be captured and expanded.

The Limitations of Existing Tools

But even though I prefer to write at the computer, I haven’t been able to find a writing program that lets me creatively adapt the structure and flow of my document. I have tried nearly everything available, but nothing fully meets my needs.

Consider Microsoft Word, for example. Even with all of its power, there are a number of things that it doesn’t handle very well. Ever tried to work with a complex Word document, numbering into the hundreds or thousands of pages? If not, I don’t recommend that you do. Large Word documents cause the program to slow down or crash and they have a tendency to become corrupted. There is nothing more disconcerting than finding out one of your book chapters is no longer usable due to bit rot.

Introducing LyX: The Document Processor

Luckily, there may a potential solution. The open source document processor, LyX, meets many of Word’s shortcomings. LyX encourages people to approach writing based on the structure and content of the document, rather than the appearance. Thus, it isn’t a "What You See Is What You Get" so much as a "What You See Is What You Mean" (WYSIWYM).

When used right, LyX focuses the user on writing the content. The program handles the mechanics of layout and output. To do so, LyX leverages the LaTeX typesetting system, which is used heavily in the engineering and science disciplines to publish dissertations, grant proposals, journal articles, and books.

When I first found LyX three or four years ago, it fundamentally changed the way that I worked. Before, I manually formatted everything in a given document. While this approach is okay for simple letters or lists, it quickly breaks down in anything longer than a few pages.

In contrast, LyX makes extensive use of tags that tell the program what a given piece of text is. The title is enclosed by a "title" tag, book sections by the "section" tag, chapter headings by the "chapter" tag, block quotes by the "quote" tag, and paragraph text by the "body" tag. The program then applies formatting rules globally and in a consistent manner. Text within LyX has a meaning and will be processed in a specific way.

And thus we have the first important component of a perfect writing environment: the program is aware of what content is. Such awareness allows for extremely complicated written pieces to be assembled in a relatively easy fashion.

A Sloppy Process

Despite its strengths, though, LyX is an incomplete solution. It focuses on the mechanical aspects of writing and ignores what is arguably the more important component: the creative process. Using LyX, I can assemble a true monstrosity of a manuscript replete with footnotes, endnotes, references, tables, lists of figures, indices, appendices, and all of the other components of something unreadable. Yet such elements are mere after-thoughts rather than the heart and soul of the work. The center is all about ideas.

Which brings me to the serious limitations inherent in most word processing programs: joining ideas together is a sloppy process and they simply don’t facilitate the sloppiness.

Every writer is creative in a different way. I personally start by making a miscellaneous brain dump. Everything relevant to the topic is listed out and then I start to type additional thoughts which seem relevant. After a while, I run out of steam which prompts research, reference collection, reference reading and chicken scratch notes. Then I groan, swear a little, go for a walk, and kick ideas around in my head.

When I come back to the computer, I move things around even further and excise any bits that now seem irrelevant (carefully saving them for later). Usually by this point, I have probably started over a few times and hopefully clarified my goals. If the process works, I get a slightly better fix on direction I intend to take and, eventually, a first draft magically seems to appear. That is when the real fun starts.

Unfortunately, LyX was designed to work linearly. You start at the beginning and continue to the end. All of the words and ideas are supposed to come marching out like perfect little soldiers on parade day. Using such a tool can be extremely frustrating to someone who doesn’t think in a linear fashion: I want the freedom to be intuitive!

And in the name of freedom, I will probably use of four or five programs spanning two operating systems: OneNote for data collection, KDissert or FreeMind for MindMapping, and Word or LyX for the actual writing. Then I go back and update my files in the other programs.

There should be a better way! I want to use the same tool from beginning to end, not continually move between the word processor, the outliner, the mind mapper, and the note taker.

A Potential Solution: Scrivener

There just happens to be a program on Mac OS X that was apparently made for scattered people: Scrivener. Scrivener employs a number of simple (yet profoundly powerful) metaphors that let you manage the anarchy of creativity. These include:

An intuitive and simple map of the project (similar to Word’s Outline View).
A corkboard that lets you create any number of "index cards" with a title and short summary of the card’s contents.
An outline view where the entire structure of your document can be seen and manipulated through drag and drop.

Scrivener leverages its corkboard metaphor and outliner by linking the "index card" to a real piece of text. Thus, the index card actually represents the physical chunk of text in the underlying draft. The index cards can be moved, nested and set into folders and any changes made to the index cards are then instantly applied to the overall structure of the document. Pretty cool, huh? A given index card can represent a section, paragraph, chapter, or sentence. Even better is that cards can be split or joined together easily.

The practical result is that the entire draft is instantly fluid. You can experiment with new ideas, split away sections of text for later use, and re-arrange things as inspiration strikes you. It substantially enhances the organic process already provided by the computer. I know a surprising number of people who have moved to the Mac platform just so that they can use Scrivener.

While other programs (like Semantik, formally known as KDissert) use similar metaphors, the use of a "summary" field is what makes Scrivener stand apart. The summary begins life as an outline point and later becomes the synopsis of the section. It can keep notes, goals, or thoughts in addition to a regurgitation of what is in the text. And no other program uses anything like it. Therefore, while you can replicate many of Scrivener’s best features through Word or LyX’s outline view, you are stuck manipulating the entire text (which becomes impractical and unwieldy very quickly).

The Outliner. Scrivener includes a high quality outliner that allows individual elements to be repositioned via drag and drop. Each outline item is connected to the text in the actual draft; therefore, changes in the outline view are immediately reflected in the structure of the document. Scrivener’s outliner is particularly effective via the inclusion of a "synopsis" field that contains a summary of the text. This summary field isn’t part of the full draft, and as a result, can begin life as an outline point that reflects the goal of the section. In contrast, the outline view in Microsoft Word uses the text of the actual document. In long or complicated documents, it is a slightly less than effective way to add new ideas.

The Corkboard. As an alternative to the outliner, Scrivener also provides a "corkboard" where parts of the document can be moved around on index cards. Each card has a title and summary that is is linked to a block of text in the document. Unlike the outliner, however, the corkboard only deals with one navigational element of the document. When at the top level, the cards might represent the parts or chapters of a book. When working on the corkboard of a book chapter, they might represent sections or individual scenes. There is no limit to the number of cards or nesting that can be done.

Scrivener’s Singular Strength

Though Scrivener handles the creative process beautifully, it isn’t a serious word or document processor. On the program’s website, the primary developer states that Scrivener is a "personal writing tool." As a result, it seems uniquely geared to creative writing; that is to say, novels and screenplays. Not many novels or screenplays have a need for a bibliography, index, appendix, footnotes, or endnotes. As a result, Scrivener cannot handle the minutiae of complicated documents.

Just forget support for hyperlinks, cross-links, cross-references, figures, legends or graphics. They’re outside the developer’s stated need and are unlikely to appear. That’s too bad, because it makes Scrivener worthless to those who need support for these features. Microsoft Word has them and LyX has them, OpenOffice pretends that it has them. However, Word, LyX, OpenOffice and other word processors completely lack the creative tools that make Scrivener a joy to use.

A Glorified Text Editor. The weakest component of Scrivener is the actual word processor. Advanced (and even not so advanced) features like footnotes, endnotes, cross references of links are completely unsupported. It is simply unsuited to writing a complicated document like an article or cross-referenced book.

A Combined Approach

The simplicity of Scrivener’s word processor causes a semi-crisis. If given my choice, I would compose nearly everything in it for the convenience of its outliner and index cards. However, it isn’t up to supporting the type of heavy lifting I am often asked to do. While I could use it as my creative crutch, that worsens the overall problem. Instead of four programs across two platforms (Window/Linux), I would be using five programs across three platforms; not what I’m aiming for. I want to simplify my work process, not make it more complicated.

I’ve tried to find what I want in the software of others and determined that it doesn’t exist. The fallback is to develop it myself by turning to opensource solutions. Since LyX already meets my needs as a document processor, it is a great place to start. As a front-end to LaTeX, it automatically supports the advanced features I need. It is cross-platform, which means that the writing environment would be the same regardless of whether I am working on Windows, Mac OS X, or Linux. Additionally, it has deep integration with revision control systems, which greatly simplify collaboration. Last, it is available under the GNU public license, which means that I can access the source code. LyX combined with the same creative tools found in Scrivener would be nearly perfect.

Almost There: LyX’s Existing Features

After a careful review of LyX, it becomes obvious that much of the functionality required for a robust outliner and corkboard is already present. At left is the outline panel in LyX. The outline is automatically generated from the heading information in a given document and any child documents. The particular example shown here is series of three essays kept in four different files: a master document and three child documents.

LyX already recognizes blocks of text and understands their contextual meaning. In fact, you can already specify the order of any element and what position it holds. Further, individual text elements can be moved from one child document to another. In other words, nearly all of the background elements for a Scrivener-like outliner or corkboard are already present.

Form and Function

What is missing is the ability to assign a summary or "synopsis" to a given chunk of text. This would ideally exist as associated meta-data and affiliated with a particular tag. Additionally, it might prove convenient to associate other meta-data in addition to the summary.

To add the "creative elements," most development could focus on creating a user interface that blends the existing outline view with the functional equivalents of the corkboard and outliner from Scrivener. Convenient drag and drop of document sections from within the document map, outliner, or corkboard would allow text to be moved and positioned. The outliner would show the complete document with the option to expand or collapse a given section. The corkboard, in contrast, would show only the elements contained in one particular outline node.

The figures below show an example of what such a user-interfaces might look like. The first figure shows the outliner and the second shows the corkboard. (Though the index cards and outline points of the example correspond to the chapters and sections of the document map, this should not necessarily be required in an ideal implementation. Instead, the user would be able to determine which index cards correspond to sections and which correspond to a specific piece of evidence, argument, or narrative. That is to say, the user should be allowed the freedom to create the outline structure, rather than have it imposed.)

Proposal

I propose to create an add-on for LyX which would combine the best features of Scrivener with the already powerful capabilities of LyX. As described above, this would consist of two inter-related tools:

1) The Outliner. The outliner would be an expandable panel where the structure and rough content of a document could be laid out prior to beginning the first draft. Specific features of the Outliner would include:

a. Each outline point would have room for a title, synopsis and type of content (drawn from the list provided by the LaTeX class).

b. Individual nests in the outline can be expanded or collapsed.

c. Elements can be dragged and dropped from one point to another. The underlying document will be updated to reflect any changes.

d. Though most points in the outline might correspond to headings (e.g. book parts, chapters, sections, or subsections), this would not be required. Instead, any block of text could be assigned to a point in the outline. It would behave in precisely the same manner as other outline elements.

2) The Corkboard. Housed in the same panel as the Outliner would also be a digital Corkboard which would display a small "Index card" with the title of an outline element in addition to the synopsis. However, in contrast to the outline, only the individual elements for a selected point in the document map would be shown. In the example below, the sections of a specific chapter, "Eragon Shadeslayer: Sociopath" are shown with their corresponding summaries. Changes in the order of the cards would be reflected in the underlying draft.

Conclusion

The proposed additions to LyX would further expand its other excellent features, making it the only document/word processor with a powerful set of creative tools. By associating blocks of text with additional meta-data (like a synopsis), working with the document structure would become a visual and interactive process, thereby allowing for much greater flexibility. Indeed, such changes would bring LyX one step closer to writing perfection.

Note: A PDF copy of this article can be found here.

Apolitically Incorrect » Writing and Literature

Hannibal, Napoleon, and Joseph Charles Minard

Hannibal Invasion of Italy (218 BC, Second Punic War)

Napoleon’s Invasion of Russia (1812-1813)

Conclusion

Unrelated Thoughts: Re-Visioning Minard with Open Source Tools

Acknowledgements and Further Reading

Similar Posts:

Working With Words, Numbers and Images: A Reading List

Similar Posts:

Image Gallery: The Notebooks of Leonardo Da Vinci

Similar Posts:

About This Book Thing

Background

Analytic Design

Tools and Technology

Conclusion

Similar Posts:

Typeset Your Curriculum Vitae – Part 3: Automatically Generate a List of Publications

The Basic Requirements

cvsplitbib and xetexCV

Creating a Categorized List

Generating the Bibliography

Examples and Class Files

Acknowledgements and Further Reading

Similar Posts:

Typeset Your Curriculum Vitae – Part 2: Extending and Customizing an Existing Document Class

Understanding Conventions

The Internal Anatomy of a Document Class

Class Description and Required Statements

\ProvidesClass

\LoadClassWithOptions

Building on the Foundation

\RequirePackage

Creating New Features and Modifying Old Ones

\def and \newcommand

\renewcommand

Rules for Creating New Macros

Managing Complex Formatting – \cvsection and \cvsubsection

Layout and Formatting

Conditional Formatting – \ifthenelse

Balancing Images and Text – \makecvtitle

Conclusion

Acknowledgements and Further Reading

Similar Posts:

Typeset Your Curriculum Vitae – Part 1: The xetexCV Document Class

xetexCV – An Attractive and Easy to Use CV Document Class

Downloads and Installation

Package Requirements

Curiculum Vitae Examples

xetexCV Class Usage

\makecvtitle

\cvsection and \cvsubsection

Hanging Notes

Choosing a Font

Other Features

Acknowledgements and Further Reading

Similar Posts:

Customizing LyX: Character Styles and the LyX Local Layout

Styles Defined

Using Character Styles in LyX

Customizing Character Styles

Example: Creating a “User Interface” Character Style

Step 1: Open your document in a text editor

Step 2: Create a Local Layout Section

Step 3: Define how you want the character style to look

Miscellaneous Things and Conclusion

Similar Posts:

Patronage in the Digital Age

Similar Posts:

Statistics With R – Part 1: An Old Dog Learns New Computing Tricks

Open Source Alternatives

Gnu PSPP

The R Statistical Project

Easing Into R

Series Introduction

Similar Posts:

Customizing LyX: Create an NIH Grant Proposal Template

Understanding the Big Picture

An Introduction to LyX and LaTeX