At first glance “literary” looks straightforward. Merriam-Webster’s Collegiate Dictionary starts off with this: “of, relating to, or having the characteristics of humane learning or literature”.
Move on to “literature,” however, and fault lines begin to emerge:
3 a (1) : writings in prose or verse; especially : writings having excellence of form or expression and expressing ideas of permanent or universal interest
The American Heritage Dictionary definition of “literary” progresses from 1., “Of, relating to, or dealing with literature: literary criticism,” to 4a, “Appropriate to literature rather than everyday speech or writing,” and finally 4b, “Bookish; pedantic.”
For “literature” AHD moves briskly from the inclusive — “1. The body of written works of a language, period, or culture” — to something less generous: “2. Imaginative or creative writing, especially of recognized artistic value.”
In other words, value judgments lurk not far below the surface of both “literary” and “literature,” and it’s not hard to see how they stir up uneasiness and outright opposition. In some quarters “literary” suggests not only bookish and pedantic, but pretentious, snobbish, affected, esoteric, incomprehensible . . .
As a longtime reader of fantasy and science fiction, I’m particularly intrigued by the way “literary” is used to characterize anything that doesn’t fit into a genre. This leads to grand generalizations, judgments, and arguments that generate plenty of heat but not much light. What it misses is that genres are primarily marketing categories developed by publishers in order to treat books as products. Categorize a novel and it’s easier to promote and sell. Writers aren’t stupid: if they want to sell, they’ll write what the publishers are buying.
Genres have been around long enough at this point that they’re embedded in readers’ heads. Audiences have developed for particular genres, subgenres, and sub-subgenres, and self-publishers who want to sell ignore this at their financial peril.
Writers who want to sell will often write to the specs of a particular genre, at least until (if they’re lucky) they develop enough of a following that their byline becomes the brand. Unfortunately, the byline brand can become as restrictive as a genre category, which is why some well-known writers who want to strike off in a different direction do so under a pseudonym.
Without market pressures, though, writing often doesn’t fit neatly into categories. A while back, in “Genres and Dump Dogs,” I wrote this:
Literary genres are like breeds — of relatively recent development, especially the notion that there are clear lines between them and everything has to fit into one category. “Literature” is more like those village dogs of indeterminate breed: it adapts to the climate and food sources available, and maybe it looks a little like this, a little like that, but you can’t say for sure that it’s a beagle or a foxhound (or a mystery or a romance). When you’re trying to tell a story, you scavenge and steal from whatever’s in the vicinity and if it works you keep it.
If “literary” came to mean “willing to scavenge and steal from whatever’s in the vicinity, all in the interest of the work,” that would be OK with me. The sky, or maybe the ocean, is the limit.