It is impossible to teach an art. You can only provide conditions which are favorable to its development. Why am I writing this, then? I guess because I find little of worth in most of the guidebooks to song composition, and find myself wishing one existed that I could read and think "Yes! That explains it!". This probably isn't that guidebook, but here it is regardless. I hope it helps somehow.
I'll start with an description of my process and work into it the concepts and techniques I have developed, and then outline some general principles for composition that are more broadly applicable. My process begins usually with a chord progression. I tend to have a store of these accumulated from hours of instrument practice, and having this large store of musical ideas is useful because your form should reflect your content; if your theme is sad, your chords should echo that somehow. Again, that isn't a rule. Marrying sad content to happy chords can be bitterly ironic and more effective than simply using sad chords. These kind of decisions are made during the process, in the same way a carpenter "works with" the grain and peculiar quality of his material in crafting a table or chair. I wont go into the process of writing chord progressions because my understanding of theory is bested outright by many other, better chord-guides on here, so lets skip to the lyrics.
First you need an idea. This is the part where all the formal rules and techniques fall silent, because true inspiration is a damn near mystical process. Nobody who experiences it understands exactly why it happens or where it comes from, but what we know for certain is it is highly intuitive. You "feel" your way though a project, like putting together flat-pack furniture in the dark. You cant see the instructions, you can only poke and twist and hammer till it feels like you have something resembling whatever it was you wanted.
Like teaching, inspiration is impossible to just do. You can only make the conditions ripe for it to happen spontaneously. You should absorb influence en masse, listening to all kinds of music to provide you with a multiplicity of stylistic ideas from which to draw when writing (more on this later). Purism is the most awful and stagnant trap to fall into, and will render your music boring, repetitive and stale. Stylistic cross-pollination is the lifeblood of originality.
But influence doesn't come only from music. A song is a poem. Poetry at its inception was inseparable from song. The epics of Homer were rhapsodised, performed aloud to music, never read quietly to oneself in the head. Being a songwriter is being a poet, so read, damn you! Or perhaps you dont think the kind of music you want to write needs "poetic" lyrics? Read anyway! Its fun! And not just literature either, you should seek to absorb as much information, about anything, as you can. Thankfully we live in an information-rich society, with whole online worlds devoted to simple ideas, historical events, people, news, fiction, art. There is no end to the options available to draw from. Seek out real experience too. Inspiration can come just as readily from the sound of ice being scraped from a windscreen or the shape of a bridge as it can from a recent news story. Because anything you write will be a product of your personality, and your personality is the summation of your experiences, everything you experience goes toward your art. Not just the momentous events.
Once an idea comes, you can select one of these melodies and start writing words for it. This is my preferred approach because taking already written words and shoehorning them into a chord structure can tend to result in awkward, wrenched meter and a messy vocal line. There are, of course, exceptions to this rule; often lines already written can be worked (with a little tweaking) into an existing song, but generally the structure of your words can be much better dictated by the flow of the existing vocal line. Lyrically, before starting to write, I fill pages of A4 with words, phrases and lines connected to my theme, trying where I can to come up with something pithy. When I put pen to paper I'm usually thinking more about writers than lyricists...my moments of inspiration have come after reading people like Yeats or TS Eliot. Always be thinking of the musicality of your words as well as what they mean. Words have beauty too...so much so that stanzas like this one:
"And sang within the bloody wood
When Agamemnon cried aloud
And let their liquid siftings fall
To stain the stiff dishonoured shroud"
(from Sweeney Among the Nightingales, by TS Eliot)
For me, dont even need to mean anything. I am content to be floored by the absolute mastery of alliteration, assonance, sibilance, syntax, external and internal rhyme, meter, and just the general gorgeousness of it all. Every word in this poem was selected with utter care, with great thought. There is no flab, he cuts right to to meat of things, to borrow a phrase from Elizabeth Taylor. I mean good god, read this shit out loud! Best of all, (unlike a lot of Eliot) it scans. You could sing it.
Try to think of interesting ways to say things, especially if they have already been said before. Cliche is your enemy and you should avoid it like the plague... *cough*. Avoid phrases in common usage, unless you have a good reason. Here's an example from the Frank Turner song "I knew Prufrock Before He Got Famous":
"Lets begin at the begining
We're lovers and we're losers
we're heroes and we're pioneers
We're beggars and we're choosers"
Leaving aside the fact that the title alludes to Eliot for no apparent reason other than perhaps to say "look at me, I'm clever", the song opens with a catalogue of cliche. The first and last lines of this stanza are particular culprits. Saying "begin at the begining" was witty in the Sound of Music, but here it feels trite and overused, and actually within the context of the song has little meaning at all. The inane repetition, too, is flabby and unnecessary. This has roughly the same syllable count as the Eliot stanza, but look how much music and elegance and variation and loveliness Eliot packed into his four lines, while Turner uses the same two words nine times. These words werent thought over or carefully chosen, they were poured out onto a notebook with no revision. Sometimes this works, often it only results in sloppy prosody.
Here is a better example of musicality in verse that didn't come from an actual poem. Joanna Newsom's harp-accompanied epic "Sawdust & Diamonds"
"And everything with wings is restless, aimless, drunk and dour
butterflies and birds collide at hot, ungodly hours
and my clay-colored motherlessness rangily reclines
come on home, now! All my bones are dolorous with vines"
This is beyond good. The symmetry of vowel and consonant sounds here is a miracle: everything/wings, restless/aimless, drunk/dour, butterflies/birds, hot/ungodly, clay/coloured, rangily/reclines, home/bones. These lines have been considered and weighed on the tongue, they scan perfectly, balance perfectly, sound perfectly. There are almost elements here of anglo saxon alliterative verse. Again, four lines, but here the words are interesting, they have curious and lovely sounds, and when yoked together, they are quite something. "Clay coloured motherlessness" has a particular sibilant grace. Allen Ginsberg is great at this yoking, in Howl alone you have phrases like "the starry dynamo in the machinery of night" and "the crack of doom on the hydrogen jukebox". These unlikely combinations have a peculiar magic. They certainly interested Dylan, who borrowed a lot of his imagery and syntactic technique from Ginsberg.
I will say that writing good lyrics is a question of practice. You should have written hundreds of aborted poems and songs before you are really ready to commit words to tape, just like you ought to have notebook after notebook of developmental poetry written before you can even think of being published. It is something you need to work at, and the best ways to do this are to read to expand your vocabulary and develop your understanding of poetic form and technique, and to practice incessantly. Only you, however, can be the final arbiter of your own quality. Look yourself in the mirror and ask if you truly feel what you have written deserves an audience, and if the answer is an honest yes, then god-dammit get recording.
In terms of form, I would caution against verses and choruses. Always avoid formulas. A chorus should only be used if you feel you have one idea, one stanza, that needs repeating over and over: the thematic focus. If you dont, then dont strain to come up with one because you feel you need a chorus to follow your verse. I have found a working method based on "units", which will be labelled A, B, C and so on. Each have different chords & sometimes different words, and can be arranged in any order you feel is appropriate. Some examples of structures I have used
A A B A C
Basically a variation on the Tin Pan Alley form with a climactic closing section tacked on at the end. Useful for something short and sweet, or if you want to signal or symbolise some story which has, say, an unhappy first and second act but a redemptive third. Or the opposite. As I say, use your own creative intuition.
A B C D B
This is a cool one, because for most of the song it feels as though there is no structure whatsoever, that you are simply moving from one section into another. One song I wrote in this form had a key change and a move from common to swung rhythm, before reprising the second "B" unit as a kind of chorus, with the same chords and lyrics but transposed to the new key. I used this to represent a false resolution, in which the narrator felt he had progressed toward something but actually had not.
One of my favorites, a verse and a chorus that abruptly end without going anywhere. Sounds abortive and pointless but if you want to symbolise something with a sudden and unexpected end or twist, this is perfect. Also, being a fan of imagism, I like things to be condensed; to squash the maximum amount of idea into the smallest possible space, and this form fits the bill perfectly.
A A B C D A B E C
A weird one, here just as an example of how damn crazy your structures can be if you abandon calling everything either a verse, chorus of middle-eight. Think Paranoid Android/Bohemian Rhapsody: a lot of musical ideas tacked on to each other. Where repetition occurs in this meandering patchwork, it is all the more obvious and therefore more meaningful. This kind of thing is good for telling a story, or for lyrical abstraction.
A A A A B
Kind of a folk form. Lots of identical verses, then a closing segment which differs slightly. This one is good if you are telling a long, folk-ballad story like John Henry's Hammer, especially when it depicts some kind of epic struggle, because its similarity to Heroic and Blank Verse in it that it can be endlessly extended is a useful symmetry to make since they tend to deal with that kind of subject matter.
I've shown these forms here not for you to copy (although you are welcome to them, of course) but to illustrate the effectiveness of thinking of structure in this way. When you stop thinking in terms of verses and choruses, of songs only being constructed of these two basic units (or three/four, if you have a pre-chorus and middle-eight), you can see that there are many things you can do in a few minutes of music that dont involve inane repetition of a preordained format, used only because "thats how you write a song". Sections with names have special places they have to go. A middle eight doesn't come after the first verse, but a section labelled C can go anywhere you like.
Over the years, I have developed a few of what I ironically call "commandments", which I use to guide the process and remind me of certain things I may occasionally forget. "Commandments" is ironic because they are nothing of the sort. I said there were no rules to this, and so these are not rules. They are statements, concepts, intended to draw attention to certain aspects of the process and guide me.
The Only Virtue of a Style is its Elasticity
The traditional rite of passage in art is finding the individual "voice" with which one is distinguished as unique and distinct from other artists. Until this voice is found, everything you produce will be an imitation of something else. The trick to finding the individual voice is combining a diverse array of influences, a little from here, a little from there, until it is hard to say where the influence ends and your voice begins. Einstein said "originality is knowing how to hide your sources". In art, the best way to do that is to have lots of them, blended together.
Once you have found your voice, though, you must be aware of its limitations and be constantly seeking to expand them. Your style should be mercurial and elastic, able to express any idea. If it cannot, if you find a subject matter which seems unsuited to your chosen voice (for instance, how many hip-hop love songs do you hear?) then you are faced with two options: abandon the subject matter and write about something more suitable, or alter your voice. Choosing the former will result in stagnation, creative tepidity, dullness. The latter will result in, obviously, newness and vibrancy, but there is always the possibility of alienating your audience, or if you dont have one yet, the fear that an audience would be critical of this new direction. This leads me on to the second commandment.
Your Reception is Irrelevant
Simply because trying to second guess taste, which is as much due to currents and trends and the peculiar beast known as discourse, is likely to lead only to populism. Follow your individual vision and disregard all other considerations, including commercial ones. If you find yourself wondering if people would like what you are doing, imagine they are all you, posses your taste and preferences and hold your views. To alter in any way the content of your art to suit what you perceive to be a public demand should be avoided at all costs. To do this is to be a designer, not an artist: one who writes to fulfill a consumer demand, rather than to fulfill an inner desire to create.
This is not to be mistaken with a command to ignore advice and criticism. A fresh perspective, especially one with experience and knowledge, can shed light on what might be astray with a piece which you were unable to see. Sometimes you are so close to the canvas you cant see the painting. But the ultimate editor and auteur of this criticism is your own artistic discretion. Always follow your vision even if it seems unorthodox or outlandish. Indeed, especially if it seems unorthodox or outlandish. You may be creating something of profound newness and importance.
All is Imperfect and Impermanent
Perfection is something that western art values have striven for since the time of the Ancient Greeks. We have idolised it to the extent that it became a truism; perfection in art was so obviously the goal that nobody ever stopped to consider what it even meant, and whether it was even possible. Perfection in practicality means a slow process of editing and revising and altering and adding, and this is good, one should not rush, but perfection is a summitless, infinite mountain. All painters know this. A painting is never finished, always there can be added more colour, detail, shade, brushwork, and part of the skill of a painter is knowing when to stop painting. Likewise, you can edit and alter and revise your work literally endlessly, but you must learn to recognise when you have effectively said what you set out to say with the song, to recognise when further changes would be fruitless. The japanese tea ceremony inspired an artistic aesthetic called Wabi-Sabi, which is centered around recognising the inherent beauty in imperfection and, rather than attempting to erase it with a more controlled and elegant attempt, accepting it as part of the whole and a microcosm of the imperfect universe.
Another principle of Wabi-Sabi is impermanence. Most art is built to last, generally out of an egoistic desire to become a canonical fixture, to be remembered for posterity, but art that follows the wabi-principle accepts that decay is a part of everything, and that considering whether your work will be remembered is futile and eventually restrictive, because it inspires the same approaches as considering your audience. You will lose touch with the creative voice and so become an imitation of yourself, populist and trite.
Only Write From a Love of Doing So
This one is obvious, but it can be surprising how often it gets forgotten when the desire for recognition or acclaim take hold. Writing is hard work, it takes a great deal of effort and will, but it should always be a labour of love. It shouldn't be a chore, an arduous task you set yourself out of some feeling of obligation or fatuous self-improvement. I write because I cannot imagine not doing, because when I wake my thoughts immediately turn to it, because I am compelled by some inner drive to do so. If you cant find inside yourself this same drive, if you find yourself slogging away at a song, having to push yourself to complete it, feeling no joy in the careful crafting and sculpting of sounds and words, then just stop. Right now, please, just stop. You really aren't cut out for this. Your lack of drive will lead to boring, insipid music of which the world is already quite inundated. If however you read what I just wrote about drive and felt immediate recognition, felt you understood exactly what was meant, then please dont stop. No matter who tells you your ambition is folly, no matter who raises their eyebrows when you shyly admit that you write songs and, just maybe, would like to do it for a living, no matter how many well meaning people will try to keep you from it, DO NOT STOP. You might be doing something really amazing.