According to the official guide there are 23 side quests. Which I will name and give the prerequisites for here. However there are others not in the guide that people have found, you can find these at the bottom of the list or in the comments.
However the symbol that looks like a simple "knife" shape (the symbols to the left and right of the quest name) could also mark the sidequests. Click the link below to check the symbol.http://www.flickr.com/photos/theicemancometh/6379521295/in/p
*Thanks to AdequateMan for this, and Tarzakul for conferming it*
Blood on the Ice - Enter and exit Windhelm a total of four times.
Forbidden Legend - You will probably just stumble across this quests in your adventure. There is either a note on boss in the college quest under saarthal, Or you could read a copy of the book "Lost Legends".
The Forsworn Conspiracy - On your first visit to Markarth you should witness a murder
No one escapes Cidhna Mine - Complete The Forsworn Conspiracy
The Golden Claw - Should pick this up in the bleakfalls barrow.
In My Time of Need - Complete main quest Dragon rising
Kynes Sacred Trials - Find Froki in his Shack, just below the snow line high in the Jerali Mountains.The shack overlooks the Southwestern rim of the rift.
Laid to Rest - There are two villagers arguing with the city steward as you arive in Morthal, converse with these townsfolk or speak to Jonna the barkeep in Moorside Inn.
Lights Out - When your in Solitude be on the lookout for a shady Argonian character, He sometimes beckons you over, speak to him and see what he wants.
The Man who Cried Wolf - While in Solitude, visit the Blue Palace and speak to the Jarl and Falk Firebeard.
The Wolf Queen Awakened - Complete side quest The Man who Cried Wolf and be level 10 or higher.
Missing In Action - Enter and exit the Whiterun buildings around the market. You should notice the old woman Fralia Gray-Mane shouting at two of the Battle-Born family members.
Promises to Keep - Talk to Louis Letrush in the Bee and Barb, Riften.
A Return to Your Roots - Begin Daedric Quest Discerning the Transmundane or main quest Elder Knowledge. Should occur when you enter Blackreach
Rise in the East - Meet Orthus Endario. He is in the docks outside Windhelm in the company office.
Rising at Dawn - Contract Vampirism
UnFathomable Depths - Must be level 14 or higher. Find the Argonian From-Deepest-Fathoms, she will be on the Riften Docks.
The White Phial - In Windhelm visit the msrket stalls in the southwest cirner of the city. Enter the White Phial shop and start a conversation with Nurelion.
Repairing the Phial - Complete main quest Throat of the World and Side quest the White Phial. Wait 72 hours and be in a city a courier should find you. (apparently this one doesnt count though)
Captured Critters* - Find one of the Five insect species that inhabbit skyrim. One of them is inside the Frostflow Lighthouse in Winterhold hold.
The Forgemasters Finger - Must be a non-Orc character and talk to the chief in any of the following Orc Strongholds;
Mar Khazgur in The Reach
Dushnikh Yal in The Reach
Narzulbur in eastmarch
Largasbur in The Rift
The Great skyrim Treasure Hunt*. - Find one of the 11 treasure maps.
Masks of the Dragon Priests* - Find one of the Dragon Priest masks. The first one i found was the wooden mask in the center of the Labyrinthian.
Note: Quest names marked with this symbol * do not appear in your quest menu list, although objectives may.
Also, to add a couple that are not in the guide, submitted by you guys;
'Tending the Flames' - Complete the Bards Collage Initiation.
'Silenced Tongues' - Which you get after entering Volunruud.
'The Book of Love' - Help out the Temple of Mara (Riftwood)
'Siege on the Dragon Cult' - Started by talking to Captain Valmir outside Forelhost (Its the peak of the small mountain directly south east of Riften)
'Repentance' - started by talking to Illia in Darklight Tower which is southwest from Riften.
Infiltration - Talk to Stalleo outside Treva's Watch
Alteration Ritual Spell - Talk to Tolfdir, I believe you have to join College of Winterhold first.(Seems weird, think it should be a mage symbol)
Lights Out! - Given by Jaree-Ra in Solitude (near the trading company and the docked ship) is a fairly easy quest.