Challenge arenas are hidden around the worlds map. There will be a small tutorial with information about them the first time you find one. I suggest you wait until later in the game to make these easier. I found the double jump ability very helpful. The vine climb ability is also required for some of them. Any arenas you've previously found will be marked on your map by a red area.
There are 6 arenas of increasing difficulties. The first couple are very easy. The time limit on the last few are very strict. You'll most likely have to play through the courses a few times and memorize the path to score 4 stars.
Scoring works on a points system. At the end of the arena, your score is tallied. Each arena has a required score for four stars. Killing an enemy scores 100 points, smashing orbs scores 500 points, and your time left adds to your score at the end. Taking damage causes you to lose 100 points.
Abadis Forest Trial #1 - 9000 Points
Aurora Village Trial #2 - 10500 Points
The Glade Trial #3 - 6000 Points
Cirromon Caverns Trial #4 - 8400 Points
The Sorrowing Meadow Trial #5 - 9000 Points
Archers' Pass Trial #6 - 4800 Points
They're easy enough to find, provided with the locations on a map. If you're still having trouble, this locations guide will show you how to get to each arena starting from the world map.
If you're having trouble completing an arena, use this walkthrough.