You don’t really need to grind for extra Amber Shards or smash every barrel. I ended the game with 50,000 unspent Amber Shards. Just make sure you open each chest you locate as you pass by it (again and again, due to the nature of the story) and you should have plenty. Running around to smash every urn and barrel will end up being a waste of time (note some missions result in relatively few Amber Shards, don’t sweat it).
Also it’s unnecessary to replay portions of days you’ve already played with a companion to get more Shards, you’ll have plenty before you pass the point of No Return (the game warns you explicitly about this).
I guess it’s worth noting that you upgrade all skills between missions at the Tree (with the Witch), at the orange anvil.