There are 6 lapis lazuli for each playhrough 3 are linked to optional bosses and the other 3 from the pot nobles:
BOSS RELATED
You need to defeat demon of hatred to get 2 and 1 from the shichimen warrior on fountainhead palace
POT NOBLES
As soon as you reach fountainhead palace and defeat Corrupted Monk head to Hirata state and talk to the pot noble he will give you a special bait that you need to feed to the great carp on fountainhead palace.
When you feed the bait to the carp return to the pot noble and there you can pick 1 lapis lazuli and buy 2 more from the noble for 6 scales each one