You should be able to find Plague Hearts pretty easily around the map as you play, but you also have the option of locating them via the radio for 350 influence. You will find one very early on in the game, and it's a lot less intimidating to attack than you might think.
The Plague Hearts are basically surrounded by plague zombies, and they will also release clouds of gas when you attack them that will damage you and increase your plague meter. You can actually kill the first Plague Heart pretty easily with a melee weapon, but it's even easier if you have explosives and guns. Placing a radio or fireworks outside to keep zombies away should give you enough time to kill the first Plague Heart without much difficulty. For the first heart, you can use melee weapons to finish it off, all you really need to do is get out of the way when the heart releases the poison gas, then get back in and keep swinging.
Explosives do the most damage, but are often tough to aim in close quarters where you'll usually find the Plague Hearts. You'll also want to bring guns and ammo. Just a few hits with a couple explosives and some gunshots on the first heart will kill it. The best strategy is to place a radio outside of the building the Plague Heart is in, sneak inside to the Plague Heart, then activate the radio remotely. If you don't have a radio, fireworks is the next best option, because they take time to start up, which will give you time to get near the plague heart. Once the noise starts, attack the plague heart. Throw bombs, shoot it, hit it with your melee weapon. Keep an eye on your mini-map to see when your noisemaker stops working, and then toss some firecrackers or set down a stationary noisemaker away from the heart, then go back and keep hitting the Plague Heart. You can see the pulsing of your noisemaker on the minimap, so even if you're busy shooting and bombing the heart and can't hear the noisemaker you can see when it stops working.
If you don't have noisemakers, it gets harder, but not impossible. You just have to alternate between killing zombies and hitting the plague heart. You'll need meds and lots of snacks, but it can be done.
Do this 4 more times, and you'll get this achievement:
In the below video, my gun jammed, and I was still able to defeat the Plague Heart with just a melee weapon. This was Plague Heart #7 in my game, so you can imagine that the first Plague Heart is a lot easier: