That's a really nice tip with the party, thanks. Yeah, I was getting invaded every 1-2 minutes so it was pretty much impossible to actually progress. That's a really dumb thing that that scenario can happen. Should be an option to play in offline mode for that reason. Especially when the people who join can one hit you with anything.
Yay! Someone invaded me so I now have to die to go back to my world to get invaded again to die again to make no progress again.