This is tied directly to another achievement called The mail always gets through, which has you deliver letters to all mailboxes in the game as well as three special packages. One of the three packages contains the cake for the inventor. While carrying the purple package, his location will be marked on the map.
However, the inventor will not grant you access through the front door of his house, so you will have to enter through the back door by bouncing off one of the small floating islands below. The rails directly under his house is where you want to get into in order to reach the back entrance.