The word "penny" goes all the way back to Old English pening and has relatives in Germanic languages, such as German (Pfennig), Swedish (penning), and Icelandic (peningur). The original British penny was worth 1/240th of a pound sterling (now it is 1/100th of a pound). When the first United States one-cent coin was minted in 1793, people just continued to use the British term to refer to it.
The original U.S. five-cent coin was called a half dime (or half disme) and it was made out of silver. During the Civil War, silver, and other metals, became scarce, and most coins went out of circulation. In 1866, the five-cent piece came back in an alloy of copper and nickel, but it wasn't a big success. The silver half-dime was again minted until 1873 and continued to circulate after that, but in 1883 — after much lobbying by nickel-mining magnate Joseph Wharton — a new nickel alloy five-cent coin was introduced. This time it went into wide circulation, and people called it the nickel. Wharton made out very well.
The dime was established by the Coinage Act in 1792, but in the act it was called a "disme." Disme (pronounced dime) was an old word, from French, for tenth, which came from the Latin decima. The more common spelling even at that time was "dime" and that was what people used as soon as it was minted.
There's no mystery to the inspiration for this name. A quarter is a quarter of a dollar. So overall, we've got two coins named for their portion of a dollar (quarter, dime), one for the material it's made from (nickel), and one for old times' sake (penny).