According to one account, Kelly climbed his first pole at the age of seven, and at nine he performed a “human fly” trick, climbing up the side of a building. He is credited with popularizing the pole-sitting fad after sitting atop a flagpole in 1924, either in response to a dare from a friend or as a publicity stunt to draw customers to a Philadelphia department store. In January of that year he sat on a pole for 13 hours and 13 minutes to publicize a movie.

In 1926, Kelly set a record by sitting atop a flagpole in St. Louis, Missouri for seven days and one hour; in June 1927, he planned to beat that record by sitting for eight days in Newark, New Jersey. He would end up sitting atop the Newark pole for twelve days, and on a pole in Baltimore’s Carlin's Park for 23 days in 1929. In 1930 he set a world record by sitting on a flagpole on top of the Steel Pier in Atlantic City, 225 feet (69 m) high, for 49 days and one hour.

At the height of his fame as a latter-day stylite, he toured 28 cities, charging admission to people who wanted to stand on roofs to see his performance stunts. He also earned an income from endorsements, personal appearances and books about his life. He called himself “the luckiest fool in the world.” He once calculated that over two decades he spent 20,613 hours sitting on flagpoles, of which 210 were in sub-freezing weather and 1400 hours in the rain. In one 1927 stunt, he climbed on a pole on a speeding biplane, sitting on a twelve-inch (30 cm) iron crossbar as the plane flew 500 feet (150 m) high. The New York Times reported that “he didn’t get around to hanging by one hand, as he promised he would.”

While pole sitting, Kelly was said to have subsisted mainly on coffee and cigarettes. He learned how to nap while sitting upright, and never was secured by more than a simple leg strap. He once claimed that he doesn’t “take as many chances as a window cleaner.” Journalist Jay Maeder wrote that “The newspapers were regularly full of pictures of Shipwreck Kelly, matter-of-factly brushing his teeth and shaving his face, hundreds of feet in the air.” He said that he was able to sleep while pole sitting by putting his thumbs in holes in the pole shafts. If he swayed, the pain in his thumbs would force him to right himself without waking him up.

His career began to decline after the Wall Street Crash of 1929, and in 1934 he was working as a gigolo at a Broadway dance hall, Roseland, then a “dime a dance” hall. Journalist H. Allen Smith found him “wearing a tail coat and silk hat, sitting on a plush divan.” By then, Kelly said, 17 people were claiming to be him, because few people knew what he looked like. In the 1930s there was less tolerance for such stunts, and police took a dim view of the disruption it caused. In 1935 he attempted to break his Atlantic City record by sitting on a pole in the Bronx, but was aloft for less than a day before he was arrested as a public nuisance. He only climbed down from the pole after police threatened to cut it down.

One of his last major public appearances was on October 13, 1939: Kelly celebrated National Donut Dunking Week by sitting on a pole atop the Chanin Building on East 42nd Street in Manhattan and eating 13 donuts dunked into a coffee cup and fed to him while he stood on his head. But by then that kind of work had pretty much dried up, he served in the United States Merchant Marine during World War II.