James Benton has been a singer of jazz, blues, and R&B during two distinct eras in Portland's musical history. He first came to prominence in the 1950s, during the heyday of the African American entertainment district along North Williams Avenue, and performed steadily until the late 1960s. During that time, he also operated a performance space on North Williams Avenue that was an important gathering place for African American musicians.

Benton emerged on the Portland music scene again in the mid-1990s. He has been a mentor to such nationally recognized jazz artists as drummer and bandleader Mel Brown and 2012 Grammy Artist of the Year Esperanza Spalding. Though he sang bebop and jazz standards, his style never strayed far from the blues and gospel music he heard during his childhood near the Arkansas/Louisiana state line.

Benton first came to Portland with his parents in 1940 from Crossett, Arkansas. The family lived in Vanport, and he graduated from Jefferson High School in 1949. Benton began frequenting the Black Entertainment District along North Williams Avenue, where he was inspired to become an entertainer. 

Hired to play semi-pro basketball for the Chicago Hottentots, eighteen-year-old Benton developed his singing abilities by learning to harmonize on road trips. In Portland, he joined The Audios, an a cappella vocal group modeled after the Four Freshmen, and learned to play drums. He then joined the Frank Martin Trio. Benton credits Martin, who was inducted into the Kansas City Jazz Hall of Fame, with helping develop his vocal skills and repertoire. 

During the mid to late 1950s, Benton maintained a theater, social club, and temporary residence for musicians called The Backyard in a garage next to his home on the corner of North Shaver Street and North Williams Avenue. Its soundproofed walls contained a stage, thirty-nine theater seats, two pianos, and a drum set. "It really was a musical family," recalled Mel Brown, who began to learn his craft at jam sessions at The Backyard. "Every day, all the musicians would congregate around 12 o'clock at this huge barbecue pit in back, and if you felt like playing, you went inside and played. The guys would stay around until about 6:00 pm, then go to their gigs. When the gigs were over, everybody came back, and we'd jam until five or six in the morning."

In 1960, Benton formed the Del Tones, a group that endured for seventeen years, performing in downtown Portland and in suburban locations such as the Beachcomber in Lake Oswego. The Del Tones also toured with bands such as the Benny Goodman Orchestra. It was also in 1960 that Benton acquired the nickname he has used ever after—Sweet Baby James—bestowed on him by a seaman known as Sweet Smellin' Eddie who frequented Williams Avenue.

Benton’s greatest impact during the second stage of his career was as a member of The Original Cats, which included veterans of the Williams Avenue scene as well as a young bassist, Esperanza Spalding. Since then, Benton formed a band with B-3 organ player Louis Pain. The band, called King Louie and Baby James, released two CDs—Live at the Waterfront Blues Festival (2005) and Around the World: Live at Jimmy Mak's (2008). Benton was living in Scappoose with his wife when he passed away in February 2016.