SAN FRANCISCO (AP) – Following his admission that he had an affair with a trusted aide’s wife, Mayor Gavin Newsom pledged to work even harder for the city as he seeks re-election in November.

Yet even before his public apology over the tryst, there was talk that the 39-year-old mayor’s heart was not totally in the job. Some of it came from Newsom himself, who openly bemoaned the toll being mayor took on his personal life and said he was unsure he wanted to do it another four years.

“Every single night – and I don’t want this to sound like some therapy session – but I sit there at bars and restaurants doing homework and getting food to go,” Newsom told the San Francisco Chronicle in late October. “That’s not what I wanted to sign up for in my late 30s. I want a little more balance.”

While aides quickly brushed off the remark as a sign of fatigue, a series of missteps in the months before and since suggest San Francisco’s most eligible bachelor was offering a revealing glimpse into life at the City Hall.

“There has been kind of a general agreement that the mayor’s office in the last six months has been relatively unfocused,” said political consultant Jim Ross, who managed Newsom’s 2003 mayoral campaign. “I’ve had people tell me the mayor doesn’t seem to be enjoying the job.”

Newsom declined interview requests this week after acknowledging the sexual relationship with Ruby Ribbey-Tourk, a former staff member and wife of his now ex-campaign manager, Alex Tourk.

Some of the miscalculations have come in the personal realm, such as Newsom’s decision to accompany actress Sonia Milos, whom he briefly dated last year, to a dinner hosted by a group co-founded by the Church of Scientology.

San Franciscans were quick to overlook the liaison until the mayor showed up at the San Francisco Symphony in September with a not-quite 20-year-old model, who happened to be a registered Republican, on his arm.

While Newsom has expressed frustration with the interest in his love life, he also has joked about it, noting that, as someone who basks easily in the limelight, he brought some of it on himself.

On the political front, Newsom also has taken his share of lumps.

When the San Francisco 49ers announced they planned to move to Santa Clara, the team blamed the decision, in part, on the mayor’s failure to court the owners and delegating to his staff the job of negotiating a deal to build a new stadium.

In between, a series of bruising battles with the Board of Supervisors, where Newsom counts only four of the 11 members as reliable allies despite enjoying approval ratings that consistently top 70 percent, has limited his ability to advance his agenda.

Several supervisors, for example, have been openly hostile to his plan for providing free wireless Internet access service citywide. His most devoted rival sponsored a ballot measure calling on the mayor to appear before the board once a month, something Newsom has refused to do since the resolution passed in November.

The power struggle between the city’s executive and legislative branches is nothing new in famously fractious San Francisco, but appears to have taken a toll.

“He has one hand tied behind his back,” said David Heller, president of the Greater Geary Boulevard Merchants Association, a business group that backed Newsom in the last election. “Whatever he wanted to do, the supervisors went against it and they are trying to do stuff to discredit him.”

Last Monday, speculation about what was going on inside the Newsom camp mounted when a local politics Web log reported that the mayor’s top spokesman had posted a half-dozen negative messages about the mayor’s critics under a fictitious name.

After at first denying the stealth strategy, the spokesman, Peter Ragone, acknowledged writing the blog posts. Whatever outrage there might have been about the incident was quickly consumed, however, by the bombshell over Newsom’s tryst with his former appointments secretary.

“Sex is not an issue in San Francisco, but betrayal is,” noted Ross, the mayor’s campaign manager in 2003. “Loyalty matters to people, and that’s a huge issue.”

The recent events perhaps have tarnished Newsom more than they would another politician because such high expectations were held for him when he assumed office in January 2004.

At 36, he was San Francisco’s youngest mayor in more than a century and with his lanky good looks, and a stylish wife at his side, he invited the inevitable Kennedy comparisons. And with Rep. Nancy Pelosi, then the House Minority Leader, and Sen. Dianne Feinstein as his political godmothers, he was quickly dubbed one of the Democratic Party’s rising stars.

He immediately scored high marks for promoting women to lead both the police and fire departments, a national first for a city anywhere near San Francisco’s size.

The reviews were more mixed on his next bold step, taken five weeks after his inauguration: throwing open the doors of City Hall to same-sex couples seeking marriage licenses, a move that disregarded California law and the pleas of his party elders.

After his marriage to Fox News Channel host Kimberly Guilfoyle collapsed under the strain of a bi-coastal relationship in January 2005, Newsom kept up a punishing pace, frequently disguising himself in a baseball cap and jeans so he could get a tourist’s-eye view of the city.

Angela Alioto, a former city supervisor, said there is no doubt Newsom has made great strides in burnishing San Francisco’s image, but it has come at a cost.

“I just think he is having a hard time in life generally,” Alioto said. “I think the divorce was more devastating to him than people thought, and the fact that he gets more press about his hair and the girl he’s dating than the programs he is killing himself over is hard.”