Tucked in the bowels of the Citigroup website is a family tree every bit as sprawling as the one belonging to the Hapsburgs, the continent-spanning family that once ruled Europe. Set against a Gustave Courbet painting of a mighty oak, the tree represents the different branches of the seminal financial conglomerate of the 1990s. The website kindly offers three different options for printing it out. I chose the shortest version, eight pages (no joke). But even then, I needed a magnifying glass. By my count, there are 271 entities, combinations, spin-offs, and name changes listed, ultimately filtering down through the decades to become the ungainly entity ridiculed as GroupCorp.

Now Wall Street wants to take a chainsaw to those magnificent branches. Investors have come to believe that Citigroup needs to be whacked down. This spring, hedge fund manager Eddie Lampert bought $800 million worth of the stock. That’s only a small percentage of the $260 billion bank, but the purchase stirred the hearts of investors, who hope that the manager who liked Sears and Kmart so much he bought the companies might be able to spur change in another boardroom.

Not that Chuck Prince, Citigroup’s chairman and C.E.O. since 2003, isn’t trying. He spun off Travelers insurance. He unloaded Citigroup’s asset-management business. He initiated a cost-trimming program that includes thousands of job cuts. Prince has spent time cleaning house after multiple scandals. He has even started taking baby steps in dealmaking.

But such efforts aren’t enough for Wall Street—and probably never will be—because Citigroup is simply too big to function well. Almost nine years after the $83 billion Citicorp-­Travelers merger, it has become clear: Sandy Weill, the company’s iconic former chairman and C.E.O., screwed up.

Executive legacies are like presidential legacies writ small. It takes years before they can be judged. But judgment time has arrived for Weill, who, until recently, was hailed as a towering figure in finance. “An American legend . . . a banking visionary,” reads the back cover of the Weill biography Tearing Down the Walls. In a sprawling profile in 2000, the New York Times Magazine played up Weill’s Brooklyn roots: He was “superficially ordinary,” “moderately articulate,” and “lacking in grace.” Yet this man put together “the most impressive money machine in modern times.”

Sure, Weill’s conglomerate makes money. But the stock is right where it was in 2001. The company’s largest shareholder is restless. Citigroup’s valuation has contracted, from a price-to-earnings ratio of 19.7 in early 1999 to about 11.7 today. Even when it reports good quarterly numbers, as it did in July, investors remain wary.

As the size of this tree shows, the group is unwieldy: It’s a consumer bank and a corporate bank, an investment bank and a retail brokerage, a back-­office transaction-services company, a wealth-management firm, and a credit-card purveyor.

While there wasn’t a business model under Weill, there was a culture: Do a deal and move on. Walk up to the line. Sometimes cross it, as Jack Grubman did when the telecom analyst-cum-­investment banker hyped stocks he knew were dogs. Weill elevated corner-office infighting to a science.

Today, Citigroup looks like the last great merger of the 20th century. Weill had successfully lobbied to have the Glass-Steagall Act, passed during the Great Depression to limit the power of individual banks, repealed so that commercial banks, insurers, and investment banks could assemble under one roof. For this?

Now we can judge Weill for what he really was: the roll-up king of Wall Street. As mom-and-pop shops fold, roll-ups encourage efficiency and gain scale. But they don’t require vision.

Weill is scrambling to salvage his legacy as his creation founders. Last year, he published a little-noticed autobiography that was large in font and long on self-justification. His deal mania is displayed on each page, starting with the title: The Real Deal.

Toward the end of the book, when Weill goes out to dinner with Deutsche Bank chairman Josef Ackermann, his thoughts of stepping aside recede. As he recounts, Ackermann “stunned me by proposing a merger,” whereupon Weill “immediately warmed to the idea.” And that plan for retirement? Come again?

A page later, our narrator has a moment of clarity: “I recognized that there would always be ‘just one more deal,’ and that I’d never retire if I gave in to that old ‘urge to merge.’ ” In June, Weill described Citigroup this way: “I don’t think it’s too big at all. . . . It has the ability to avoid problems that may happen in one [region] and not in others.”

Where is the triumphalist of old? “He never talked that way in his life,” says CreditSights analyst David Hendler.

Citigroup’s stagnation doesn’t cast a new light on Weill alone. He belonged to an entire class of C.E.O.’s who had their financial triumphs in the ’80s and ’90s; like Weill, they were lauded for their vision, and most still are. But they were really just cobbling together disparate companies for the purpose of building empires and extracting riches for themselves. Did Sumner Redstone have a true vision? He split his organization, but the performance of the parent company has been lackluster. What about Jack Welch, Mel Karmazin, or Barry Diller? All were fortunate enough to make deals in an era when assets went up. Sure, it’s better to be lucky than good. But luck has run out for Sandy, and it’s running out for these guys too.

When such fading legends step down and leave their jobs to people like Prince, the new leaders function as the rebound boyfriend: a transitional figure, not marriage material. The market soon sours on him.

The solution for Prince, or his successor, is to break up Citigroup. The stock price does not reflect the com­pany’s worth. A hedge fund puts its value at as high as $69 a share, even though the stock traded in the low $50 range as of late July.

Citi brought in Gary Crittenden as chief financial officer this spring, a promising move. He’d been part of big spinoffs at American Express and, before that, the breakup of the retail conglomerate Melville.

Disappointingly, he’s gotten the message that Citi doesn’t need to be broken up. Citi plans to cut costs and improve returns by shifting its spending from slow-growth areas, such as credit cards, to high-return emerging markets.

Alas, Chuck is going to be reluctant to hack away at the tree. He’s stuck defending the creation of his mentor. But the rot comes from the roots.