Last December, Charney was served with a notice of inspection by the U.S. Immigration and Customs Enforcement agency, the largest investigative branch of the Department of Homeland Security, informing him that he needed to prepare documentation on all of his workers for review. Charney and his growing team of lawyers and consultants have taken this as a warning. The company has given the feds records for thousands of workers; Charney says he hasn’t heard a word in response. Since then, he’s spent his days bracing for a raid at any time on American Apparel’s factories.

American Apparel is the largest clothing manufacturer in the United States, and in downtown Los Angeles Charney employs about 4,000 sewers, cutters, dyers, and other workers, most of whom were not born in the U.S. Although they have all provided documentation, he is still concerned about the legality of the majority of them. Now he is desperately trying to bulk up his workforce in order to keep his operations running smoothly. This year, he has hired an additional 2,000 workers, many of whom found out about the jobs from fliers that Charney himself handed to them on downtown L.A. sidewalks.

“I’ve spent every moment of my existence from the minute I wake up—I have a stomachache, and I get up, and this is what I do,” says Charney. “I do this every day. I do it on Sunday. I don’t even remember when it’s Friday. One day it’s Saturday, and one day it’s Monday. I just keep going and going and going.”

Maybe someday Dov Charney will be known as a tireless crusader for immigrant workers’ rights, but until then his current wicked reputation will probably remain in place. Mention him and people make a sound of distaste and then ask if he is really an exhibitionist-pornographer-compulsive masturbator. This mantle is one that Charney both encourages and abhors but is hard-pressed to shrug off. The public has reacted strongly to the images of out-of-control carnality beamed down from American Apparel’s billboards and splayed across its ads, often photographed by Charney himself, that show young bodies in various states of undress, sporting his company’s clothing.

This sexual aura has cut both ways for American Apparel. Charney has made his name in part through controversy. He once famously masturbated in front of a female journalist. Charney says the reporter, for the now-defunct Jane, took the masturbation out of context. “I was a younger man,” he says, wearily. “The lines were blurred between paramour and reporter.” The reporter has said that her tape recorder or notebook was in full view at all times and that the relationship was professional.

Since the resulting 2004 article was published, four female employees of American Apparel have filed three lawsuits against Charney. One suit was settled; another was dropped; and the third, by former sales representative Mary Nelson, 36, alleges that Charney wore a skimpy thong that barely covered his genitals. During Nelson’s initial job interview, which was held at Charney’s home, she says he referred to female employees as “sluts.” Nelson’s attorney, Keith Fink, told the Los Angeles Times that she was wrongfully terminated after she consulted with a lawyer. The suit was sent into binding arbitration at the beginning of the year; a settlement has not yet been reached.

Charney insists that Nelson, who worked for American Apparel for a little more than a year, was a bad employee who swore compulsively and hustled him, often referring to him as “donkey cock.” In court papers, Charney’s lawyers portray Nelson as a sales rep who performed below expectations. Charney says she was the mastermind of the suits, and he even drew a diagram for me of how three of the four women went to the factory roof to conspire to file them. He can talk for hours about what he calls their scheme and their betrayal.

Still, Charney hasn’t denied the majority of the allegations. His own lawyers have stated in court documents that “American Apparel is a sexually charged workplace where employees of both genders deal with sexual conduct, speech, and images as part of their jobs.” Charney has said that his behavior is the norm in the fashion industry and shouldn’t be considered harassment. He has defended himself by saying he is in the business of making underwear. He points out that in addition to being the company’s creative director, he is also one of its fit models—a simple explanation for why he would stride around his offices half-dressed. He has said that the real reason he had the underwear on was to show his employees and ask them how it looked. He has also said he “test-drives” the underwear to see how it fits “in action.”