Both sides announced the settlement - to be paid by American Apparel's insurance company - on the morning a trial was to start in federal court in Manhattan.
Reading from a statement outside court, Allen said he hoped the outcome "would discourage American Apparel or anyone else from ever trying such a thing again."
American Apparel president Dov Charney told reporters it wasn't his decision to settle. The company's insurance company "controlled the defense" in the case, he said.
"I'm not sorry of expressing myself," he said.
Allen, 72, sued the clothier last year for $10 million after the advertisements turned up on billboards in Hollywood and New York, and on a Web site. Using a frame from the film "Annie Hall," the ads depicted Allen as a Hasidic Jew - long beard, side curls, black hat - and featured Yiddish text meaning "the holy rebbe."
Court papers filed on Allen's behalf had described the actor-director as one of the most influential figures in the history of American film, and say he believes maintaining strict control over his image has been critical to his success.
The papers claimed Allen hasn't done commercials in the United States since 1960s, when he was a struggling standup comic. The billboards, he says, falsely implied he endorsed a clothing line known for its racy advertising - a "blatant misappropriation and commercial use of Allen's image."
American Apparel lawyers have called the $10 million demand "outrageous," especially since the billboards were taken down after a week. They also have threatened to call Allen's former longtime companion, actress Mia Farrow, and his current wife, Soon-Yi Previn, as witnesses to show that his image has already been devalued by scandal. Previn is Farrow's adopted daughter.
Farrow starred in several of Allen's movies, including "Crimes and Misdemeanors." Their relationship ended in 1992, when she discovered he was having an affair with Previn, then 22.
On Monday, Allen blasted American Apparel, calling their First Amendment defense "sheer nonsense," and accusing of it of trying "to smear me."
Charney insisted there were no hard feelings, saying the billboards were misunderstood.
"We would never try to malign the dignity of Mr. Allen," he said. "I have respect for Mr. Allen. ... I hope to meet him on more friendly terms at a different point."
Copyright 2009 The Associated Press.