The Lost Wm. A. Spencer Mansion - 85 Fifth Avenue

Commercial structures had invaded lower Fifth Avenue when this photograph was taken.  from Old New York Houses, 1900, (copyright expired)

In 1836 Fifth Avenue was extended through land owned by the estate of John Cowman.  The new block front on the east side of the avenue, between 16th and 17th Streets, was portioned into building lots.  The 26-foot wide lot at the corner of 16th Street was sold that year for the princely sum of $8,600 to Gardiner G. Howland.  He paid $500 less for each of the two lots next door.

In the meantime, Captain William Augustus Spencer had distinguished himself in the U.S. Navy.  He was born in 1792 to Ambrose Spencer (a United States congressman, New York State senator, and State Attorney General among other achievements) and Laura Canfield.

Spencer had married Eleonora Eliza Lorillard, daughter of Peter A. Lorillard, in 1823.  The couple had one child, Lorillard, born in 1827.  Not long after Eleonora's death on August 15, 1843 at the age of 42, William married her sister, Catherine.

William Spencer, now retired from military service, set out to provide his bride a substantial new home.  He purchased the two lots at the northeast corner of Fifth Avenue and 16th Street from Gardiner G. Howland for $21,500--more than $710,000 today.  According to New York historian William S. Pelletreau in his 1900 Early New York Houses, "Upon the lot thus purchased, Captain Spencer erected a mansion which for size and elegance surpassed anything on Fifth avenue at that time."

The double-wide residence was 49-feet wide and stretched back 149 feet on 16th Street.  To the rear was a single-story conservatory, so important in upscale, early 19th century homes.  Pelletreau described it as "in itself a thing of beauty."    Three stories high, the Italianate-style mansion featured expected elements--architrave framed openings with substantial cornices, floor-to-ceiling parlor windows that opened onto a cast iron balcony, and a handsome cornice with paired brackets.  The 16th Street elevation, however, was somewhat unusual.  A slightly projecting central section rose to a triangular pediment above the roof line.  The stately treatment would be seen in civic buildings--schools and police stations, for instance--in later decades.

William Pelletreau said "the entire mansion was finished and furnished in a style commensurate with the wealth and social position of its occupants, and in the days when the avenue was a street of palaces where trade had never set its foot, the 'Spencer mansion' was foremost among its equals."

No. 85 Fifth Avenue was, of course, filled with expensive artwork and furnishings.  Two of its sculptures, by American Henry Kirke Brown, had been purchased earlier, in 1845, for $800.  Brown had been in New York at the time from his studio in Rome trying to decide whether or not to return to America.  According to art historian Sippa Salenius in Sculptors, Painters, and Italy, Spencer gave the artist disappointing advice.  He "strongly advised him to remain in Italy indefinitely, reasoning that America was not a suitable place for an artist."

The Spencers next-door neighbors provide an interesting side note, given the early Victorian period.  Edwin Penfold and Thomas H. Faile purchased the empty lot on September 18, 1852 for $10,000.  They erected No. 87 Fifth Avenue and, according to Pelletreau, "They were both bachelors and both wealthy, and here they made their homes during the remainder of their lives, living in a style of most elegant leisure, and evidently studying their own enjoyment more than anything else."

Captain Spencer would not enjoy his luxurious home for long.  He died on March 3, 1854, passing title to the the mansion to Lorillard.   Catherine, of course, remained in the house and moved among society and involved herself in charitable causes.

On February 14, 1874 The New York Herald informed its readers that the Home of the Destitute Children of Seamen on Staten Island was over-taxed.  "The many recent disasters at sea and the hard times have brought many applicants for the care of the institution," it explained.  The article said that any donations of clothing or food would be accepted "at Mrs. Spencer's, No. 85 Fifth avenue, corner Sixteenth street."  And on July 2, 1879 The New York Herald reported that she had opened her cottage in Newport.

Catherine died in the house in March 1882.  Lorillard and his family continued on at No. 85.  He and his wife, the former Sarah Johnson Griswold, were prominent in the highest levels of society.  When, for example, Caroline (known familiarly as Carrie) Astor, the daughter of William B. and Caroline Schermerhorn Astor, married Orme Wilson on November 18, 1882, the Spencers were on the exclusive guest list.

The weddings of the Spencers' own eight children drew much attention, as well.  Eleanor was the envy of other young heiresses when she married Prince Virginio de Vicovaro Cenci in 1870, earning her the title of Princess de Vicovaro.  And when Lorillard Spencer, Jr. married Caroline Suydam Berryman in St. Thomas's Church on October 3, 1882, The New York Times called it "the first fashionable wedding of the season."

Although he and Sarah spent much of their time in Paris, Lorillard stubbornly refused to abandon lower Fifth Avenue.  As society migrated up the avenue, he and a handful of wealthy neighbors fought to maintain their exclusive enclave.  Following his death in Paris on January 20, 1888 at the age of 62, title to the house passed to three sons, Charles, Lorillard and William Augustus.

Sarah now spent almost all of her time abroad and the mansion was almost immediately leased to Levi Parsons Morton who became United States Vice President in March the following year.   In 1888 Gilbert L. Harney hinted at the encroachment of business buildings around the house in his The Lives of Benjamin Harrison and Levi P. Morton. "If it were standing alone, it would be considered magnificent in its outward appearance.  but crowded among so many--and some of them much larger--monuments of architectural skill, it assumes modest proportions."

Harney described, "There is a wide hall running from the street doors to the dining-room at the rear.  On the right wall hangs a large portrait of President Garfield.  On the left is a large painting, by Constant, of an Eastern dwelling.  The door on the right leads into the parlors; the door on the left into the library.  The stairway, also, leads up from the hall.  The dining-room, at the end of the halls, is almost as wide as the house."

On November 14, 1890 The Evening World said of Morton, "He is immensely rich, and has a $100,000 home in Washington, a 950 acre farm at Rhinecliff-on-the-Hudson, which he calls 'Eilersiie Stock Farm,' and a magnificent brown-stone house at 85 Fifth avenue, corner of Sixteenth street."

Although the Mortons spent most of their time in Washington, they continued to lease the Fifth Avenue house through 1894.  There was a bit of trouble the month after the Vice President's term of office came to an end in 1893.

On April 16 the Mortons were out of town.  The mansion was in charge of a caretaker and his wife.  The couple took advantage of having the commodious house at their disposal; going so far as giving the caretaker's father, her mother and her brother took up temporary residence.  And they helped themselves to the Morton wine cellar, as well.

The New York Times reported "Pedestrians in Fifth Avenue, near Sixteenth Street, about 10'o'clock last night, were startled by cries of 'Murder! Help! He is killing me!' issuing from the conservatory of the residence of ex-Vice-President Levi P. Morton, at 85 Fifth Avenue."

A concerned crowd gathered on the street, "wondering who was being murdered in the ex-Vice-President's house."  A policeman, Edward Dilon, rang the bell, which was opened the brother, described by the newspaper as a "short, red-faced man."  He attempted to diffuse the problem by explaining that the caretaker and his wife had simply gotten into a quarrel.

Then his sister appeared.  "The caretaker's wife, who seemed to be almost delirious from drink, asked the officer to arrest her husband, saying that he had beaten her."  Officer Dilon saw no evidence of physical violence.  He declined to make the arrest and left.

Now, swearing "vigorously," the red-faced brother ordered the crowd to leave.  When one "big brawny" man did not move fast enough from in front of the stoop, the irate brother rushed down and struck him.  It was a bad idea.

"The big man seized the little man and pounded him," reported the article.  But when the smaller man pulled out a large knife, "the big man deeming discretion the better part of valor, took to his heels and started toward Union Square."  The vengeful red-faced man was in close pursuit until he tripped and fell, slashing himself across the check with his weapon.  "By the time the little man had picked himself up his big foe had disappeared," concluded the report.

Morton was elected Governor of New York in the fall of that year, ending his occupancy of No. 85.  In the meantime, Sarah Spencer had purchased a villa in Switzerland in 1892 and continued to maintain her Paris residence.   With the Mortons gone, Charles used the Fifth Avenue mansion on and off before the end of the century; however the neighborhood was no longer fashionable and business buildings were crowding in around it.

The mansion was demolished in 1898, replaced by developer Leo Wise's 13-floor commercial building designed by Louis Korn, which survives.