One of the first occurred in Columbus, Miss., April 25, 1866, when a group of women visited a cemetery to decorate the graves of Confederate soldiers who had fallen in battle at Shiloh. Nearby were the graves of Union soldiers, neglected because they were the enemy. Disturbed at the sight of the bare graves, the women placed some of their flowers on those graves, as well. [source]
The day of traditional observance comes from a veterans' organization:
HEADQUARTERS GRAND ARMY OF THE REPUBLIC
Washington D.C. May 5, 1868
General Orders No. 11
I. The 30th day of May, 1868, is designated for the purpose of strewing with flowers, or other decorating the graves of comrades who died in defense of their country during the late rebellion, and whose bodies now lie in almost every city, village and hamlet churchyard in the land. In this observance no form of ceremony is prescribed, but Posts and comrades will, in their own way, arrange such fitting services and testimonials of respect as circumstances may permit.
We are organized, Comrades, as our regulations tell us, for the purpose among other things, “of preserving and strengthening those kind and fraternal feelings which have bound together the soldiers, sailors and marines who united to suppress the late rebellion.” What can aid more to assure this result than by cherishing tenderly the memory of our heroic dead? We should guard their graves with sacred vigilance. All that the consecrated wealth and taste of the nation can add to their adornment and security is but a fitting tribute to the memory of her slain defenders. Let pleasant paths invite the coming and going of reverent visitors and fond mourners. Let no neglect, no ravages of time, testify to the present or to the coming generations that we have forgotten as a people the cost of a free and undivided republic.
If other eyes grow dull and other hands slack, and other hearts cold in the solemn trust, ours shall keep it well as long as the light and warmth of life remain in us.
Let us, then, at the time appointed, gather around their sacred remains and garland the passionless mounds above them with choicest flowers of springtime; let us raise above them the dear old flag they saved; let us in this solemn presence renew our pledge to aid and to assist those whom they have left among us as a sacred charge upon the Nation’s gratitude – the soldier’s and sailor’s widow and orphan.
II. It is the purpose of the Commander-in-Chief to inaugurate this observance with the hope that it will be kept up from year to year, while a survivor of the war remains to honor the memory of his departed comrades. He earnestly desires the public press to call attention to this Order, and lend its friendly aid in bringing it to the notice of comrades in all parts of the country in time for simultaneous compliance therewith.
Department Commanders will use every effort to make this Order effective. By Command of:
John A. Logan