You have to understand the Southern culture of honor. Wiki link.

To southerners, it doesn't matter that we (the royal we) lost the civil war, and it doesn't matter whether we were on the side of right or not. What matters is that we fought for what we believed in. The statues, plaques, and memorials of Confederate leaders, soldiers, battles, and so forth are implicit reminders of the same honor that rolls through us today.

There is a historical component, too, of course, but if history was all we cared about, we could be satisfied by erecting monuments to inventors, builders, teachers, politicians, and more. The monuments of confederate figures are meaningful for other reasons. They aren't commemoratives of great intelligence or great moral virtue. They are commemoratives of courage, sacrifice, and faith. When we look at the statue of Robert E Lee, we don't think, "He tried to protect slavery," (immoral) or "He was a great military strategist" (arguably untrue). We expressly think, "He fought for what he believed in," and "He tried to protect his home."

So I personally think it's a mistake to take down Confederate statues unless they are replaced with something else. Not just modern art or whatever. It should be something that reminds us about our inherent virtues of honesty, industriousness, and courage. Something to carry us away from the uncritical hero worship that has become our reality.