It kinda wasn't... was an issue, but a major oversimplification at that. The only real way one can break it down, is to look at the different groups of people in society... so to start...
For rich southerners, slavery was an issue but the bigger fear was of industrialization. Northern industry needed cotton for their factories and there were disputes over pricing that created some resentment between the north and the south. The real kicker, however, was the fact that the south was more or less feudal... you had the rich upper class, little to no middle class, a lot of poor people who often worked with or just above the slaves, and then the slaves... and one thing that kept the rich in power was their ability to manipulate the poor which brings us to the next group.
The general, southern populace, which was predominantly poor, was lead to believe that their way of life was at risk. They were told that if they let the North take over, moral abominations of every kind would ensue and they would lose their identity. They would starve (more than what they were) and be treated the same as slaves with no future in sight.
The slaves were just kind of there... some who found out ran off to the north, but a large number of slaves continued to work as slaves even after emancipation because they had no clue that the civil war was going on- thanks to their owners shielding them from the outside world...
The abolitionists were definitely against slavery. A lot of contention between the north and the south came as new states arose in the west... one thing that popped up in government was whether or not a newly created state would be a slave or a free state... and ultimately, an agreement was drafted, which designated some new states as slave and some as free, but this agreement was hated by everyone.
The white middle class/poverty- many were apathetic to the cause, but ultimately, the belief of unity prevailed as many realized that the union between the northern and southern states helped to ensure our sovereignty. This was strengthened even more by Lincoln's leadership and by the realization that the British were keeping a close eye on the war... had the original tide kept going of lost battles and piss poor generals for the north, the British would have flanked the north for the south... With that said, there were some among them who were abolitionists- but most probably didn't care one way or another.
The industry owners, who came together to birth the Republican party were socially progressive, but only as long as it benefitted them. This proved itself when republicans abandoned black politicians and investments in the south after reconstruction. To them, the south was a swath of untamed land and resources that craved industry... farming and cotton were great, but so much was being wasted not to mention it was common knowledge that a slave was and is an inefficient worker compared to a waged worker... Furthermore, many felt like they were being screwed by the plantations...
And then there was Lincoln... Lincoln is a mixed figure because he wasn't the full blown figure of freedom and equality that people think he was. Had he not been shot, more former slaves would have been sent to Liberia and during several speeches, he reiterated his support for segregation. He did free them, but not until later in the war... and it could be argued that because of how the south was holding out at the end, that there was some hope that freedom would cause the slaves to rebel, which would be the last domino to fall in the war.