I've been a swimmer my entire life. I've competed at a relatively high level and upon becoming a triathlete, I was consistently the first one out of the water. Now let me tell you, if I had gone out and swam laps for an hour without swimming consistently beforehand, I would be supremely sore. Let me explain.
The best swimmers are very efficient. Each stroke transfers the greatest amount of power to the water while the body maintains the most streamlined possible position. Highly efficient swimmers can cross the pool in surprisingly few strokes. On the other hand, people who can swim, but aren't necessarily excellent swimmers are much less efficient. They may take many more strokes to cross pool because their hands are cavitating (capturing air bubbles as they enter the water) and because they're shortening their stroke, not taking advantage of their full reach.
The result of this can be that the limiting factor in the less efficient swimmer's workout will be cardiovascular. They can swim their ass off and while their heart may be pumping and they're breathing hard, their arms won't get tired or sore because their form prevents them from exerting their maximum potential force on the water.
Unless I'm mistaken and you are a very efficient swimmer already, I would recommend checking out some videos on swimming form on YouTube. Trust me, once you get your stroke nailed and you start taking long, powerful, efficient strokes, you will definitely start to feel the burn in your shoulders and lats.
I'm also curious about how your hour worth of laps is structured. When I hear "swam laps for an hour" I think swam for an hour straight. No stopping on the wall. Swimming for an hour straight is a hell of a workout regardless of your form. I do that as part of base training or when I'm swimming in open water but it's not even that common for me (nor was it very common in conventional swim training). If what you're talking about is swimming for an hour with frequent stops on the wall, that makes a bit more sense. In that case I would say, try lengthening the duration of your swims as you get more efficient and lengthen out those strokes. Also, don't be afraid to swim (or at least with a lower tempo) . Really grabbing the water and taking nice long powerful strokes will give you a much better workout that rapidly thrashing through the pool.
One you get to the point where you can swim for long periods at a time without getting fatigued cardiovascularly, you can start introducing speed work. I'm talking about interval training... 10 x 100's on 1:30 or 2:00, 5 x 200's on 3:00 etc. These will definitely test your cardio if you're pushing hard but make sure you don't sacrifice your stroke. Stay long and smooth, just increase your tempo.
If you get to this point and you really are looking for some consistent workouts, Google "swimming interval workouts" and you will find millions of them. Even better, join a masters swim team and you'll have a coach who can create workouts just for you and where you're at.
In general, swimming is a killer workout, unequalled by any other activity in many regards.
Hope this was helpful.