I had this exact issue when I swapped from using DVI to Display Port a while back. I found two different ways to get vibrance to "stick". You can create a custom resolution in Nvidia control panel with a higher refresh rate than 60hz (not sure if this works for higher refresh rates, but my guess is probably as long as it's higher than whatever you're using now). The downsides to this are that you can't use DSR with a custom resolution and whatever risks are associated with overclocking your monitor.

The second method, which is what I use now, involves using a program called Custom Resolution Utility (CRU) to make it look like a higher refresh rate is native to your monitor. Just adding another resolution with a higher refresh rate fixes the issue. This means you can use your standard refresh rate and DSR with working vibrance, though the higher refresh rate will show up as an option.

How to on using CRU to do this: note: If you plan on using DSR disable it now and enable it again after following these steps.

1. Download CRU from a site called MonitorTests. Just google this. (I'm new to posting so I'm afraid to link stuff in fear of being flagged as a spam bot)
2. Run CRU.exe
3. In the top box labeled "Detailed Resolutions" you should see your current resolution/refresh rate.
4. Click the Add.. button just below that.
5. Select "LCD Standard" from the drop box on the window that pops up.
6. Change the refresh rate at the bottom to anything above 60hz. I chose 67hz. (Don't go overboard with this as monitors usually have an upper limit before they stop working and go back to default. I was able to go up to 75hz before this happened, though my monitor isn't supposed to go that high at 1920x1080) Ideally you will just be using your standard refresh rate so this number doesn't matter as long as it's not super high.
7. Click Ok and you should see the new resolution listed with your native resolution.
8. Before you close CRU change the drop box setting at the bottom of the main window to "Custom extension block (disabled". And click the Edit.. button next to it.
9. Make sure the check box next to Disable custom extension block is checked and hit Ok.
10. Click Ok to close the CRU window, then run Restart64.exe that came with the CRU download.
11. Vibrance should be working now, make sure you open Nvidia control panel and change your refresh rate back to default, or leave it higher if that's what you want.
12. You should be able to enable DSR with vibrance working now.