Your assumption is correct. You only start to pay gift tax once your total gifts to anyone and everyone (besides your spouse) exceeds the lifetime gift exemption of $11.58mm. That’s not just to one person, but the sum of all gifts to everyone.

If you gift more than $15k to a single person in a single year, you have to file a gift tax return, even if you don’t owe any taxes on it because you haven’t yet gifted more than the lifetime gift exemption. The IRS just wants a record of it so they can begin to sum your gifts in your lifetime.

That’s the lifetime gift tax exemption for this year. If new tax legislation is passed, it’s (somewhat) possible a reduction in the lifetime gift exemption will be included. However any gift amounts given in previous tax years that are over whatever new lifetime gift exemption amount won’t be taxed, they will be “grandfathered” in.

If you’re dealing with these kinds of numbers, I’m assuming (and recommending) you consult with an estate attorney.