At Avila, the launch at the north side of the bay at Port San Luis Beach is sheltered and very easy launch. It's a short paddle across the bay to the sea caves at Fossil Point. Tour operators run guided tours to the sea caves the same as they do in La Jolla.
There's also a couple of usually easy launches further south at Shell Beach.
Gaviota is a great place to paddle, but as Hunters Pa said, the surf can be an issue. I guess if the surf's up you could launch and land from the pier using the boat hoist.
Hope you have a great time.