That is exactly what you want to do. By default, the plugin uses h3 tags for the titles. You can adjust the margins in your themes stylesheet to your liking. If you would prefer to not style all h3 tags on your site and only target the accordion titles, you can use the class name instead:
h3.accordion-title {
/* Your styles here */
}
You can also use .accordion-content
to style the content block.