JAMIN

Alt is crucial part of SEO because it describe the image for engines and screen readers. whereas, the caption is optional and can be used only when you need to provide additional information about the image to website visitors.

FOR CAPTION
to display caption is an easy task. just you need to put this code where needed ,
the_post_thumbnail_caption();

If you want to displey only if the caption is available then you can do the following code.
if(the_post_thumbnail_caption())
{
the_post_thumbnail_caption();
}

FOR ALT
To display image alt in three section.
1st: Get Thumbnail ID
You can get thumnail id by get_post_thumbnail_id(get_the_ID())
2nd: get alt by help of id
get_post_meta($thumb_id,’_wp_attachment_image_alt’,true);
3rd: display the alt if exist
if(count($alt)) {echo $alt; }

Combining all you can write in this method

$thumb_id=get_post_thumbnail_id(get_the_ID());
$alt= get_post_meta( $thumb_id, ‘_wp_attachment_image_alt’, true );
if(count($alt)) {echo $alt; }