If you have index.php within the WordPress URLs, there are three main ways to remove this.
#1 Go to Settings > Permalinks and check under Custom Structure to see if this has been included as part of the Permalink. If so remove this or change it and retest your pages and posts to ensure they work correctly.
#2 ensure this line of code is within your .htaccess file in the root of your WordPress installation
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress#3 Check with your hosting company or your server administrator that mod_rewrite has been enabled on your server.
Ensure you backup your website before you make these changes and test your homepage, posts, custom posts and pages afterwards.