To remedy this, go to "Pages and Files" and search for the titles of the two pages. Click on the small arrow adjacent to the file's name and then click "redirect". An option will appear to remove the redirect (you can do this for one or both pages).
Rather than using a redirect, consider simply renaming the page. If you do so, you must be sure to update all links to the newly renamed page, but the renaming feature is a good way to keep page names consistent if you have one that was initially set up incorrectly.
When renaming a page, you will be prompted to set up a redirect. You may do this, but beware of redirect loops.