You can read the differences fullly here: https://raw.githubusercontent.com/SeleniumHQ/selenium/master/java/CHANGELOG
Firefox is no more the default driver from v3. To test with FF, you need to use geckodriver, which is based on the evolving W3C WebDriver spec, and uses the wire protocol in that spec, which is liable to change without notice.
Other major changes:
Stability fixes in Grid and all nodes in Grid can now offer help
Updated to the latest version of HtmlUnitDriver
Re-enabled log gathering for standalone server
Firefox profile is passed to both legacy FirefoxDriver and geckodriver
Selenium3.X is no longer capable of running Selenium RC directly, rather it does it through emulation and the WebDriverBackedSelenium interface
As far as your question is concerned, if you should upgrade your tests to the new version of Selenium?
Yes, if you wanna work with the latest Firefox v47.0.1 as well as with the new features of Selenium3, then you should upgrade your tests to new version of selenium. But I would suggest, wait until Selenium v3 if competely released before upgrading to newest version of selenium because beta version may have seen many bugs which could create problems with your test cases.
Learn more about Selenium 4 Features here!