Spark has a web accessible monitoring interface, as well as, several other mechanisms like remoting, and file-serving that use http protocols. Spark uses the Jetty web server, client and java servlet container for these functionalities.
The web UI is started from within the SparkContext class by instantiated a SparkUI class, which is sub-class of the WebUI class. WebUI uses Spark utility and Jetty utility functions to spin up the web server and attach handlers to server web content.
Current version of Java
$> java -version
What java executable is command line using
$> ls -al /usr/bin/java
What versions of java exist on system
$> /usr/libexec/java_home -v version.xx
… i.e. if you execute..
$> /usr/libexec/java_home -v 1.7
… it’ll output the runtime lib directory. If you you use a version number that’s not installed it’ll just tell you that it can’t find a directory. In which case you need to install it.
You can also list the java frameworks library directory, but I’ve found this to be inconsistent with new java versions I install…
$> ls /System/Library/Frameworks/JavaVM.framework/Versions/
To update the command line java we need to download the jdk, not just the jre that the default “google java download” prompts you to. We then need to make sure our env vars (ie JAVA_HOME) are set in .bash_profile.
JDK 8 Installation for OS X (oracle docs)
OS X Platform Install FAQ
Java SE Development Kit 8 Downloads
Sometimes it’s necessary to have multiple jdk versions on your system depending on what you’re running/compiling. Search for “java archive” (it’s at the bottom) to list previous java versions on downloads page…
Java SE Downloads
If we want to switch between java versions we can use the JAVA_HOME env var..
$> export JAVA_HOME=`/usr/libexec/java_home -v version.xx`