編譯器無法啟動
1. 用vs編譯C++文件時無法啟動程序,並且找不到指定文件怎麼辦
1、用 VS 打開工程,點擊菜單 項目 - 屬性。
2. Spark Shell因為Scala編譯器原因不能正常啟動怎麼解決
Spark Shell由於Scala編譯器原因不能正常啟動
使用SBT安裝完成Spark後,可以運行示例,但是嘗試運行spark-shell就會報錯:
D:\Scala\spark\bin\spark-shell.cmd
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J:
Found binding in
[jar:file:/D:/Scala/spark/assembly/target/scala-2.10/spark-assembly-0.9.0-incubating-hadoop1.0.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/D:/Scala/spark/tools/target/scala-2.10/spark-tools-assembly-0.9.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
14/04/03 20:40:43 INFO HttpServer: Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
14/04/03 20:40:43 INFO HttpServer: Starting HTTP Server
Failed to initialize compiler: object scala.runtime in compiler mirror not found.
** Note that as of 2.8 scala does not assume use of the java classpath.
** For the old behavior pass -usejavacp to scala, or if using a Settings
** object programatically, settings.usejavacp.value = true.
14/04/03
20:40:44 WARN SparkILoop$SparkILoopInterpreter: Warning: compiler
accessed before init set up. Assuming no postInit code.
Failed to initialize compiler: object scala.runtime in compiler mirror not found.
** Note that as of 2.8 scala does not assume use of the java classpath.
** For the old behavior pass -usejavacp to scala, or if using a Settings
** object programatically, settings.usejavacp.value = true.
Failed to initialize compiler: object scala.runtime in compiler mirror not found.
at scala.Predef$.assert(Predef.scala:179)
at
org.apache.spark.repl.SparkIMain.initializeSynchronous(SparkIMain.scala:197)
at
org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:919)
at
org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:876)
at
org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:876)
at
scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:876)
at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:968)
at org.apache.spark.repl.Main$.main(Main.scala:31)
at org.apache.spark.repl.Main.main(Main.scala)
Google
之還是不求解。只是在SBT的網站上看到Q&A裡面有個問題提到了:http://www.scala-sbt.org/release
/docs/faq#how-do-i-use-the-scala-interpreter-in-my-code。這里說代碼中怎麼修改設置。顯然不
適合我。
繼續求解。注意到錯誤提示是在2.8以後才有的,原因是有一個關於編譯器解釋權Classpath的提議被接受了:Default compiler/interpreter classpath in a managed environment。
繼續在Google中找,有一篇論文吸引了我的注意:Object Scala Found。裡面終於找到一個辦法:
「
However, a working command can be recovered, like so:
$ jrunscript -Djava.class.path=scala-library.jar -Dscala.usejavacp=true -classpath scala-compiler.jar -l scala
」
於是修改一下\bin\spark-class2.cmd:
rem Set JAVA_OPTS to be able to load native libraries and to set heap size
set
JAVA_OPTS=%OUR_JAVA_OPTS% -Djava.library.path=%SPARK_LIBRARY_PATH%
-Dscala.usejavacp=true -Xms%SPARK_MEM% -Xmx%SPARK_MEM%
rem Attention: when changing the way the JAVA_OPTS are assembled, the change must be reflected in ExecutorRunner.scala!
標紅的部分即是心添加的一個參數。再次運行\bin\spark-shell.cmd:
D:>D:\Scala\spark\bin\spark-shell.cmd
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J:
Found binding in
[jar:file:/D:/Scala/spark/assembly/target/scala-2.10/spark-assembly-0.9.0-incubating-hadoop1.0.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/D:/Scala/spark/tools/target/scala-2.10/spark-tools-assembly-0.9.0-incubating.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
14/04/03 22:18:41 INFO HttpServer: Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
14/04/03 22:18:41 INFO HttpServer: Starting HTTP Server
Welcome to
____
__
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 0.9.0
/_/
Using Scala version 2.10.3 (Java HotSpot(TM) Client VM, Java 1.6.0_10)
Type in expressions to have them evaluated.
Type :help for more information.
14/04/03 22:19:12 INFO Slf4jLogger: Slf4jLogger started
14/04/03 22:19:13 INFO Remoting: Starting remoting
14/04/03 22:19:16 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://spark@Choco-PC:5960]
14/04/03 22:19:16 INFO Remoting: Remoting now listens on addresses: [akka.tcp://spark@Choco-PC:5960]
14/04/03 22:19:16 INFO SparkEnv: Registering BlockManagerMaster
14/04/03
22:19:17 INFO DiskBlockManager: Created local directory at
C:\Users\Choco\AppData\Local\Temp\spark-local-20140403221917-7172
14/04/03 22:19:17 INFO MemoryStore: MemoryStore started with capacity 304.8 MB.
14/04/03 22:19:18 INFO ConnectionManager: Bound socket to port 5963 with id = ConnectionManagerId(Choco-PC,5963)
14/04/03 22:19:18 INFO BlockManagerMaster: Trying to register BlockManager
14/04/03 22:19:18 INFO BlockManagerMasterActor$BlockManagerInfo: Registering block manager Choco-PC:5963 with 304.8 MB RAM
14/04/03 22:19:18 INFO BlockManagerMaster: Registered BlockManager
14/04/03 22:19:18 INFO HttpServer: Starting HTTP Server
14/04/03 22:19:18 INFO HttpBroadcast: Broadcast server started at http://192.168.1.100:5964
14/04/03 22:19:18 INFO SparkEnv: Registering MapOutputTracker
14/04/03
22:19:18 INFO HttpFileServer: HTTP File server directory is
C:\Users\Choco\AppData\Local\Temp\spark-e122cfe9-2d62-4a47-920c-96b54e4658f6
14/04/03 22:19:18 INFO HttpServer: Starting HTTP Server
14/04/03 22:19:22 INFO SparkUI: Started Spark Web UI at http://Choco-PC:4040
14/04/03 22:19:22 INFO Executor: Using REPL class URI: http://192.168.1.100:5947
Created spark context..
Spark context available as sc.
scala> :quit
Stopping spark context.
14/04/03 23:05:21 INFO MapOutputTrackerMasterActor: MapOutputTrackerActor stopped!
14/04/03 23:05:21 INFO ConnectionManager: Selector thread was interrupted!
14/04/03 23:05:21 INFO ConnectionManager: ConnectionManager stopped
14/04/03 23:05:21 INFO MemoryStore: MemoryStore cleared
14/04/03 23:05:21 INFO BlockManager: BlockManager stopped
14/04/03 23:05:21 INFO BlockManagerMasterActor: Stopping BlockManagerMaster
14/04/03 23:05:21 INFO BlockManagerMaster: BlockManagerMaster stopped
14/04/03 23:05:21 INFO SparkContext: Successfully stopped SparkContext
14/04/03 23:05:21 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
14/04/03
23:05:21 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon
shut down; proceeding with flushing remote transports.
Good。瀏覽器打開http://Choco-PC:4040,就可以看到Spark的狀態、環境、執行者等信息了。
這個Fix可能只是適用與我的情況。如果還有問題可以再找找相關的資料。
期間還碰到不能找到文件的錯誤。最後發現是JAVA_HOME設置沒有對。如果你碰到問題了,可以打開腳本的回顯,然後找找原因。