关于SparkStrategies$InMemoryScans$的一个线程问题分析，网上资料比较少，发出来供参考，待解决

就是在写spark数据分析这个栏目的内容，我的内容是从虚拟机中安装idea进行运行程序，但是最后我倒回去运行spark sql时很奇怪地发现以下几个问题

Exception in thread “main” java.lang.NoClassDefFoundError: org/apache/spark/sql/execution/SparkStrategies

Caused by: java.lang.ClassNotFoundException: org.apache.spark.sql.execution.SparkStrategies

Caused by: java.util.zip.ZipException: invalid LOC header (bad signature)

我用了几个方法：

1、第一种是重新导入pom.xml，发现不行

2、重新创建maven项目，不行

3、换了个idea版本重新创建maven项目，不行

主要是网上关于这方面的内容比较少，可参考也比较少，最后也没有进行下去。

最主要的是我用一模一样的内容在windows打开idea创建一个maven进行执行意外发现是可以的。

我可以排除以下几个问题：

1、不会是代码问题，更不可能是找不到什么类

2、也不会是缺失什么jar包或者jar包损坏问题，若损坏重新导入是可以解决的

3、不会是版本冲突问题

等

得出结论是：

1、有可能是在centos中运行idea产生高延迟，某个线程被卡死

2、有怀疑是存储问题，但是没找到具体线索，因为有一段显示我的存储量好像相当少了，虽然我并不太明白什么意思

MemoryStore started with capacity 366.3 MB

最后我还是换到windows使用idea，运行是很正常的

若有友友们有自己的推测，请在评论区说明，我去尝试一下

最后再给出完整报错信息

Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
22/03/22 08:47:11 INFO SparkContext: Running Spark version 2.3.2
22/03/22 08:47:12 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
22/03/22 08:47:12 INFO SparkContext: Submitted application: CaseClassSchema 
22/03/22 08:47:13 INFO SecurityManager: Changing view acls to: root
22/03/22 08:47:13 INFO SecurityManager: Changing modify acls to: root
22/03/22 08:47:13 INFO SecurityManager: Changing view acls groups to: 
22/03/22 08:47:13 INFO SecurityManager: Changing modify acls groups to: 
22/03/22 08:47:13 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
22/03/22 08:47:14 INFO Utils: Successfully started service 'sparkDriver' on port 34298.
22/03/22 08:47:14 INFO SparkEnv: Registering MapOutputTracker
22/03/22 08:47:14 INFO SparkEnv: Registering BlockManagerMaster
22/03/22 08:47:14 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
22/03/22 08:47:14 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
22/03/22 08:47:14 INFO DiskBlockManager: Created local directory at /tmp/blockmgr-6a67837d-e670-4817-926e-439687a00acb
22/03/22 08:47:14 INFO MemoryStore: MemoryStore started with capacity 366.3 MB
22/03/22 08:47:14 INFO SparkEnv: Registering OutputCommitCoordinator
22/03/22 08:47:14 INFO Utils: Successfully started service 'SparkUI' on port 4040.
22/03/22 08:47:14 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://hadoop01:4040
22/03/22 08:47:15 INFO Executor: Starting executor ID driver on host localhost
22/03/22 08:47:15 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 45496.
22/03/22 08:47:15 INFO NettyBlockTransferService: Server created on hadoop01:45496
22/03/22 08:47:15 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
22/03/22 08:47:15 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, hadoop01, 45496, None)
22/03/22 08:47:15 INFO BlockManagerMasterEndpoint: Registering block manager hadoop01:45496 with 366.3 MB RAM, BlockManagerId(driver, hadoop01, 45496, None)
22/03/22 08:47:15 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, hadoop01, 45496, None)
22/03/22 08:47:15 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, hadoop01, 45496, None)
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/sql/execution/SparkStrategies$InMemoryScans$
  at org.apache.spark.sql.execution.SparkStrategies.InMemoryScans$lzycompute(SparkStrategies.scala:401)
  at org.apache.spark.sql.execution.SparkStrategies.InMemoryScans(SparkStrategies.scala:401)
  at org.apache.spark.sql.execution.SparkPlanner.strategies(SparkPlanner.scala:45)
  at org.apache.spark.sql.catalyst.planning.QueryPlanner.plan(QueryPlanner.scala:63)
  at org.apache.spark.sql.execution.QueryExecution.sparkPlan$lzycompute(QueryExecution.scala:72)
  at org.apache.spark.sql.execution.QueryExecution.sparkPlan(QueryExecution.scala:68)
  at org.apache.spark.sql.execution.QueryExecution.executedPlan$lzycompute(QueryExecution.scala:77)
  at org.apache.spark.sql.execution.QueryExecution.executedPlan(QueryExecution.scala:77)
  at org.apache.spark.sql.Dataset.withAction(Dataset.scala:3254)
  at org.apache.spark.sql.Dataset.head(Dataset.scala:2489)
  at org.apache.spark.sql.Dataset.take(Dataset.scala:2703)
  at org.apache.spark.sql.Dataset.showString(Dataset.scala:254)
  at org.apache.spark.sql.Dataset.show(Dataset.scala:723)
  at org.apache.spark.sql.Dataset.show(Dataset.scala:682)
  at org.apache.spark.sql.Dataset.show(Dataset.scala:691)
  at CaseClassSchema$.main(CaseClassSchema.scala:31)
  at CaseClassSchema.main(CaseClassSchema.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.sql.execution.SparkStrategies$InMemoryScans$
  at java.net.URLClassLoader$1.run(URLClassLoader.java:370)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
  ... 17 more
Caused by: java.util.zip.ZipException: invalid LOC header (bad signature)
  at java.util.zip.ZipFile.read(Native Method)
  at java.util.zip.ZipFile.access$1400(ZipFile.java:60)
  at java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:734)
  at java.util.zip.ZipFile$ZipFileInflaterInputStream.fill(ZipFile.java:434)
  at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
  at sun.misc.Resource.getBytes(Resource.java:124)
  at java.net.URLClassLoader.defineClass(URLClassLoader.java:462)
  at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
  ... 23 more

Process finished with exit code 1