I am running into an issue with my Maven POM file where its unable to find spark dependency and is returning with error: Failed to read artifact descriptor for org.apache.spark:spark-streaming-kafka_2.10:jar:1.2.1
I've confirmed its not an issue with any corporate firewall as every other dependency is loaded in properly, just this one.
I have also been able to confirm in my maven settings it is attempting to pull from the following repo. I tried deleting the .m2 repo on my local machine in order to reload it, still no dice.
http://repo.maven.apache.org/maven2/org/apache/spark/spark-streaming-kafka_2.10/1.2.1/
Attached below is my pom file
<groupId>my.group.id</groupId>
<artifactId>sentiment</artifactId>
<version>1.0-SNAPSHOT</version>
<name>NPITWITTER</name>
<properties>
</properties>
<dependencies>
<dependency>
<groupId>com.sparkjava</groupId>
<artifactId>spark-core</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka_2.10</artifactId>
<version>1.2.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.2.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.10</artifactId>
<version>1.2.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_2.10</artifactId>
<version>1.2.1</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.10</artifactId>
<version>1.2.1</version>
</dependency>
</dependencies>
In Intellij, I had the same issue,
I did the proxy settings also, and ran the command
mvn –U clean install
but it's not working.I got its resolved with the below setting enabled (highlighted in yellow)
In Intellij, go to File -> Settings -> Build, Execution, Deployments -> Maven Check the box, ‘Always update snapshots’
I guess IntelliJ comes with and ambedded Maven installation, so just make sure your IDE is pointing to the right Maven home directory: and User settings file: values, these are under:
File
Settings
Build, Execution, Deployments
Build Tools
from drop downMaven
from drop downGo to file => Settings => Build, Execution, Deployment => Maven => Maven home directory: change from Bundled (Maven 3) to Bundled (Maven 2)
I had to just solve the issue. Please, go in the Maven tab of the
Build, Execution, Deployments
and set to the settings provided.In my case, it was proxy related problem. I am highlighting few of areas where you may need to verify and correct the same.
Hope this will help.
Resolved it, was due to a proxy setting that was enabled. Re-ran with
mvn -U clean install
and all was repaired.