Run Cypher for Apache Spark examples (CAPS)

Question

I know that this a wide question, but it would be helpful for neo4j users those who aren't in a field of scala programming.

I am in need to use Cypher for Apache Spark Project in order to create a Neo4j graph from a data-frame and then storing it into Neo4j.

I tried to integrate the project in eclipse, but with no luck , i get a wired error, so i couldn't get the help of an IDE.

What i need is a way to run the Scala example classes of this maven project ??

In another scala maven project a contributor give me this cmd :

mvn scala:run -DmainClass=org.opencypher.example.HelloCypher

I tried to trick it, but the trick dosen't work.

Answer 1

I found a way to run the Cypher for Apache Spark Project .

For me what i need is the InputDataFrameExample,(which creates a graph from a dataframe and then store it in neo4j).

Without an IDE

The unique environnement that works for me is Linux(Ubuntu 18 for me) , in windows 10 the project is simply dosen't work without an ide , you can find why in this issue .

The steps to make it run in ubuntu are :

1)Download CAPS-Cypher for Apache Spark

2) run mvn clean install in the project folder

3) start your neo4j , run service neo4j start

4) add this code(config of the connection to your Neo4j) to InputDataFrameExample(add only what is missing):

import org.opencypher.spark.api.CAPSSession

import java.net.URI

import org.opencypher.spark.api.io.neo4j.Neo4jPropertyGraphDataSource

import org.opencypher.spark.api.io.neo4j.Neo4jConfig

import org.opencypher.okapi.api.graph.GraphName



 val session: CAPSSession = CAPSSession.local()

 val boltWriteURI: URI = new URI("bolt://localhost:7687")

 val neo4jWriteConfig: Neo4jConfig = new Neo4jConfig(boltWriteURI, "neo4j", Some("your_Neo4j_password"), true)

 val neo4jResult: Neo4jPropertyGraphDataSource = new Neo4jPropertyGraphDataSource(neo4jWriteConfig)(session)

 val neo4jResultName: GraphName = new GraphName("neo4jgraph");

  neo4jResult.store(neo4jResultName, yourGraph);

5) run mvn scala:run -DmainClass=org.opencypher.spark.examples.DataFrameInputExample inside of the spark-cypher-examples

6) finally you could point your browser to

localhost:7474 and run the cypher query match(n) return n to see if that works or not.

With an IDE (IntelliJ IDEA)

IntelliJ IDEA can run the project in the two environments (LINUX & WINDOWS)

these are the steps in order to run CAPS with IntelliJ IDEA:

1) download IntelliJ IDEA and install it.

2) Open IntelliJ IDEA click on Configure -> Click on Plugins -> Click on Install JetBrains Plugins -> Search for Scala -> Click on install button => this will install the scala plugin needed by the project

3) Download CAPS-Cypher for Apache Spark

4) Import your project in intelliJ IDEA (click on import project >external resources and choose maven then next until the project is open)

important : you need to have jdk 8

5) add this code(config of the connection to your Neo4j) to InputDataFrameExample(add only what is missing):

import org.opencypher.spark.api.CAPSSession

import java.net.URI

import org.opencypher.spark.api.io.neo4j.Neo4jPropertyGraphDataSource

import org.opencypher.spark.api.io.neo4j.Neo4jConfig

import org.opencypher.okapi.api.graph.GraphName



 val session: CAPSSession = CAPSSession.local()

 val boltWriteURI: URI = new URI("bolt://localhost:7687")

 val neo4jWriteConfig: Neo4jConfig = new Neo4jConfig(boltWriteURI, "neo4j", Some("your_Neo4j_password"), true)

 val neo4jResult: Neo4jPropertyGraphDataSource = new Neo4jPropertyGraphDataSource(neo4jWriteConfig)(session)

 val neo4jResultName: GraphName = new GraphName("neo4jgraph");

  neo4jResult.store(neo4jResultName, yourGraph);

6) start your neo4j service: service neo4j start and bin\neo4j.bat start on windows

7) back to IntelliJ IDEA and run the class (Inputdataframe) as Application

8) finally you could point your browser to

localhost:7474 and run the cypher query match(n) return n

to see if that works or not.

Answer 2

I am pretty much in the same boat. I have been able to build the project. You will need to make some changes in the POM.xml. Defaults did not work for me

I did below:

1. Create a new Maven project
2. Registered all okapi artifact dependencies and dependencies for other modules such as spark-cypher
3. Add Scala nature to the project
4. Did a maven build and maven install from eclipse
5. Build was success

Post that I included org package under src/main/scala

1. Ran CustomDataFrameInputExample.scala as Scala application. It ran fine.
2. Still struggling while reading from CAPS local session.

Hope this helps.