How to configure Cygnus in relation to Orion and C

Question

We have Orion, Cygnus, and Cosmos installed, and are trying to get the connections between them working: via the broker Orion messages are to be forwarded to Cygnus, which in turn is to write those to the Cosmos database.

We know that Orion is working properly (it has been tested and used before), and have tested Cygnus with the test python script (as explained in https://github.com/telefonicaid/fiware-cygnus/blob/master/doc/quick_start_guide.md). Currently we are trying to configure Cygnus so that it receives messages from Orion, and subsequently writes them to the Cosmos database.

1) How do we configure Cygnus/ which parameters need to set, in order to obtain the simplest working link from Orion, via Cygnus, to Cosmos? Is there a simple working example for us to see? (is the agent_1.conf the only config file that needs to be set in Cygnus?)

2) How/by what means do we subscribe Cygnus to Orion?

3) And how do we persist the data to Cosmos?

We have read quite a few answers to similar questions here on StackOverflow; and have read documentation regarding the above on either Github or the Fiware site, but seem to not be able to get it to work...

Many thanks!

Answer 1

Assuming you installed Cygnus from RPM, the minimum configuration you need is about giving values to this set of parameters in a /usr/cygnus/conf/agent_1.conf file (there exists a /usr/cygnus/conf/agent.conf.template file that you may copy):

cygnusagent.sources = http-source
cygnusagent.sinks = hdfs-sink
cygnusagent.channels = hdfs-channel

cygnusagent.sources.http-source.type = org.apache.flume.source.http.HTTPSource
cygnusagent.sources.http-source.channels = hdfs-channel
cygnusagent.sources.http-source.port = 5050
cygnusagent.sources.http-source.handler = com.telefonica.iot.cygnus.handlers.OrionRestHandler
cygnusagent.sources.http-source.handler.notification_target = /notify
cygnusagent.sources.http-source.handler.default_service = def_serv
cygnusagent.sources.http-source.handler.default_service_path = def_servpath
cygnusagent.sources.http-source.handler.events_ttl = 10
cygnusagent.sources.http-source.interceptors = ts gi
cygnusagent.sources.http-source.interceptors.ts.type = timestamp
cygnusagent.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.GroupingInterceptor$Builder
cygnusagent.sources.http-source.interceptors.gi.grouping_rules_conf_file = /usr/cygnus/conf/grouping_rules.conf

cygnusagent.sinks.hdfs-sink.type = com.telefonica.iot.cygnus.sinks.OrionHDFSSink
cygnusagent.sinks.hdfs-sink.channel = hdfs-channel
cygnusagent.sinks.hdfs-sink.enable_grouping = false
cygnusagent.sinks.hdfs-sink.hdfs_host = cosmos.lab.fiware.org
cygnusagent.sinks.hdfs-sink.hdfs_port = 14000
cygnusagent.sinks.hdfs-sink.hdfs_username = hdfs_username
cygnusagent.sinks.hdfs-sink.hdfs_password = xxxxxxxx
cygnusagent.sinks.hdfs-sink.oauth2_token = xxxxxxxx
cygnusagent.sinks.hdfs-sink.file_format = json-column
cygnusagent.sinks.hdfs-sink.hive_server_version = 2
cygnusagent.sinks.hdfs-sink.hive_host = cosmos.lab.fiware.org
cygnusagent.sinks.hdfs-sink.hive_port = 10000
cygnusagent.sinks.hdfs-sink.krb5_auth = false

cygnusagent.channels.hdfs-channel.type = memory
cygnusagent.channels.hdfs-channel.capacity = 1000
cygnusagent.channels.hdfs-channel.transactionCapacity = 100

In addition to the above, you will need:

• This file /usr/cygnus/conf/grouping_rules.conf to be created; you may make a copy from /usr/cygnus/conf/grouping_rules.conf.template.
• You will need an account in the global instance of Cosmos, you may get it at https://cosmos.lab.fiware.org. Once registered, the portal will print your credentials. Both username and password must be configured in the file above.
• You will need an OAuth2 token, you may get it at https://cosmos.lab.fiware.org:13000 as explained here. Once obtained, it must be configured in the file above.

Now, you are able to run Cygnus as a standard application:

$ /usr/cygnus/bin/cygnus-flume-ng agent --conf /usr/cygnus/conf -f /usr/cygnus/conf/agent_1.conf -n cygnusagent -Dflume.root.logger=INFO,console

If you want to run Cygnus as a service, you will need an additional /usr/cygnus/conf/cygnus_instance_1.conf (there is another template within the installation) with the following default content:

CYGNUS_USER=cygnus
CONFIG_FOLDER=/usr/cygnus/conf
CONFIG_FILE=/usr/cygnus/conf/agent_<id>.conf
AGENT_NAME=cygnusagent
LOGFILE_NAME=cygnus.log
ADMIN_PORT=8081
POLLING_INTERVAL=30

Then, you can proceeed as with any other service:

$ service cygnus start