I'm building an iOS app in Swift with a Firebase backend. I need advanced search options for my app. I've been told I should use Flashlight/ElasticSearch via this github link https://github.com/firebase/flashlight
Assume my app's name is SneakerSearch and my
Firebase Project_ID is- sneakersearch-az12
Firebase Web_API_Key is- abc123XYZ000...
Firebase App_Url is- gs://sneakersearch-az12.appspot.com
I need some advice with the steps for setting this up as I'm new to Heroku and Node.js. I never learned either before but I've already installed the Heroku tool belt and 'sudo gem heroku install' is done.
I've listed the github directions/steps and the the step(s) I took for each direction. I need assistance with directions: 1,4,5,6,9,10,12, and 15.
1.Install and run ElasticSearch or add Bonsai service via Heroku
1. I make an account at Heroku.
1.-QUESTION: Do I need to create the name of my app and deploy it after I've setup my Heroku account or should I wait until step #9?
2.git clone https://github.com/firebase/flashlight
2.I clone the above into my iOS app's project folder
3.npm install
3.At the prompt I first I run "npm init" then I run "npm install"
4. edit config.js (see comments at the top, you must set FB_URL and FB_SERVICEACCOUNT at a minimum)
4A.-QUESTION: What is "FB_SERVICEACCOUNT"? I'm using Firebase 3 and I couldn't find anything on a service account in my console. Is this another Firebase account that I have to setup for my project using node.js in addition to the Firebase swift project I've already made?
4B.-QUESTION: Where do I "edit config.js" and set my FB_URL and FB_SERVICEACCOUNT at? Should this be in my package.json file?
5.node app.js (run the app)
5.-QUESTION: In terminal I ran "node app.js" and got "throw err;^". Why did I get this error?
6.curl -X POST http://localhost:9200/firebase
6.-QUESTION: What is this for?
7.cd flashlight
7.switched to the flashlight dir
8.heroku login
8.logged in to Heroku
9.heroku create (add heroku to project)
9A-QUESTION: When I first made my account at Heroku should I have created a new app with my app's name and deployed it using the git instructions it has listed? If I should not have done that at first will running "heroku create" manage that process for me?
9B-QUESTION: Do I just run "heroku create" or do I run "heroku create -app's name- here"?
10.heroku addons:add bonsai (install bonsai)
10.-QUESTION: Do I need to make an account at Bonasi.io and install it before this step or does this set up a bonsai account for me? I've never used bonsai before.
11.heroku config (check bonsai instance info and copy your new BONSAI_URL - you will need it later)
11.I guess this question would depend on step #10.
12.heroku config:set FB_NAME=<instance> FB_TOKEN="<token>" (declare environment variables)
12.-QUESTION: What is the Firebase TOKEN? Is this my Web_API_Key, App_URL, or Project_ID? I could not find anything specific to "token" inside my firebase console. I'm using Firebase 3.
13.git add config.js (update)
git commit -m "configure bonsai"
13. commit with message
14.git push heroku master (deploy to heroku)
14. push to master
15.heroku ps:scale worker=1 (start dyno worker)
15.-QUESTION: What is this for?
This is a 2 part answer with the 1st part going over the Github directions and ending on Step 19. The 2nd part extends more info that couldn't fit into the 1st part and it will begin on step 19 in detail. I'll have to add it to another question and link it to this.
Here are the
Github
steps listed in order. FYI I kept the original steps in line with how the originalGithub author
listed them but underneath I put a bunch of sub-steps with detailed explanations and directions under each one.Assuming you already created a
Firebase
project and got yourGoogleService-Info.plist
fileOpen your
GoogleService-Info.plist
file. The followingvariables
from theGithub directions
correlate withGoogleService-Info.plist keys
Underneath each
Github
step I have comments and then directions. Inside the directions where I use the below values you should instead use yourproject's values
. So for the step examples I'll use thesekeys
andvalues
:Swift Data Model: (this will match what you put in your mappings object in Step 19)
VC with file paths where I send data to Firebase and searchSnkPath is a separate file path where Bonsai will run it's searches on:
Inside
FBDatabase
the path where my data to search is stored atroot/searchSnkPath/autoID
and it has 2keys
namedsneakercondition
andsneakername
that represents the data. The path I want to pull my search results from isroot/searchSnkPath
If I want to search on the
searchSnkPath
and query on those 2keys
then inside theconfig.js
file I'd findexports.paths
and inside there is where I would set the info to search onStep 4 covers all this
Important after you run
git clone https://github.com/firebase/flashlight
(step 2A) you need tocd
into theflashlight
folder (step 2B) as ALL the steps from that point on happen inside theflashlight
folder and NOT your main Xcode project's folder. Nothing will not work inside your Xcode project's folder because it doesn't have a package.json filePart 1 - Github steps:
1.comments:
-
Bonsai
will get added at step 10-You DO need to have the following steps 1A-1E in order before you proceed
1.directions:
1A. Download and install
Node.js
from https://nodejs.org/en/download/1B. Download and install the
Heroku Toolbelt
(it's been renamedHeroku CLI
) at https://devcenter.heroku.com/articles/heroku-cli. FYI I used the OS X Installer and not OS X Homebrew.1C. After you've downloaded and installed the
Heroku Toolbelt/CLI
log on to Heroku.com and create an account1D. After creating a
Heroku
account no need to create a newHeroku App
You will do this in step 9 using the command line. It's very easy1E. Open terminal and run:
node -v
to find out the currentnode version
your running (for example you getv6.9.1
). In step 2D you will have to make sure the version from there matches this output.2.comments:
-Assume the Xcode project lives on the desktop inside a folder named
sneakerSearchFolder
-Inside terminal you
cd
to the sneakerSearchFolder folder- i.e. run:cd Desktop/sneakerSearchFolder
-Once inside the sneakerSearchFolder you
clone
the github repo
at https://github.com/firebase/flashlight-Important You then
cd
toflashlight
folder i.e. run:cd flashlight
-You will need the
node version
your running according to https://devcenter.heroku.com/articles/deploying-nodejs-run:
node -v
to get whatnode version
is running inside theflashlight
folder (i.ev6.9.1
) later on when you make aHeroku instance
you will need this according to thedevcenter.heroku
link above-Now that your inside the
flashlight
folder, if thenode version
you just ran doesn't match the version from step 1E, you will need to update the version inside the flashlight folder to make them match. Upgrading Node.js to latest version If both versions match then you don't have to worry about this-Assuming your node versions match, Open the
package.json
file inside yourflashlight
folder and add an"engines"
object with the key's value being thenode version
you are currently using2.directions:
2A. in terminal navigate to whatever folder your Xcode project is in
2B. run:
git clone https://github.com/firebase/flashlight
2C. run:
cd flashlight
2D. run:
node -v
2E. Open the
flashlight
folder then openpackage.json
. Add the following to the file.2E-example. FYI here's an example. Open
package.json
and after the"dependencies"
closing brace
, add it there. BE SURE TO ADD a COMMA after the closing brace. And don't add the "v" before the version number. Save the file.From this point on it's very important that your inside your
flashlight
folder and not your main project's folder for everything to work otherwise you will get errors3.comments:
-You should still be inside your
flashlight
folder-Log into
Firebase
, inside the console of your FB project, go to theProject's Settings
(little round icon next toOverview
), inside Project Settings chooseSERVICE ACCOUNTS
, go to the sectionFirebase Admin SDK
. There are 2 things you have to do here. 1. Find and copy yourFirebase service account
, it looks like firebase-admin-81772@sneakersearch-az12.iam.gserviceaccount.com and 2. at the bottom of the page you will have to click theGenerate New Private Key
button, it will download anUnknown file
that you will need to rename toservice-account.json
. If it names the file something besidesUnknown
just rename that toservice-account.json
. After you rename the file drag it into yourflashlight
folder because Step 4BFB_SERVICEACCOUNT
will need to access this file from there. MAKE SURE YOU PUT THE FILE INSIDE THEFLASHLIGHT FOLDER
!-This isn't listed in the github steps but it's necessary. You must add
Firebase Server SDK Credentials
to your project-You will need to be inside your
flashlight
folder to run the $npm install firebase-admin --save
command or you'll get errors because it will look for apackae.json
file. The file is already inside yourflashlight
folder and not inside your main Xcode project's folder-Follow the directions from https://firebase.google.com/docs/server/setup, inside the
Initialize the SDK
section. You will need 2 of the values from theUnknown
file (which should now be renamedservice-account.json
) to initialize it. The values are on lines 5-private_key
and 6-clientEmail
. FYI theclientEmail
andFirebase service account
are the same thing-Here's what you are
initializing
inside theSDK
:-FYI inside the
Initialize the SDK
section you also have the option of using the top part where it sayspath/to/serviceAccountKey.json
and you can instead supply the path to the renamed Unknown file. I choose the bottominline
part as it was easier. If your following these steps you don't need to worry about this though.-Inside the
flashlight
folder there is anapp.js
file, copy and paste the above code and put it at the top of the file-Back inside terminal on the command line run:
npm install
-If everything is all good the only warnings you should have are
No repository field
andNo license field
3.directions:
3A. Make sure your inside the
flashlight
folder run:pwd
3B. Log into your
Firebase Console
'sSERVICE ACCOUNTS
page and click theGenerate New Private Key
button to download anUnknown
file.3C. Rename the
Unknown
file toservice-account.json
and put the file inside yourflashlight
folder3D. Inside the
service-account.json
file copy thekey
namedclient_email
or you can copy the service account info on yourFirebase SERVICE ACCOUNTS
page3E. Inside terminal run:
npm install firebase-admin --save
3F. Copy what's inside the section
Initialize the SDK
and initialize the fields with these values:3G. Inside the
flashlight
folder, open theapp.js
file, paste/save the above code inside of it with the correct values3H. run:
npm install
4.comments:
-Inside your
flashlight
folder there is a file namedconfig.example.js
, open it-Inside this
config.example.js
file is where you find theFB_URL
andFB_SERVICEACCOUNT
variables (they are listed on lines 13 and 23)-You need to change
exports.FB_URL = process.env.FB_URL || 'https://<YOUR APP>.firebaseio.com';
toexports.FB_URL = process.env.FB_URL || 'whatever_your_DATABASE_URL_is';
(be sure to putwhatever_your_DATABASE_URL_is
inside single or double quotes)-on line 23
exports.FB_SERVICEACCOUNT = process.env.FB_ACC || 'service-account.json';
is what is used to access theservice-account.json file
from step 3C (there is no need to add or change anything here as it will access it on it's own)-You will now have to setup the path where your data sits at (like the searchPath used to send data to FB inside the ViewController above), the index (Step 17), and the type (similar to your data model) that you want to monitor on line 64,65, and 66
-
path
is where yourdata
sits at insideFB
. It's where you want to pull search results from i.e.<DATABASE_URL>/searchSnkPath
-
index
exp in Step 17-
type
exp in Step 19-FYI lines 69-79 would set up another path that you would want to monitor. You can delete or comment these out if you want. You can also create more paths to monitor i.e. If you had a path
<DATABASE_URL>/searchClothingPath
then you would set up search on that also. You can also add more paths if you want to i.e:-FYI
Fields
are thekeys
that will be indexed in ES. This is an optional thing to add. If you had 10keys
and only wanted 2 of them to beindexed
then you would add this. This means of the 10 keys only those 2keys
would be searchable. You can also use the parse function inside theconfig.js
file to do the same thing-Afterwards save/close the
config.example.js
and rename itconfig.js
4.directions:
4A. Open the
config.example.js
file and on line 13 changeexports.FB_URL = process.env.FB_URL || 'https://<YOUR APP>.firebaseio.com';
toexports.FB_URL = process.env.FB_URL || 'https://sneakersearch-az12.firebaseio.com';
4B. Line 23 is what is used to access the
service-account.json
file (as long as you renamed theUnknown file
in step 3C just move on)4C. Line 64,65,66, are what I want to monitor and on line 67 I added the
fields key
for the twoFirebase Database Keys
I want to search on although it is not necessary4D. Save/close the
config.example.js
file. Now rename the fileconfig.js
.5.comments:
-You won't run this command until the very end so skip it for now but there is something you should know. Running
node app.js
runs your Heroku app on your local machine. It will look for ElasticSearch locally which you don't have so you will get connection errors. If you want to run it locally then you will have to run theexport BONSAI_URL="<your_bonsai_url>"
code at Step 12B-Optional which is needed to resolve local connection errors. If your app falls asleep/crashes (exp near the end) you will have to run theexport BONSAI_URL="<your_bonsai_url>"
command again. Other then that Heroku will run the app automatically when you push code up to it.-As of now for this step you need to open your
app.js
file and according to https://docs.bonsai.io/docs/nodejs add some code. You should add the code anywhere after thevar elasticsearch = require('elasticsearch')
declaration.-The directions also says to run $
export BONSAI_URL="https://username:password@my-awesome-cluster-123.us-east-1.bonsai.io"
(in other wordsexport BONSAI_URL="<your_BONSAI_URL>"
) which is what you would do ONLY if you want to run your app locally to use ES. It's not meant for remote5.directions:
5. Open the
app.js
file and add the code below and save the file:6.comments:
-Skip this step because at this point you don't have a local
ES
on your cpu to index6.directions:
6. SKIP THIS STEP
7.comments:
-No need to run this, you should still be inside your
flashlight
folder7.directions:
7. SKIP THIS STEP
8.comments:
-Open terminal
-You will have to enter your
Heroku
'semail address
andpassword.
As you enter the password it will show up blank8.directions:
8A. run:
heroku login
8B. At the prompt enter your
email address
andpassword
to login into Heroku9.comments:
-This is why you didn't need to create a
heroku
app at step 1D. Now you will create and name your heroku app-After you type in
create
you need to type in whatever name you want your heroku app's name to be. i.e. I choose the namesneakersearchinstanceAtoZ
-After you run this command if you login into
Heroku
and you should see the app's name listed. Go to the right corner, click the9 dots
, chooseDashboard
9.directions:
9. run:
heroku create sneakersearchinstanceAtoZ
10.comments:
-You will need to add a
credit card
to yourHeroku
account before you run this step otherwise after you run it it will say you have to do it. It's still a free plan though. I'm not sure if you can add aBonsai add on
without a card-To add a credit card go all the way to the right side, click the circular icon, choose
Account Settings
, then chooseBilling
-You now need to add
bonsai
to yourheroku app
. You do this by runningheroku addons:add bonsai --app <your-app-name>
-
<your-app-name>
is whatever theheroku
app's name you created in step 9-after you create a
bonsai cluster
it will then want you to open up the cluster by runningheroku addons:open bonsai
10.directions:
10A. log on to
Heroku
and add a credit card to theBilling Page
10B. run:
heroku addons:add bonsai --app sneakersearchinstanceAtoZ
//same exact name from step 910C. run
heroku addons:open bonsai
11.comments:
-Running
heroku config
will get all yourenvironment variables
and their corresponding values. When you see yourbonsai's url
, copy it's value, you'll need it for steps 12B and 17-This is the exact same url you will see from step 10C
-The
bonsai url
might look something likehttps://abc123a01:01abc12de45xyz34@xyz-012345.us-east-1.bonsaisearch.net
11.directions:
11A. run:
heroku config
11B. Copy the
bonsai url
that is returned12.comments:
-12A is REQUIRED. You need to use your project's
GoogleService-Info.plist
'sPROJECT_ID
which is theFB_NAME
and theAPI_KEY
which is theFB_TOKEN
for this step-12B is optional. If you want to connect to
ElasticSearch
locally run this. You will have to set yourbonsai url
so that runningnode app.js
won't throw connection errors. You do this by running:export BONSAI_URL="<your_bonsai_url>"
. You will need to run this command per terminal session or if your Heroku app falls asleep/crashes.-Do not use spaces before or after the equal sign
12.directions:
12A(REQUIRED). run:
heroku config:set FB_NAME=sneakersearch-az12 FB_TOKEN=0012abc789xyz00019
12B(Optional).run:
export BONSAI_URL="https://abc123a01:01abc12de45xyz34@xyz-012345.us-east-1.bonsaisearch.net"
13.comments:
-If you run this you might get an error that says: The following paths are ignored by one of your .gitignore files: config.js Use -f if you really want to add them.
-It's saying to run
git add config.js
then add-f
at the end13.directions:
13A. run:
git add config.js
13B. If there is an error run:
git add config.js -f
14.directions:
14. run:
git commit -m "configure bonsai"
15.directions:
15. run:
git push heroku master
16.comments:
-the answer to what this command does is here Can someone explain "heroku ps:scale web=1"
16.directions:
16. run:
heroku ps:scale worker=1
17.comments:
-Before you begin this step read @DoesData updated answer (below mines) to use curl -X PUT instead of POST
-Grab the
bonsai url
you copied from step 11B-Paste the url after the
curl -X POST
command and make sure you add/firebase
to the end of it-what this does is it creates an index named
/firebase
that points to yourbonsai url
. Open yourconfig.js
file and look forexports.paths
(line 62). That has a key/value pair namedindex: "firebase"
(line 65) , the value is pointing to the index you just created and accesses the ES cluster from there-Basically in your config.js file, on line 65 the
index
valuefirebase
has to exactly match the name/firebase
that your adding to the endcurl -X POST
command. If the names don't match then none of this will work.-if successful you should get this response back:
{"acknowledged":true}
17.directions:
17. run:
curl -X POST https://abc123a01:01abc12de45xyz34@xyz-012345.us-east-1.bonsaisearch.net/firebase
18.comments:
-running this command will start the app locally. You can also run
npm start
but either way it will launch it on your local machine and you will need to run the code in Step 12B(Optional) for this to work18.directions
18. run:
node app.js
FYI Step 19 is very important. You MUST set your MAPPINGS OBJECT and it's KEYS. I don't have enough chars left to explain it in detail here. This is what Step 4's
type
key uses to get set by. Watch this video to explain how to set yourmappings object
: https://www.youtube.com/watch?v=h3i3pqwjtjA&feature=youtu.be19.directions:
19A. --open
TextEdit
orSublime
then create a file and name itsneakerfile.json
. Save it and drag the file into yourflashlight
folder. You need the.json
extension.19B. -Inside that file add the following code, save and close the file:
19C. grab your
BONSAI_URL
runheroku config:get BONSAI_URL
. My BONSAI_URL is https://abc123a01:01abc12de45xyz34@xyz-012345.us-east-1.bonsaisearch.net19D. run:
curl -XPOST https://abc123a01:01abc12de45xyz34@xyz-012345.us-east-1.bonsaisearch.net/firebase/sneakers/_mapping -d@sneakerfile.json
19E. run:
curl -XGET <BONASI_URL>/firebase/sneakers/_mapping?pretty
Done with Part 1. Part 2 will explain Step 19 in detail
Further Info-
Heroku commands:
To check on your
dyno
run:
heroku ps
If you get the following then refer to the links below
After creating your
Heroku app
yourdyno
is under theHeroku Free Plan
. It will eventuallyfall asleep
/crash
if it's not being used within a certain time frame. To stop sleeping/crashing followEasy way to prevent Heroku idling?
https://devcenter.heroku.com/articles/free-dyno-hours
https://devcenter.heroku.com/articles/dyno-sleeping
Other commands:
To get back to the
command prompt
pressCtrl+C
You can also contact Heroku Support or Bonsai Support as they are very helpful
I will post Part 2 very soon.
The above answer is great. However, there are far too many comments for my update to be useful there so I'm placing it here.
If you follow the above steps you will encounter an error on step:
Newer versions of elasticSearch no longer allow you to create a new index using POST, but you can use PUT. So you want to run
and the response should be:
{"acknowledged":true,"shards_acknowledged":true}
See this question for more information.
Update
I also switched over to Algolia because the documentation was a lot more clear. However, I would say that the elastic search documentation is in drastic need of an update. I would recommend that those interested in elastic search try to deploy their code using Firebase cloud functions rather than Heroku. I haven't been able to find a guide for this, but the node.js code should be similar except it will be deployed to the Firebase cloud instead of Heroku. This should make the process a lot simpler. You can check out Firebase cloud functions here