I'm trying to setup scrapy settings to work with test and production environment on local and also on scrapinghub.
And I would like to know if there is any way to set this variable (for example as the following) on shub deploy:
And then at settings.py:
if env == "test":
var1 = some_ip
var2 = username
elif env == "prod":
var1 = some_ip
var2 = username
Or... maybe there is a cleaner way to this?
Thank you for reading!
PS: I want to automate the settings depending of the environment where is the spider/project located instead of changing the variables manually.
EDIT: Found a better solution. Check the answer.
If you want to control it using environment variables you could do the following: export an environment variable DEPLOY_TARGET
that would contain test
or prod
and execute the following command to deploy
shub deploy $DEPLOY_TARGET
Finally, I solved my problem following these steps:
So now my settings look like this:
- And then my
__init__.py
look like this:
- Next, modified
_environment.py
as the following:
I hope I have been helpful!
PS: Remember that you need to have two projects on scrapinghub, one for production and one for testing to do this, also you need to deploy from the same project as the following:
And then you just have to (reference):
shub deploy test
Deploys to test.
shub deploy prod
Deploys to production.
shub deploy
Deploys to test.