Using Eve's DB layer without HTTP

Question

In my application, the MongoDB collections need to be updated by a server-side script job (IE: a cron job that scrapes/pulls from other APIs every 30minutes). What I really want to do is make updates to the MongoDB collections, but have the data be validated against the schema and include metadata (updated, created, etc).

The two ways that come to mind to solve this is:

1. Have a fake client to do HTTP POST/PUT/PATCHES. However, this means this fake client would have to deal with things like authentication/authorization/last-modified-since.
2. Use PyMongo to interact with the DB directly. However, this means I wouldn't have the data validation, or the metadata stored.

Does Eve have hooks for the database so that I can do Eve-rich database updates without HTTP?

Answer 1

As of v0.5 (currently on the development branch but you can pull and use it right away) you can use post_internal for adding data:

   Intended for internal post calls, this method is not rate limited,
   authentication is not checked and pre-request events are not raised.
   Adds one or more documents to a resource. Each document is validated
   against the domain schema. If validation passes the document is inserted
   and ID_FIELD, LAST_UPDATED and DATE_CREATED along with a link to the
   document are returned. If validation fails, a list of validation issues
   is returned.

It would probably make sense to add more internal methods to cover all CRUD operation which are now available via HTTP. You can still invoke them right away, though.

Update: v0.5 has been released with _internal methods available for all CRUD operations.

Answer 2

In case you wanted to do the following in a custom endpoint...

1. accept some POSTed data
2. manipulate the data in some way
3. perform a post_internal()
4. return the result as a response

...you'd do it something like this:

from eve.methods.post import post_internal
from eve.render import send_response

def my_custom_endpoint(**kwargs):

    data = json.loads(request.data.decode())

    # <manipulate data here>    

    resp = post_internal('crew', data)
    return send_response('crew', resp)

In reality you're probably better off to use Eve's Event Hooks to do this sort of thing. But in case there were some situation that Event Hooks didn't cover, this approach might be useful.

Answer 3

I was able to run this in a separate script that can be run periodically by jenkins. The app in run.py that I'm importing is the one I had by the end of the eve quickstart.

from run import app
from eve.methods.post import post_internal

payload = {
    "firstname": "Ray",
    "lastname": "LaMontagne",
    "role": ["contributor"]
}

with app.test_request_context():
    x = post_internal('people', payload)
    print(x)

post_internal runs eve.utils.parse_request, which relies on flask.request, so with app.test_request_context() is required. app.app_context() isn't sufficient for this method.

Read the docs for appcontext and reqcontext if you're new to flask (like me).