Python - Flask API service - Loop through API data

2019-07-25 11:55发布

I'm trying to develop an API service that after collecting data continuously from different sources, elaborate that data, and keep that information ready from being requested from another service.

I'm using Flask and it's working on top of a single thread.

My problem is about the slowlines accessing the cached data. The gathering process it works pretty well, but sometimes when I ask for the data the server it's busy because in the meantime it's fetching the data from the sources.

How should I set up the thread to avoid this?

Here down I'm explain what I've done thru some simplified functions and classes.

The Fetching Service

class FetchingService():
    def __init__(self,url=None,speed=None):
        self.url = url 
        self.speed = speed # speed in seconds 
        self.last_updated = None # last time the service was used

    def loadData(self):
        time = { 'timestamp': int(time.time()) }
        try:
            r = requests.get(self.url) # get the data from the url
            data = r.json() # if is a json
            data.update({ 'status': True })
            data.update(time)
            return data
        except:
            data = { 'status': False }
            data.update(time)
            return data

    def isTimeToUpdate(self):
        if self.last_updated is not None: # check if is the time to run an update
            delta = int(time.time()) - self.last_updated
            if delta >= self.speed:  # if passed enougth time 
                return True # is time to update
            else:
                return False # too early
        else: 
            return False # last_updated not set

    def run(self):
        if self.last_updated is None:
            self.last_udated = int(time.time())
        if self.isTimeToUpdate():
            self.loadData()

The Caching Service

class Caching():
    def __init__(self,instances):
        self.data = []
        self.instances = instances

    def run(self):
        if self.instances:
            for i in instances:
                self.data.append(i.run())

    def queryLast(self):
        if self.data:
            return self.data[-1]

The Thread

class myThread():
    def __init__ (self,thread_time,instances):
        self.speed = thread_time 
        self.data = None 
        self.instances = instances 
        self.dataLock = threading.Lock()
        self.myThread = threading.Thread()

    def primaryThread(self):
        self.myThread = threading.Timer(self.speed, self.run, ())
        self.myThread.start() 

    def run(self):
        with self.dataLock:
            self.data.run()
        self.primaryThread()

    def activate(self):
        with self.dataLock:

            self.data = Caching(self.instances)

        self.primaryThread()
        return 'active'

    def queryLast(self,key,value):
        with self.dataLock:
            data = self.data.queryLast()
        self.primaryThread()
        return data 

The Main

a = FetchingService(url='/api/test/',10)
b = FetchingService(url='/api/data/',5)
c = FetchingService(url='/boh/',100)
instances = [ a , b , c ]
t = myThread(1,instances)

The Flask views

@application.route('/active/', methods=['GET'])
def app_start():
    result = t.activate()
    return jsonify(result)

@application.route('/last/', methods=['GET'])
def app_query():
    result = t.queryLast()
    return jsonify(result)

0条回答
登录 后发表回答