AppEngine上:MEMCACHE下5秒失踪同一呼叫?(Appengine: Memcache

2019-11-03 06:14发布

这是我的代码的memcache获取和设置(一切都指向_mc_get)

def _mc_get(key, retrieve_func):
  value = memcache.get(key)
  if value is not None:
    logging.debug("Memcache: hit %s", key)
    return value

  logging.debug("Memcache: missing + adding %s", key)
  value = retrieve_func()
  if not memcache.add(key, value, 10):
    logging.error('Memcache set failed.')
  return value


def mc_get_all_orders():
  return _mc_get('Order:all', Order.query().fetch)


def mc_get_tasks_by_order(order_key):
  return _mc_get('Order:all', Task.query(Task.order == order_key).order(Task.action).fetch)


def mc_get_tasks_by_orders(order_keys):
  return _mc_get('Order:order_id:%s' % sorted([k.id() for k in order_keys]),
                 Task.query(Task.order.IN(order_keys)).order(Task.action).fetch)


def mc_get_all_producetypes():
  return _mc_get('ProduceType:all', ProduceType.query().fetch)

此功能正在AppEngine上> 40秒。

@app.route('/orders/schedule/all/', methods=['GET', 'POST'])
@login_required
def generate_schedule_for_all_orders():
  """Merge all the orders together in one big schedule

  TODO: put range by default.
  TODO: think of default empty scenario
  """
  orders = key_dict(mc_get_all_orders())

  # TODO: somehow there's a lot of orphaned tasks in development. For now,
  # we will just query for non orphaned tasks to prevent errors piling up
  tasks = mc_get_tasks_by_orders(orders.keys())

  ds = sorted([t.date for t in tasks])
  start, end = ds[0], ds[-1]
  dates = range_for_dates(start, end)

  return render_template('backward_schedule.html',
                         tasks=tasks,
                         orders=orders,
                         producetypes=key_dict(mc_get_all_producetypes()),
                         dates=dates,
                         display_order_info=True)

我想知道为什么花了这么长时间,所以我插在我的_mc_get logging.debug报表,看看我被击中或丢失内存缓存。 令我惊讶,我缺少的memcache 100%,如果我等待更长的时间超过3秒(这适用于我的开发和生产服务器)

日志首次访问页面: 注意时间

DEBUG    2014-06-11 23:16:11,765 mc.py:27] Memcache: missing + adding Order:all
DEBUG    2014-06-11 23:16:11,808 mc.py:27] Memcache: missing + adding Order:order_id:[4578366418059264L, 4741094138970112L, 5260063627280384L, 5664683906301952L, 5673479999324160L, 5761440929546240L]
DEBUG    2014-06-11 23:16:12,255 mc.py:27] Memcache: missing + adding ProduceType:all
INFO     2014-06-11 23:16:13,517 recording.py:665] Saved; key: __appstats__:071700, part: 105 bytes, full: 64362 bytes, overhead: 0.002 + 0.010; link: http://localhost:8080/_ah/stats/details?time=1402528571756

页面第二接入(作为第一个访问加载完成后尽快重新加载)

INFO     2014-06-11 23:16:13,534 module.py:639] default: "GET /orders/schedule/all/ HTTP/1.1" 200 1236559
DEBUG    2014-06-11 23:16:15,638 mc.py:24] Memcache: hit Order:all
DEBUG    2014-06-11 23:16:15,699 mc.py:24] Memcache: hit Order:order_id:[4578366418059264L, 4741094138970112L, 5260063627280384L, 5664683906301952L, 5673479999324160L, 5761440929546240L]
DEBUG    2014-06-11 23:16:15,706 mc.py:24] Memcache: hit ProduceType:all
INFO     2014-06-11 23:16:16,937 recording.py:665] Saved; key: __appstats__:075600, part: 56 bytes, full: 15643 bytes, overhead: 0.000 + 0.004; link: http://localhost:8080/_ah/stats/details?time=1402528575619

第三接入到同一个页面(只是击中刷新):(这是字面上第二接入到同一个页面后4秒)

INFO     2014-06-11 23:16:17,112 module.py:639] default: "GET /orders/schedule/all/ HTTP/1.1" 200 1236559
DEBUG    2014-06-11 23:16:23,362 mc.py:27] Memcache: missing + adding Order:all
DEBUG    2014-06-11 23:16:23,415 mc.py:27] Memcache: missing + adding Order:order_id:[4578366418059264L, 4741094138970112L, 5260063627280384L, 5664683906301952L, 5673479999324160L, 5761440929546240L]
DEBUG    2014-06-11 23:16:23,888 mc.py:27] Memcache: missing + adding ProduceType:all
INFO     2014-06-11 23:16:25,153 recording.py:665] Saved; key: __appstats__:083300, part: 105 bytes, full: 64339 bytes, overhead: 0.002 + 0.011; link: http://localhost:8080/_ah/stats/details?time=1402528583353
INFO     2014-06-11 23:16:25,169 module.py:639] default: "GET /orders/schedule/all/ HTTP/1.1" 200 1236559

为什么我的内存缓存到期如此迅速?

Answer 1:

  1. 有对内存缓存条目没有保证,理论上可以随时驱逐由于内存压力。

  2. 您可以使用memcache.add只设置的值,如果它不存在。 您的超时时间为10秒,在23点16分13秒加入的内存缓存,当你在23时十六分23秒访问它,它都将丢失。 这是意料之中的,因为它是围绕超过10秒。 它永远不会打印,如果你觉得值,函数立即返回的内存缓存补充,因为失败。

  3. 这是有点奇怪,无论mc_get_all_orders和mc_get_tasks_by_order使用相同的密钥,但不同的查询



文章来源: Appengine: Memcache missing under 5 seconds for the same call??