CacheService.putAll ignoring keys over a certain v

Question

I have a problem with putAll() method of google cache for script. The method seems working but only with first (~100) pairs

In my script i have to cache a big number of rows (~2500 * 3), each row in a different key, and I found malfunction in cache service. To identify the problem I wrote a simple code

function myFunction() {
  var cache = CacheService.getScriptCache();
  var toCache = {};
  for (var i = 0; i < 2000; i++){
   toCache["key"+i] = "value"+i;
  }
  cache.putAll(toCache);
  var tmp;
  for (var i = 0; i < 2000; i+=10){
   var a = cache.get("key"+i);
   if (a == null) { tmp = "key"+i; break; }
  }

  tmp = tmp;
}

As in the past the cache service seems to be unpredictable, and does not handle errors during processing. Putting cache with multiple calls is slower, and does not ensure that the limit will not change in the future, since it's not even mentioned in the documentation...

Thanks

Answer 1

How about this answer? I think that there might be several approaches. So please think of this as one of them.

In your situation, when i is 100, a returns null. By this, the for loop is finished. When the following script is used, it is found that there are the properties more than 100 in the CacheService.

var cache = CacheService.getScriptCache();
var toCache = {};
var keys = [];
for (var i = 0; i < 2000; i++) {
  var key = "key" + i;
  keys.push(key);
  toCache[key] = "value" + i;
}
cache.putAll(toCache);
var r = cache.getAll(keys);

Logger.log(keys.length) // 2000
Logger.log(Object.keys(r).length) // 900

But also it is found that although 2,000 properties are pushed, only 900 properties are retrieved. And I couldn't find the regularity of the lost properties. From this result, I thought that there might be the maximum number of properties which can be pushed.

Experiment:

As an experiment, I investigated the number of retrieved properties with the increase of the number of pushed properties. Each property was pushed at once. Following figure is the result.

From this figure, for above script, it is found that 900 properties is the maximum number of properties which can be pushed at once. This didn't depend on the size of values less than 100 kBytes. Also the following points could be obtained.

• When the properties from 1 to 900 are pushed, all properties can be correctly retrieved.
• When 1,000 properties are pushed, 1-900 properties are retrieved. Properties after 900 cannot be retrieved. It is considered that those properties are not pushed.
• When 2,000 properties are pushed, 900 properties are retrieved. But I couldn't find the regularity of them.
• After 900 properties were pushed using key names of "keyA1, keyA2, keyA3,,,", when 100 properties are pushed using key names of "keyB1, keyB2, keyB3,,,", 1,000 properties can be correctly retrieved.
  • But after 900 properties were pushed using key names of "keyA1, keyA2, keyA3,,,", when 101 properties are pushed using key names of "keyB1, keyB2, keyB3,,,", only 900 properties are retrieved. In this case, the properties of "keyA" were overwritten by 101 properties of "keyB".

These results were the same with CacheService.getScriptCache(), CacheService.getUserCache() and CacheService.getDocumentCache(). But each method of CacheService.getScriptCache(), CacheService.getUserCache() and CacheService.getDocumentCache() can be used as the individual storage. So when 3 methods are used, 3,000 properties can be used.

Summary:

1. Maximum number of properties which can be pushed once is 900.
2. When the properties are added 100 properties after 900 properties were pushed, 1,000 properties can be saved.
   • It is considered that this is the maximum number.
3. Methods of CacheService.getScriptCache(), CacheService.getUserCache() and CacheService.getDocumentCache() independent.
   • When CacheService.getScriptCache(), CacheService.getUserCache() and CacheService.getDocumentCache() are used, 3,000 properties can be used.

Workaround:

From above results, when you push the properties of "~2500 * 3", how about the following workaround?

• Put several values to one property.
  • The maximum size of a property is 100 kBytes.
• Use methods of CacheService.getScriptCache(), CacheService.getUserCache() and CacheService.getDocumentCache().
  • By this, 3,000 properties can be used.

I think that the properties of "~2500 * 3" can be pushed to CacheService by combination of these.

Note:

• When you use CacheService.getDocumentCache(), please use the container-bound script.
• In this report, I confirmed with my account. I'm not sure whether all results of other users are the same. I'm sorry for this situation.

Reference:

• CacheService

If above results has already been reported and/or this was not what you want, I'm sorry.