Freemarker Template Cache are in same content when

Question

When I debug the free marker template cache, I noticed that for the same FTL source file, the freemarker template cache object has content, locale, encoding properties. When this cache value was pushed into cache with TemplateKey, the Key includes file name, locale, encoding, parse etc. The problem is: For same FTL file,if it has different locale, same file content will be cached multiple times in memory (per different locale, encoding etc). For a large web site which support all locales, this is big waste of memory.

It is a very big memory issue. Here we are talking about 30 times duplicated memory cache in Freemarker Template Cache.

I may be wrong because I may not touch other scenarios which in the case of same FTL source file will be cached as different content when locale/encoding are different. Please let me know if this case is existing.

I would strong recommend FreeMarker designer to thinking about this issue and see if any way to improve the memory usage.

Thanks.

Rocky

Answer 1

Yes, if you pass different locales in while you don't actually have separate template versions for those locales (like foo_en.ftl, foo_fr.ftl, foo_de.ftl), it wastes resources. This is admittedly something that should be improved (I'm the main maintainer nowadays). Note that if your app doesn't actually support 30 locales, don't just pass the visitor locale to getTemplate or something like that, if that's what's going on. This mechanism is only meant to be used when you actually have different templates for the different locales, which of course really have to be cached separately. If you don't have different templates for different locales, then you should just set the locale in the Environment (if at all). (Also note that if you have a sufficiently low cache size limit then the issue rather means less efficient caching than high memory usage.)

Regarding how to fix this... that's not really a SO topic, but whatever...

So as it works, the same template file loaded with different locales will result in different Template objects, and thus they are all loaded(!) and cached separately. Those Template objects will be the same in content (assuming you don't have different files for each locale, of course), except that their "locale" setting will be different. Sadly, I can't just reuse the parsed content of a Template-s to create an identical one with different "locale" setting, because the elements inside a Template are aware of the Template object (they point back to it). So, I believe, the solution will be that the locale of a Template-s won't depend on the locale requested, but on the locale of the template file according the locale code at the end of its name. This behavior could be a Configuration-level setting (for backward compatibility). A new cache entry is still needed for each requested locale, because if the template lookup usually means I/O, so its outcome need to be cached.

Answer 2

Thanks for your reply.

Another point point is that we set lcoalelookup=false, so we are not using file like foo_en.ftl, foo_fr.ft, foo_de.ftl. In this case, one template file has same content whatever what locale is. We are not able to change the design at this point. (By the way, templatesource was cached, so it should not have IO issue)

So far I see if we want to reduce the template cache memory, one solution is in take over Cache manage, when get cache.put(key, value), we can transform the key and remove "locale" if localelookup = false. When Cache.get(key), before return value, make a cloned value and update locale in new template object. (need in cachetemplate, template level). The problem is the key class (TemplateKey, CachedTemplate) are private, and Template does not extend clonable. Is any chance to make next release patch to support the customer cache key transformation ? All the change in freemarker is to expose the classes, but not function was impacted.

Actually the better solution is: inside freemarker, if it has different keys, such like TemplateKey, TemplateKeyWithOutLocal. When create key object, it can use either TemplateKey or TemplateKeyWIthoutLocale per localelookup setting. When return cache template object, make template clone and set locale correct.

Thanks.