[ Limiting the size of memcache storage ]
I am using the FUSE file system fuse.py and plan to use memcache to store the files. I want to have a small amount of memcache space so that I know what files are being stored and what files are being replaced.
I understand that Memcache has slabs which stores data of that particular size. But is it possible to have only one slab? For example have one slab of 5mb, with each page size 1mb so that I can store maximum of 5 files. And if I need to add the 6th one the LRU policy will remove a file and make space for it.
Basically I want to use memcache as a cache for files and once a file is removed from memcache, store the change in db server.
Answer 1
Don't make assumptions on the implementation of a system based on how this or other systems might be implemented; the only thing you can rely on is what's in the published API documentation, and in this case, Google App Engine does not provide a callback when entries expire (which will either expire when the entry reaches its TTL or is replaced due to LRU), nor does it back them up to other storage systems.
You shouldn't use memcache for durable storage such as a file system; please consider one of the following systems instead for durable storage:
- Google Cloud Bigtable - wide-column store (HBase is based on Bigtable)
- Google Cloud Datastore - document-oriented storage
- Google Cloud SQL - SQL database, as the name implies
- Google Cloud Storage - blob/object storage
You can, of course, cache their data in Memcache, but since it can expire, you should store the originals in a more durable storage system, and cache the frequently-accessed chunks/files/metadata in Memcache.
