page-cache | lightweight PHP library for full page cache | Caching library

I am trying to improve the page speed of a site I run by serving static content directly from NGINX rather than the request hitting PHP.

I have webpages at paths like this:

• gamea.com.mysite.com
• anotherb.net.mysite.com
• finalc.org.mysite.com

When a page is generated for these, it's stored in a path like this:

• /storage/app/page-cache/games/game/gamea_com/1.c
• /storage/app/page-cache/games/anot/anotherb_net/1.c
• /storage/app/page-cache/games/fina/finalc_org/1.c

The path structure takes the first 4 letters of the subdomain, and then follows by another folder with the full path and replaces "." with "_" - e.g. "gamea.com" = "/game/gamea_com/". The actual cache page file is stored as "1.c"

How might this be accomplished through NGINX? I'm a bit stuck, I did find this article but I'm unsure how to use it in my case - can anyone provide an example NGINX config that points NGINX to the correct path as described above?

I thank you whoever can help me with this!

Today a system exists that will write packet-capture files to the local disk as they come in. Dropping these files to local disk as the first step is deemed desirable for fault-tolerance reasons. If a client dies and needs to reconnect or be brought up somewhere else, we enjoy the ability to replay from the disk.

The next step in the data pipeline is trying to get this data that was landed to disk out to remote clients. Assuming sufficient disk space, it strikes me as very convenient to use the local disk (and the page-cache on top of it) as a persistent boundless-FIFO. It is also desirable to use the file system to keep the coupling between the producer and consumer low.

In my research, I have not found a lot of guidance on this type of architecture. More specifically, I have not seen well-established patterns in popular open-source libraries/frameworks for reading the file as it is being written to stream out.

My questions:

1. Is there a flaw in this architecture that I am not noting or indirectly downplaying?

2. Are there recommendations for consuming a file as it is being written, and efficiently blocking and/or asynchronously being notified when more data is available in the file?

3. A goal would be to either explicitly or implicitly have the consumer benefit from page-cache warmth. Are there any recommendations on how to optimize for this?

Im trying to index PDF text to a python lib called Scout. I have tried doing the same thing with elasticsearch too. In both cases I can't figure out how to post text to an index in bulk, using python.

After a lot of research, I believe I need to use async http request. The only problem is, I don't understand async calls nor do I understand what a Scout python 'client' really is. I'm a self-taught programmer and still have many things I don't understand. my thought is the client cant stay open for a loop to keep using the connection. I have seen coding concepts like "await" and "sessions" in many books on programming. However, I don't know how to implement these concepts. Can someone help me write some python code that will successfully post new documents to a running scout server and explain how it's done?

Here is My attempt:

I've already asked this question on cs.stackexchange.com, but decided to post it here as well.

I've read several blogs and questions on stack exchange, but I'm unable to grasp what the real drawbacks of memory mapped files are. I see the following are frequently listed:

1. You can't memory map large files (>4GB) with a 32-bit address space. This makes sense to me now.

2. One drawback that I thought of was that if too many files are memory mapped, this can cause lower available system resources (memory) => can cause pages to be evicted => potentially more page faults. So some prudence is required in deciding what files to memory map and their access patterns.

3. Overhead of kernel mappings and data structures - according to Linus Torvalds. I won't even attempt to question this premise, because I don't know much about the internals of Linux kernel. :)

4. If the application is trying to read from a part of the file that is not loaded in the page cache, it (the application) will incur a penalty in the form of a page-fault, which in turn means increased I/O latency for the operation.

QUESTION #1: Isn't this the case for a standard file I/O operation as well? If an application tries to read from a part of a file that is not yet cached, it will result in a syscall that will cause the kernel to load the relevant page/block from the device. And on top of that, the page needs to be copied back to the user-space buffer.

Is the concern here that page-faults are somehow more expensive than syscalls in general - my interpretation of what Linus Torvalds says here? Is it because page-faults are blocking => the thread is not scheduled off the CPU => we are wasting precious time? Or is there something I'm missing here?

1. No support for async I/O for memory mapped files.

QUESTION #2: Is there an architectural limitation with supporting async I/O for memory mapped files, or is it just that it no one got around to doing it?

QUESTION #3: Vaguely related, but my interpretation of this article is that the kernel can read-ahead for standard I/O (even without fadvise()) but does not read-ahead for memory mapped files (unless issued an advisory with madvice()). Is this accurate? If this statement is in-fact true, is that why syscalls for standard I/O maybe faster, as opposed to a memory mapped file which will almost always cause a page-fault?

I have a Django project that I am using with memcached and Docker. When I use sudo docker-compose up in development I'd like the entire cache to be cleared. Rather than disabling caching wholesale while in development mode, is there a way to run cache.clear() as noted in this question on each re-run of sudo docker-compose up?

I am not sure whether this should go in:

1. docker-entrypoint.sh
2. Dockerfile
3. docker-compose.yml
4. Somewhere else?

I have problem with getting attribute value in new_grid.phtml. If i make it like this:

I want /vendor/* to be ignored except /vendor/magento/module-page-cache/.

Based on this question: .gitignore exclude folder but include specific subfolder

I came up with following gitignore