Implementing a strategy to programatically load ba

Most shared hosting is limited to 1GB Virtual Memory and that is not much.

I'm in a similar situation and my scripts are getting terminated by the server for resource limit errors and it's wreaking havoc on our App (almost unusable at certain times of the day).

This is caused by the server throwing errors (Resource Limit Reached).

On CPANEL error logs i get a whole bunch of:

    "Cannot allocate memory: couldn't create child process: ..."

Checking everything else, Bandwidth and Diskspace and even CPU use is very minimal, but it's that darn Memory limit that's the Achilles heel.

That made me realize that it's more important for me (and devs like me on limited servers) to design my app towards minimizing memory use, rather than speed, responsiveness, near-real time data, and other priorities.

So im looking at a bit of redesign.

Looking at my resource usage stats, I can see that while my App is indeed hitting over 1GB memory (up to 3GB even), this is certainly not constant and the memory spikes are spaced apart, and that the spikes are not caused by a single script or run, but rather caused by simultaneous small requests piling up (4-5 request within a 1-2 second span)

I can pretty much conclude that my App (a few simultaneous backend calls ) does not consume 1GB, and thus I do not need more memory on my server (which is much more expensive = VPS). What i need is to a better way to "spread" the load throughout the day (fill those 0% MEM usage gaps).

I have no control over the concurrent connections and simultaneous requests, they're just gonna keep coming (mostly from client side applications depending on my Server for data).

But if i just gracefully handle the requests and manage them myself I can make the ajax requests beyond limit (ie: 5 simultaneous requests) fall in line by throwing a flag to make the front end clients abandon the request and try again after a few seconds, like getting another priority number.

This way, it doesn't matter what time the GUI/front ends are fired up and when the intervals start ticking, the application will self heal or self organize and manage the intervals so they dont hit the server at the same time. it's additional work but I would rather terminate the calls myself INSTEAD OF THE SERVER THROWING RESOURCE LIMIT ERRORS!

I have several ideas already how to execute this "self load balancing act" but i'm looking for the best way.

Basically i just need a way to count the total number of calls currently running.. Basically i need to check-in/check-out calls..

Ill have a simple log keeping track of how many scripts are currently running.. that's it.

This would add additional read/write overhead at the beginning and end of each script, but i think it's a small price to pay.. I have plenty of CPU power left over..

So I wanted to ask the community any other pointers/tips for my situation? And more importantly, what is the LEAST MEMORY CONSUMING way to keep track of these millisecond executions? on file (disk) or a quick select and update SQL call on a small mySQL table? using a separate / faster database? although i think that would eat more memory. im already connecting and accessing mySQL so might as well reuse the connection. once again, i dont care about speed anymore at this point, slow and steady w/o memory errors, that's the goal.