Server optimization for long running script

Question

I have a system running on a remote VPS whose main feature is a long running script (it could run for about 10 hours, depending on the user input).

The following are some of its features:

Database insertion and selection (MySQL)
HTTP Post (cURL)

The following are some of the VPS/system features:

Intel(R) Xeon(R) CPU E5-2430 0 @ 2.20GHz, 11 cores
Real memory of 1024 MB total
OpenVZ Network Interface
CentOS Linux 6.4
Apache version 2.2.15
PHP Version 5.3.3
MySQL version 5.1.69

Basically, the long running script should execute a lot of HTTP Post and Database selects and deletes (for a queue) per second. Each HTTP Post will send 50 kb out from my server to another uknown server, and each local Database operation will involve few bytes.

What do I need? I need a server that perform per SECOND, AT LEAST:

10 HTTP Post involving 50 kb (total: about 500 kbps)
10 selects and 10 deletes in the Database (total: about 20 simple and light operation per second [for a queue])

Now, my server is performing 20% of what I need, but I feel that I can get much more from it (since the script itself is already optimized). So I want some tips and good pratices about server configurations in order to optimize its performance, cause I am very noob on it. For example:

OS, Apache, PHP and MySQL configuration (cache, enable or disable certain features, CPU and memory limit, etc);
Disabling fool processes runing on CentOS by default;
Configuring properly the server network interface;
Increasing Server CPU cores or Memory or even changing the whole server features;
Updating MySQL to 5.6 version

I (and, of course, another noobs) would apreciate a lot if anyone could help.

Edit:

Just got it! (can't answer my onw question... check my comment bellow) :D

I don't have so much experience on servers, I just picked it up pre-configured and made some simple tests with lots of input data. I don't even know where to begin. The PHP script I am running is not mine, but it's known that it is pretty good for default servers (it uses thread). I think some current configuration is not allowing the script to run at its high performance, cause the max memory used was just about 25% and the CPU was 20%. — cawecoy, Jul 04 '13 at 20:30
How big is your database? What is the IO subsystem configuration? — toppledwagon, Jul 05 '13 at 00:36
Database queue table has between 100 000 and 500 000 rows with 4 fields each (very light rows). I am using InnoDB and the configuration is the default one. — cawecoy, Jul 05 '13 at 13:04
Well, I just did the following test: I started 2 copies of the same script and I got exactally the double of performance. Also the CPU usage went up to the double of before (10%) and the memory to 33%. Now I am sure that for some reason my VPS is limiting the resources for each running script. But I still dont know how to solve this :/ — cawecoy, Jul 05 '13 at 15:03

score 1 · Answer 1 · answered Jul 04 '13 at 20:25

1

I'm eager to say that your script is FAR from optimised by looking at the performance you get the and the server specs you gave. Even your typical smartphone should get much better results than the ones you tell us.

Is that "another unknown server" responding fast?
Does your script spawn only one request at a time, wait for the results, and only then perform the next request, or does it spawn multiple requests in parallel?
Is the database the bottleneck? Does it lack indexes? Does it have some big databases but running with the default settings? From your text it doesn't sound like it, but just making sure.

From the OS point of view, I guarantee you that the default CentOS is good enough for much better performance than the ones you gave us and disabling couple of background processes might give you something like 0.5-1.0% performance boost, if that, and no way 500% like you need. The default CentOS install is not starting with the handbrake on.

answered Jul 04 '13 at 20:25

Janne Pikkarainen

32,082

Actually the script was not written by me, but it's popular and reliable, as well as the uknow server (it's an Amazon's webservice). The script runs in parallel (using thread) and now it is using 5% of CPU and 18% of memory. So I believe it's about the CPU and Memory limit and Database configuration (it's default). – cawecoy Jul 04 '13 at 20:46
Could you give me some tips about it? Should I increase resources limits at PHP or Apache configuration? And how about the database, some tips? – cawecoy Jul 04 '13 at 20:48
I successfully got to set up a nice server to suit my needs. I've got a VPS that perform a better req/sec (Linode). I also increase the PHP memory_limit and increase some mysql limits conf. With all this settings now I get a so much better performance. – cawecoy Jul 09 '13 at 13:25

Server optimization for long running script

Edit:

1 Answers1

Linked