How To Optmize An Apache2 Web Server For Speed
In an effort to serve our eCommerce customers better, I thought that providing them with a super fast website would be a good thing, so I used a bunch of tricks that I have learned over the years to speed things up a little bit once our site had been migrated for a while and the inevitable snags had been worked out.
First, Some Benchmarking
To investigate the raw speed of our un-optimized CentOS web server, I thought that I would install a range of test websites at my personal site so I could have some calibration information with which to evaluate the speed of our eCommerce website.
Here's the test URLs I prepared:
A) www.graham-leach.com/Apache2/HelloWorld - Loads a static text file that says "Hello World'.
B) www.graham-leach.com/Apache2/PHPHelloWorld - Executes a dynamic PHP script that says "Hello World!".
C) www.graham-leach.com/Apache2/GIFHelloWorld - Loads a static 1Mb animated GIF file that says "Hello World!"
D) www.holisticpethelp.com - Loads our eCommerce website featuring static text and images and also dynamic PHP and MySQL commands.
Initial Benchmarks - No memcached, USA Client
www.graham-leach.com/Apache2/HelloWorld: 2.260 seconds
www.graham-leach.com/Apache2/PHPHelloWorld: 2.177 seconds
www.graham-leach.com/Apache2/GIFHelloWorld: 7.139 seconds
www.holisticpethelp.com: 13.191 seconds
Initial Benchmarks - No memcached, Asia Client
www.graham-leach.com/Apache2/HelloWorld:
www.graham-leach.com/Apache2/PHPHelloWorld:
www.graham-leach.com/Apache2/GIFHelloWorld:
www.holisticpethelp.com:
Benchmark Analysis
What do the above numbers tell me? Well, it looks like the fastest page load I can expect with this configuration is in the 2 second range, which is pretty great! Unfortunately, the amount of utility offered by the web pages in that speed range are very limited.
Once the web server is forced to start pulling information off a hard disk, it dramatically slows down. If the file is a single 1Mb image, the web server starts to drag and performs only in the 7 second range, which is not that great.
But when complex web pages start to be delivered, as with the Drupal CMS, which may feature dozens of images and hundreds of SQL queries, performance drops off dramatically and pages start to load in the 11 second range, whch is terrible!
Target Benchmark
If 2 seconds is great (but useless) and 10 seconds is terrible (but useful), what time are we targeting for our eCommerce website, you may ask?
Well, according to www.webpagetest.org, the average load time target for websites in 2020 is about 5 seconds.
So it looks like we need to figure out a way to double our website speed so we can deliver web pages in about half the time. Hey, it looks like we have our work cut out for us...so let's begin!
Ways To Make A Website Faster
Here's some easy ways to make a web server feel faster:
A) Keep as much of the website content as possible in memory (fast) via some kind of caching strategy, so the web server doesn't have to pull the files off the disk (slow) when they are required.
B) Reduce the impact of images by compressing them when uploading to the website, so they are as small as possible in transmission from the web server to the web client.
C) Reducing the size of text files (like Cascading Style Sheet, or .css files) as much as possible, so they are as small as possible in transmission from the web server to the web client.
D) Reducing the amount of database lookups as much as possible
How To Speed Up Apache2 Using memcache
Introducing memcached
Due to an unfortunate naming convention, there's a great deal of confusion between memcached, which is a server technology, and memcache, which is a plug-in or client Application Programming Interface (API) that enables applications to avails of the memcache server technology.How To Install The memcached Server
# yum install memcached
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirror.qoxy.com
* centos-sclo-rh: mirror.qoxy.com
* centos-sclo-sclo: mirror.qoxy.com
* epel: my.fedora.ipserverone.com
* extras: mirror.qoxy.com
* remi-php54: mirrors.thzhost.com
* remi-php55: mirrors.thzhost.com
* remi-php56: mirrors.thzhost.com
* remi-php71: mirrors.thzhost.com
* remi-safe: mirrors.thzhost.com
* updates: mirror.qoxy.com
Resolving Dependencies
--> Running transaction check
---> Package memcached.x86_64 0:1.4.15-10.el7_3.1 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
============================================================================================================================================================================================================
Package Arch Version Repository Size
============================================================================================================================================================================================================
Installing:
memcached x86_64 1.4.15-10.el7_3.1 base 85 k
Transaction Summary
============================================================================================================================================================================================================
Install 1 Package
Total download size: 85 k
Installed size: 176 k
Is this ok [y/d/N]: y
Downloading packages:
memcached-1.4.15-10.el7_3.1.x86_64.rpm | 85 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : memcached-1.4.15-10.el7_3.1.x86_64 1/1
Verifying : memcached-1.4.15-10.el7_3.1.x86_64 1/1
Installed:
memcached.x86_64 0:1.4.15-10.el7_3.1
Complete!
#
How To Configure The memcached Server
The memcached configuration file is located at /etc/sysconfig/memcached.
Here's what mine looked like:
PORT="11211"
USER="memcached"
MAXCONN="1024"
CACHESIZE="64"
OPTIONS=""
From long experience, I knew that the CACHESIZE="64" parameter was likely going to prove too small, so I changed it to a more generous value:
CACHESIZE="512"
and then saved the changes.
How To Turn On The memcached Server
The memcached server can be started using the service <servicename> start command:
# service memcached start
Redirecting to /bin/systemctl start memcached.service
How To Check That The memcached Server Is Running
The service start commands don't always tell you a lot about the status of the service they refer to, so it's always a good idea to check to make sure that the memcached service has started properly, and is happy, using the service <servicename> status command:
# service memcached status
Redirecting to /bin/systemctl status memcached.service
● memcached.service - Memcached
Loaded: loaded (/usr/lib/systemd/system/memcached.service; disabled; vendor preset: disabled)
Active: active (running) since Sun 2019-12-15 17:17:36 HKT; 5s ago
Main PID: 15370 (memcached)
CGroup: /system.slice/memcached.service
└─15370 /usr/bin/memcached -u memcached -p 11211 -m 64 -c 1024
Dec 15 17:17:36 vm.yougrow.net systemd[1]: Started Memcached.
#
It's Smart To Periodically Restart memcached
Many years of experience tells us that memcached can get confused. When it does, it bogs down to the point where it fails to respond to incoming web page requests. In the past this has led to stressful hours where we could not identify the source of the problem, and customers could not log into our eCommerce website, which was a real problem. So, out of an abundance of caution, it makes sense to us to restart the memcached server every day at 6am. This is done by editing the system crontab:
# crontab -e
# GL 2019-12-15 memcached bogs Apache2 down if it is not restarted frequently
# Every day at 06h00
00 06 * * * /bin/systemctl restart memcached.service
How To Enable memcached Load At System Startup
Seeing as memcached is so useful and critical to the web server responding quickly to customers, it would be great to have it be automatically loaded whenever the system boots.
This can easily be done using the systemctl enable command:
# systemctl enable memcached.service
Created symlink from /etc/systemd/system/multi-user.target.wants/memcached.service to /usr/lib/systemd/system/memcached.service.
[root@vm ~]# service memcached status
Redirecting to /bin/systemctl status memcached.service
● memcached.service - Memcached
Loaded: loaded (/usr/lib/systemd/system/memcached.service; enabled; vendor preset: disabled)
Active: active (running) since Sun 2019-12-15 19:34:17 HKT; 2min 14s ago
Main PID: 24146 (memcached)
CGroup: /system.slice/memcached.service
└─24146 /usr/bin/memcached -u memcached -p 11211 -m 64 -c 1024
Dec 15 19:34:17 vm.yougrow.net systemd[1]: Started Memcached.
The memcached service will now start automatically at system boot.
How To Get memcached Statistics
Now that memcached is installed and enabled we can ask it to report on itself:
# telnet localhost 11211
Trying ::1...
Connected to localhost.
Escape character is '^]'.
stats <- you need to type this in
STAT pid 24146
STAT uptime 844
STAT time 1576410499
STAT version 1.4.15
STAT libevent 2.0.21-stable
STAT pointer_size 64 <- this setting is a concern
STAT rusage_user 0.010459
STAT rusage_system 0.065922
STAT curr_connections 10
STAT total_connections 11
STAT connection_structures 11
STAT reserved_fds 20
STAT cmd_get 0
STAT cmd_set 0
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 0
STAT get_misses 0
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 7
STAT bytes_written 0
STAT limit_maxbytes 67108864
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 524288
STAT hash_is_expanding 0
STAT bytes 0
STAT curr_items 0
STAT total_items 0
STAT expired_unfetched 0
STAT evicted_unfetched 0
STAT evictions 0
STAT reclaimed 0
END <- you need to type this in
quit
Connection closed by foreign host.
#
Trying ::1...
Connected to localhost.
Escape character is '^]'.
stats <- you need to type this in
STAT pid 24146
STAT uptime 844
STAT time 1576410499
STAT version 1.4.15
STAT libevent 2.0.21-stable
STAT pointer_size 64 <- this setting is a concern
STAT rusage_user 0.010459
STAT rusage_system 0.065922
STAT curr_connections 10
STAT total_connections 11
STAT connection_structures 11
STAT reserved_fds 20
STAT cmd_get 0
STAT cmd_set 0
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 0
STAT get_misses 0
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 7
STAT bytes_written 0
STAT limit_maxbytes 67108864
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 524288
STAT hash_is_expanding 0
STAT bytes 0
STAT curr_items 0
STAT total_items 0
STAT expired_unfetched 0
STAT evicted_unfetched 0
STAT evictions 0
STAT reclaimed 0
END <- you need to type this in
quit
Connection closed by foreign host.
#
How To Configure Apache2 To UseThe memcached Server
For Apache2 to take advantage of the memcached server, it must be configured to load the memcache module.
How To Use phpinfo To Check If Apache2 Has Loaded The memcache Module
# httpd -M | grep "memcache"
socache_memcache_module (shared)
How To Check That Apache2 Is Using memcached
Unfortunately, there's no really easy way to see if Apache2 is using the memcache module, even if it may have the module loaded. It can only be checked programmatically. Happily, I found this this code on www.stackoverflow.com that performs just such a check:
<?php
if (class_exists('Memcache')) {
$server = 'localhost';
if (!empty($_REQUEST['server'])) {
$server = $_REQUEST['server'];
}
$memcache = new Memcache;
$isMemcacheAvailable = @$memcache->connect($server);
if ($isMemcacheAvailable) {
$aData = $memcache->get('data');
echo '<pre>';
if ($aData) {
echo '<h2>Data from Cache:</h2>';
print_r($aData);
} else {
$aData = array(
'me' => 'you',
'us' => 'them',
);
echo '<h2>Fresh Data:</h2>';
print_r($aData);
$memcache->set('data', $aData, 0, 300);
}
$aData = $memcache->get('data');
if ($aData) {
echo '<h3>Memcache seems to be working fine!</h3>';
} else {
echo '<h3>Memcache DOES NOT seem to be working!</h3>';
}
echo '</pre>';
}
}
if (!$isMemcacheAvailable) {
echo 'Memcache not available';
}
?>On my system, even with the memcached server installed, the code produced this result:
Optimized Benchmarks (USA)
Seeing as the USA is on the other side of the planet, these can be considered "worst case" performance figures for our eCommerce website:www.graham-leach.com/Apache2/HelloWorld:
www.graham-leach.com/Apache2/PHPHelloWorld:
www.graham-leach.com/Apache2/GIFHelloWorld:
www.holisticpethelp.com: 8.803 seconds
Optimized Benchmarks (Asia)
But testing closer to home, and thereby eliminating network latency, got me these figures, which can be considered "best base" performance figures:
www.graham-leach.com/Apache2/HelloWorld: 0.569 seconds
www.graham-leach.com/Apache2/PHPHelloWorld: 0.474 seconds
www.graham-leach.com/Apache2/GIFHelloWorld: 2.764
www.holisticpethelp.com: 7.380 seconds
www.graham-leach.com/Apache2/PHPHelloWorld: 0.474 seconds
www.graham-leach.com/Apache2/GIFHelloWorld: 2.764
Properly Sizing memcached
Out of the box, memcached is sized a little small for our eCommerce website. We know this from running memcached on our old CentOS 5 server. Experience tells us that the correct cache size is at least 256Mb, perhaps even as much as 512Mb. Well, in this case it is best to err on the side of caution and give memcached a lot of room to do its thing, so we are going to set the cache to be 512Mb.
The configuration file for memcached is in /etc/sysconfig/memcached, and I made mine look like this:
PORT="11211"
USER="memcached"
MAXCONN="1024"
CACHESIZE="512"
OPTIONS=""
# service memcached restart
Redirecting to /bin/systemctl restart memcached.service
# service memcached status
Redirecting to /bin/systemctl status memcached.service
● memcached.service - Memcached
Loaded: loaded (/usr/lib/systemd/system/memcached.service; enabled; vendor preset: disabled)
Active: active (running) since Sun 2019-12-15 19:56:48 HKT; 7s ago
Main PID: 25404 (memcached)
CGroup: /system.slice/memcached.service
└─25404 /usr/bin/memcached -u memcached -p 11211 -m 512 -c 1024
Dec 15 19:56:48 vm.yougrow.net systemd[1]: Started Memcached.
# telnet localhost 11211
Trying ::1...
Connected to localhost.
Escape character is '^]'.
stats
STAT pid 25404
STAT uptime 24
STAT time 1576411030
STAT version 1.4.15
STAT libevent 2.0.21-stable
STAT pointer_size 64
STAT rusage_user 0.003262
STAT rusage_system 0.014138
STAT curr_connections 10
STAT total_connections 11
STAT connection_structures 11
STAT reserved_fds 20
STAT cmd_get 0
STAT cmd_set 0
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 0
STAT get_misses 0
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 7
STAT bytes_written 0
STAT limit_maxbytes 536870912
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 524288
STAT hash_is_expanding 0
STAT bytes 0
STAT curr_items 0
STAT total_items 0
STAT expired_unfetched 0
STAT evicted_unfetched 0
STAT evictions 0
STAT reclaimed 0
END
quit
Connection closed by foreign host.
#
How To Install & Configure PHPMemcachedAdmin
Installation:
As of the writing of this document, version 1.3 of PHPMemcachedAdmin can be downloaded from:
https://github.com/elijaa/phpmemcachedadmin/archive/master.zip
Installation was very straightforward. I went to the root of my web server and executed the following commands:
# wget https://github.com/elijaa/phpmemcachedadmin/archive/master.zip
--2019-12-15 20:32:51-- https://github.com/elijaa/phpmemcachedadmin/archive/master.zip
Resolving github.com (github.com)... 13.229.188.59
Connecting to github.com (github.com)|13.229.188.59|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/elijaa/phpmemcachedadmin/zip/master [following]
--2019-12-15 20:32:51-- https://codeload.github.com/elijaa/phpmemcachedadmin/zip/master
Resolving codeload.github.com (codeload.github.com)... 13.250.162.133
Connecting to codeload.github.com (codeload.github.com)|13.250.162.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/zip]
Saving to: ‘master.zip’
[ <=> ] 130,301 --.-K/s in 0.03s
2019-12-15 20:32:52 (4.12 MB/s) - ‘master.zip’ saved [130301]
#
I then unzipped the master-zip file:
# unzip master.zip
Archive: master.zip
f5b59eb46aa6e299ad24a03c8ab2fe041d38e16d
creating: phpmemcachedadmin-master/
extracting: phpmemcachedadmin-master/.gitignore
creating: phpmemcachedadmin-master/Config/
inflating: phpmemcachedadmin-master/Config/Memcache.sample.php
inflating: phpmemcachedadmin-master/LICENSE
creating: phpmemcachedadmin-master/Library/
inflating: phpmemcachedadmin-master/Library/Bootstrap.php
creating: phpmemcachedadmin-master/Library/Command/
inflating: phpmemcachedadmin-master/Library/Command/Factory.php
inflating: phpmemcachedadmin-master/Library/Command/Interface.php
inflating: phpmemcachedadmin-master/Library/Command/Memcache.php
inflating: phpmemcachedadmin-master/Library/Command/Memcached.php
inflating: phpmemcachedadmin-master/Library/Command/Server.php
creating: phpmemcachedadmin-master/Library/Configuration/
inflating: phpmemcachedadmin-master/Library/Configuration/Loader.php
creating: phpmemcachedadmin-master/Library/Data/
inflating: phpmemcachedadmin-master/Library/Data/Analysis.php
inflating: phpmemcachedadmin-master/Library/Data/Error.php
inflating: phpmemcachedadmin-master/Library/Data/Version.php
creating: phpmemcachedadmin-master/Library/HTML/
inflating: phpmemcachedadmin-master/Library/HTML/Components.php
creating: phpmemcachedadmin-master/Public/
creating: phpmemcachedadmin-master/Public/Scripts/
creating: phpmemcachedadmin-master/Public/Scripts/Highcharts/
inflating: phpmemcachedadmin-master/Public/Scripts/Highcharts/highcharts.js
inflating: phpmemcachedadmin-master/Public/Scripts/Highcharts/standalone-framework.js
inflating: phpmemcachedadmin-master/Public/Scripts/script.js
creating: phpmemcachedadmin-master/Public/Styles/
inflating: phpmemcachedadmin-master/Public/Styles/Style.css
inflating: phpmemcachedadmin-master/README.md
creating: phpmemcachedadmin-master/Temp/
extracting: phpmemcachedadmin-master/Temp/.version
creating: phpmemcachedadmin-master/View/
creating: phpmemcachedadmin-master/View/Commands/
inflating: phpmemcachedadmin-master/View/Commands/Commands.phtml
creating: phpmemcachedadmin-master/View/Configure/
inflating: phpmemcachedadmin-master/View/Configure/Configure.phtml
inflating: phpmemcachedadmin-master/View/Footer.phtml
inflating: phpmemcachedadmin-master/View/Header.phtml
creating: phpmemcachedadmin-master/View/LiveStats/
inflating: phpmemcachedadmin-master/View/LiveStats/Frame.phtml
inflating: phpmemcachedadmin-master/View/LiveStats/Stats.phtml
creating: phpmemcachedadmin-master/View/Stats/
inflating: phpmemcachedadmin-master/View/Stats/Error.phtml
inflating: phpmemcachedadmin-master/View/Stats/Items.phtml
inflating: phpmemcachedadmin-master/View/Stats/Slabs.phtml
inflating: phpmemcachedadmin-master/View/Stats/Stats.phtml
inflating: phpmemcachedadmin-master/commands.php
inflating: phpmemcachedadmin-master/composer.json
inflating: phpmemcachedadmin-master/configure.php
creating: phpmemcachedadmin-master/docker/
inflating: phpmemcachedadmin-master/docker/Memcache.docker.php
inflating: phpmemcachedadmin-master/docker/README.md
inflating: phpmemcachedadmin-master/docker/docker-compose.yml
creating: phpmemcachedadmin-master/docker/phpmemcachedadmin/
inflating: phpmemcachedadmin-master/docker/phpmemcachedadmin/Dockerfile
inflating: phpmemcachedadmin-master/docker/phpmemcachedadmin/build.sh
extracting: phpmemcachedadmin-master/docker/phpmemcachedadmin/start.sh
inflating: phpmemcachedadmin-master/index.php
inflating: phpmemcachedadmin-master/spam.php
inflating: phpmemcachedadmin-master/stats.php
#
I then renamed the folder to memcached, and entered it:
# mv phpmemcachedadmin-master memcached
# cd memcached
Everything looked ok, so I had a peek at it using the web interface:
Hmmmm......seems there's a couple of things to look into before we are done
How To Fix The PHPMemcachedAdmin Temp Directory Problem
To resolve this error:Warning : Temporary directory 'Temp/' is not writable, please fix this error and try again.
The Temp folder needs to be:
- Owned by apache:apache.
This is accomplished using the following command(s):
# chown apache:apache Temp
How To Fix the PHPMemcachedAdmin Configuration File Problem
To resolve this error:
Error : Configuration file or folder is missing, please fix this error and try again.
The Memcache.php.sample file in the Config directory needs to be:
- Copied to Memcached.php
- Owned by apache:apache
- Set to SELINUX Read/Write mode
This is accomplished using the following command(s):
# cd Config
# cp Memcached.sample.php Memcached.php
# chown apache:apache Memcached.php
# chcon unconfined_u:object_r:httpd_sys_rw_content_t:s0 Memcache.php
As you can see, the size of the memory pool is 512Mb.
BUT THERE'S A PROBLEM
The statistics all read 0, meaning that our Drupal eCommerce website isn't taking advantage of the memcached server at all!
How To Integrate memcached with Drupal
Install PECL memcache
Install Drupal memcache module
https://ftp.drupal.org/files/projects/memcache-7.x-1.6.tar.gz
Change settings.php
Turn the Drupal memcache Module on:
After a little while, see what PHPMemcachedAdmin reports:
Conclusion
Getting Drupal to use memcached was fun, but it's hardly the end of the optimization journey...not by a long shot!
I found more optimization tips can be found at analyze.websiteoptimization.com/wso, which I might look into very soon!
P.S. After watching the server for a couple of days, I reduced the cache size to 128Mb
P.P.S. After watching the server for a week, I reduced the cache size to 80Mb
REFERENCES:
https://tecadmin.net/install-memcached-with-pecl-memcache-on-centos-rhel/https://unix.stackexchange.com/questions/50639/httpd-cant-write-to-folder-file-because-of-selinux
https://devops.ionos.com/tutorials/install-and-configure-memcached-on-centos-7/
https://tecadmin.net/install-memcached-with-php-on-ubuntu/
https://stackoverflow.com/questions/1463670/how-to-check-if-memcache-or-memcached-is-installed-for-php
https://serverpilot.io/docs/how-to-install-the-php-memcache-extension
http://analyze.websiteoptimization.com/wso
No comments:
Post a Comment